AFM-Based Observation and Robotic Nano-manipulation [1st ed. 2020] 9811505071, 9789811505072

Citation preview

Shuai Yuan · Lianqing Liu · Zhidong Wang · Ning Xi

AFM-Based Observation and Robotic Nano-manipulation

AFM-Based Observation and Robotic Nano-manipulation

Shuai Yuan Lianqing Liu Zhidong Wang Ning Xi •

•

•

AFM-Based Observation and Robotic Nano-manipulation

123

Shuai Yuan Shenyang Jianzhu University Shenyang, Liaoning, China Zhidong Wang Department of Advance Robotics Chiba Institute of Technology Chiba, Japan

Lianqing Liu Shenyang Institute of Automation, Chinese Academy of Sciences (CAS) Shenyang, Liaoning, China Ning Xi University of Hong Kong Hong Kong, China

ISBN 978-981-15-0507-2 ISBN 978-981-15-0508-9 https://doi.org/10.1007/978-981-15-0508-9

(eBook)

Jointly published with Science Press The print edition is not for sale in China Mainland. Customers from China Mainland please order the print book from: Science Press. © Science Press and Springer Nature Singapore Pte Ltd. 2020 This work is subject to copyright. All rights are reserved by the Publishers, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publishers, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publishers nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publishers remain neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

The ultimate goal of nanotechnology is to fabricate novel materials, devices and systems at atomic and molecular scale. Nano-observation and manipulation are critical technology for implementing nanotechnology goal. Nanotechnology not only opens up new frontiers for fundamental scientific research, but also provides broad prospects for the development of applied technology. This is mainly reflected in two aspects: on the one hand, discovering new physical, chemical and biological phenomenon, exploring new nanomaterials and verifying or establishing new models and theories at nanoscale; on the other hand, making use of these nanomaterials to produce devices with novel physical or chemical properties. Nano-observation, nano-assembly and nano-fabrication are important prerequisites and key technologies for developing nano-science research, discovering new characteristics of nanoscale things and implementing manufacture. Therefore, it has become a research orientation for scientists from different fields, which developed several kinds of nano-manipulation methods based on different mechanisms. These methods include self-assembly-based nano-manipulation, optical tweezer-based nano-manipulation, dielectrophoresis-based nano-manipulation, SEM-based nanomanipulation and AFM-based nano-manipulation. Compared with SEM, TEM, DEP, optically induced DEP and other maneuvering tools, AFM has attracted wide attention for its high motion accuracy, controllable and repeatable manipulation mode and unique mechanical mechanism. It overcomes the shortcomings of the above-mentioned manipulation methods in assembling nano-electronic devices and performs high precision nano-manipulation on the foundation of high-resolution observation. Therefore, it is the most potential nano-manipulation tool at present. This book aims at the problem of tip localization errors and manipulation instability caused by various uncertainties such as system nonlinearity, system thermal drift and tip broaden effect in AFM nano-manipulation environment. Based on the correction of observation image, accurate tip localization of combining stochastic idea with landmark observation is adopted to plan manipulating

v

vi

Preface

trajectory and develop virtual nano-hand-based nano-manipulation technology. Tip model is established, and landmark is actively configured for improving the accuracy of tip localization, which solves the instability problem in single-tip manipulation. In this book, Chap. 1 is an introduction, which presents the core conception of nanotechnology, that is, the basic methods of nano-observation and nanomanipulation, and analyzes the application characteristics and key problems of AFM-based nano-manipulation. Chapter 2 introduces the development of AFM robotic nano-manipulation. Chapter 3 discusses the reconstruction of AFM scanning image based on thermal drift compensation. In Chap. 4, on the basis of thermal drift image compensation, blind tip modeling algorithm is used to estimate the tip morphology and reconstruct the scanning image to improve the image precision. In Chap. 5, a landmark observation method is proposed to reduce the tip position uncertainty due to thermal drift and nonlinearity. Chapter 6 furtherly studies strategy of the tip path planning in the complex environment. Chapter 7 is to design an AFM-based nano-manipulation platform on the foundation of tip localization and virtual nano-hand, which can illustrate the effectiveness of the proposed method. In this book, the calibration scheme of model parameters is designed in the system implementation, and a lot of related experimental research and validation work is carried out. The experimental result shows that the proposed theory and method can improve the efficiency, reliability and stability of AFM-based nano-manipulation, which lays a foundation for further application of robotic nano-manipulation. Shenyang, China Shenyang, China Chiba, Japan Hong Kong, China July 2019

Shuai Yuan Lianqing Liu Zhidong Wang Ning Xi

Acknowledgements

The authors express their gratitude to Associate Professor Jing Hou, Professor Fangjun Luan, the graduate student Tianshu Chu, Likai Liu and Lixue Yin who assisted the authors in compiling the manuscripts. The authors are also grateful to the National Natural Science Foundation of China (Approval number: 61305125, 91748212, U1613220), the national doctoral funding and special assistance projects (Approval number: 2013M530955, 2014T70265) for their support. July 2019

Shuai Yuan Lianqing Liu Zhidong Wang Ning Xi

vii

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Introduction of Nanotechnology . . . . . . . . . . . . . . . . . . 1.1.1 Development and Application of Nanotechnology 1.1.2 Characteristics of Nanotechnology . . . . . . . . . . . 1.1.3 The Key Nanotechnology: Nano-observation and Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Primary Nano-observation Methods . . . . . . . . . . . . . . . . 1.2.1 Optical Microscopic Observation . . . . . . . . . . . . 1.2.2 SEM/TEM Based Observation . . . . . . . . . . . . . . 1.2.3 STM Based Observation . . . . . . . . . . . . . . . . . . 1.2.4 AFM Based Observation . . . . . . . . . . . . . . . . . . 1.3 Primary Nano-manipulation Methods . . . . . . . . . . . . . . . 1.3.1 Self-assembly Based Nano-manipulation . . . . . . . 1.3.2 Optical Tweezer Based Nano-manipulation . . . . . 1.3.3 DEP Based Nano-manipulation . . . . . . . . . . . . . . 1.3.4 SEM Based Nano-manipulation . . . . . . . . . . . . . 1.3.5 AFM Based Nano-manipulation . . . . . . . . . . . . . 1.4 Application Characteristics and Problems of AFM Based Nano-manipulation . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

1 1 2 7

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

8 9 9 9 11 12 17 17 17 19 20 21

....... .......

23 26

2 AFM Based Robotic Nano-manipulation . . . . . . . . . . . . . . . . . . . 2.1 AFM Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Analysis of AFM Atomic Force-Distance Curve . . . . . 2.1.2 Three Work Modes of AFM . . . . . . . . . . . . . . . . . . . . 2.2 AFM Based Robotic Nano-manipulation . . . . . . . . . . . . . . . . 2.2.1 Static Image Based Offline Nano-manipulation . . . . . . 2.2.2 Augmented Reality Based Robotic Nano-manipulation . 2.2.3 Local Scan Based Nano-manipulation Using Landmark Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

33 33 34 35 36 36 38

...

40

. . . . . . .

. . . . . . .

ix

x

Contents

2.3 Stochastic Approach for AFM Based Robotic Nano-manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Precision Analysis of AFM Tip Driver . . . . . . . . . . 2.3.2 Real-Time Tip Localization Analysis in Task Space 2.3.3 AFM Based Nano-manipulation Using Virtual Nano-hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

..... ..... .....

42 43 43

..... .....

45 46

3 AFM Image Reconstruction Using Compensation Model of Thermal Drift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Reconstruction Theory of AFM Thermal-Drift Image . . . . . . 3.1.1 Newton Iteration Method . . . . . . . . . . . . . . . . . . . . . 3.1.2 Image Interpolation Method . . . . . . . . . . . . . . . . . . . 3.1.3 Thermal Drift Correction Method for Scanning Image 3.2 Reconstruction Method for Thermal Drift Image . . . . . . . . . 3.2.1 Compensation Model for Thermal Drift . . . . . . . . . . 3.2.2 Thermal Drift Offset Vector . . . . . . . . . . . . . . . . . . . 3.2.3 Offset Vector Calculation . . . . . . . . . . . . . . . . . . . . . 3.2.4 Integral Image Reconstruction . . . . . . . . . . . . . . . . . 3.3 Simulation and Experimental Analysis . . . . . . . . . . . . . . . . . 3.3.1 Simulation and Analysis of Thermal Drift Image . . . . 3.3.2 Experiment and Analysis of Reconstruction of Thermal Drift Image . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

49 49 50 51 59 61 61 63 66 70 72 72

.... ....

73 81

. . . . . . . . . . . .

4 AFM Image Reconstruction Algorithm Based on Tip Model . . . 4.1 Theoretical Basis of AFM Tip Blind Modeling Reconstruction 4.1.1 Basic Concepts of Mathematical Morphology . . . . . . . 4.1.2 Mathematical Description of Tip Imaging Process . . . . 4.1.3 Tip Morphology Estimation Algorithm . . . . . . . . . . . . 4.2 A Method for Improving the Speed of Tip Modeling Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Pre-estimation of Tip Morphology . . . . . . . . . . . . . . . 4.2.2 Improvement of Algorithm Core . . . . . . . . . . . . . . . . . 4.3 Method for Improving the Accuracy of Tip Modeling . . . . . . 4.3.1 Definition of Denoising Threshold . . . . . . . . . . . . . . . 4.3.2 Estimation of Denoising Threshold . . . . . . . . . . . . . . . 4.4 The Experiment of AFM Image Reconstruction . . . . . . . . . . . 4.4.1 Tip Topography Estimation . . . . . . . . . . . . . . . . . . . . 4.4.2 Scanning Image Reconstruction of Carbon Nano-tubes and Nano-particles . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

83 83 83 86 88

. . . . .

. . . . .

. . . . .

. . . . . . . .

. . . . . . . .

. 92 . 92 . 95 . 99 . 99 . 101 . 102 . 102

. . . 104 . . . 106

Contents

xi

5 Stochastic Approach Based AFM Tip Localization . . . . . . . . . . 5.1 Research of AFM Tip Localization . . . . . . . . . . . . . . . . . . . 5.1.1 Stochastic Approach Based AFM Tip Localization Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Nano-manipulation Coordinate System Defined on AFM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Analysis of Landmark Observation Model Based on Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Landmark Definition . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Analysis of Landmark Observation . . . . . . . . . . . . . . 5.2.3 Analysis of Horizontal Observation of Landmark . . . 5.2.4 Optimal Estimation of Tip Position Based on Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Establishment of Tip Motion Model . . . . . . . . . . . . . . . . . . 5.3.1 PI Based Motion Model . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Creep Model of PZT . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 System Thermal Drift Model . . . . . . . . . . . . . . . . . . 5.4 Simulation Experiment of Tip Localization Based on Landmark Observation . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . 107 . . . . 107 . . . . 107 . . . . 111 . . . .

. . . .

. . . .

. . . .

113 113 114 115

. . . . .

. . . . .

. . . . .

. . . . .

117 121 121 122 123

. . . . 125 . . . . 127

6 Path Planning of Nano-Robot Using Probability Distribution Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Path Planning for Landmark Observation Using Probability Distribution Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Tip Path Planning in Task Space . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Basic Path Planning in Single Landmark Environment . 6.2.2 Path Planning in Multi-landmark Environment . . . . . . 6.3 Simulation and Experimental Verification . . . . . . . . . . . . . . . 6.3.1 Path Planning Based on Dijkstra Method . . . . . . . . . . 6.3.2 Path Planning Based on Ant Colony Algorithm . . . . . . 6.4 Landmark Dynamic Configuration . . . . . . . . . . . . . . . . . . . . . 6.4.1 Definition of Landmark Domain . . . . . . . . . . . . . . . . . 6.4.2 Virtual Nano-hand Method . . . . . . . . . . . . . . . . . . . . . 6.4.3 Nano-manipulation Simulation Based on Virtual Nano-hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 AFM-Based Nano-manipulation Platform . . . . . . . . . . 7.1 Hardware and Software Implementation of System . 7.1.1 Hardware Platform . . . . . . . . . . . . . . . . . . . 7.1.2 Software Implementation . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . 129 . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

130 136 136 141 143 143 144 148 150 152

. . . 153 . . . 155 . . . .

. . . .

. . . .

157 157 158 159

xii

Contents

7.2 AFM Tip Localization . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Framework of Tip Localization System . . . . . . . 7.2.2 Model Parameters Calibration and Experimental Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 Accuracy Improvement of Tip Localization . . . . 7.3 AFM Nano-manipulation . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Virtual Nano-hand Nano-manipulation . . . . . . . 7.3.2 Demonstration of AFM Nano-manipulation . . . .

. . . . . . . . 161 . . . . . . . . 161 . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

162 174 178 178 180

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Chapter 1

Introduction

Abstract Nanotechnology is science and technology of making materials at atom and single molecule scale, and studies the properties and applications of materials ranging from 0.1 to 100 nm. Nanotechnology is a technology based on many modern advanced science and technology, which is the combination product of modern science and technology. This chapter first introduces the development history and application fields of nanotechnology, the characteristics of nanotechnology, and then explains primary nano-scopic observation methods, such as optical microscopic observation, SEM/TEM based observation, STM based observation, AFM based observation, etc. Finally, the primary nano-manipulation methods are introduced, including self-assembly technology, optical tweezers, DEP, SEM, AFM, etc. Keywords Nanotechnology

1.1


Nano-observation
Nano-manipulation

Introduction of Nanotechnology

Nanotechnology refers to study of the properties and interactions of substances at atomic and molecular scale, and preparation of nano-materials, fabrication of nano-devices, and detection and characterization at nano-scale [1]. Nanotechnology is extension of modern science and technology on the scale from 0.1 to 100 nm, which brings research of traditional physics, chemistry, materials science and biology into a new field and opens up a new technological development path for industrial manufacturing (involving electronics, machinery, materials and other fields), bioengineering and information engineering. Nanotechnology has been recognized as the major driving force of economic growth in the 21st century by the scientific community. Therefore, it has attracted great attention from all countries in the world, especially developed countries. Among these countries, the research, development, and industrialization of nanotechnology in the United States, Japan, Europe and other countries and regions are gradually taking shape. Nanotechnology

Authors: Shuai Yuan, Lianqing Liu, Tianshu Chu, Likai Liu. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_1

1

2

1

Period

18-19th Century : Invention of steam engine and the First Industrial Revolution Millimeter technology

19-20th Century: Discovery of electromagnetic and the Second Industrial Revolution

Introduction

20th Century: Nanotechnology and the Third Industrial Revolution Nanotechnology

Micro-technology

BC: Stone Age 1m

1 mm

1 µm Machining scale

100 nm

1 nm

Fig. 1.1 Relation between the development of science and technology, and fabrication scale

will bring revolutionary changes to these fields of life science, information and communication, manufacturing, environmental protection, energy development and utilization, materials and other scientific and technological fields. It will become a new scientific and technological representative in the third industrial revolution (As shown in Fig. 1.1) [2]. Nanotechnology will create more new materials devices for human beings, change people’s habits formed over thousands of years, and also make annotation of human life a new definition, which will have a significant impact on the development of biomedicine.

1.1.1

Development and Application of Nanotechnology

An important milestone in the development of nanotechnology is shown in Fig. 1.2. The invention of STM (scanning tunneling microscope) [3] tools, the emergence of new materials/structures and new methods have laid the foundation of nanotechnology. In 1959, Professor Richard Feynman proposed the idea of manipulating and controlling substances at atomic and molecular scale in the report of the annual meeting of the American Physical Society. This report is regarded as the origin of nanotechnology [4]. In 1962, Professor Ryogo Kubo of Tokyo University in Japan proposed the quantum confinement theory to explain the discontinuity of the energy levels of metal nano-particles, which is a very important milestone, making people have a better understanding of the electronic structure, morphology and properties of nano-particles [5].

1.1 Introduction of Nanotechnology

3

Nanotechnology Milestones

1959

Feynman's talk 1981

STM invention

1985 1986

C60 discovery AFM invention 1990

1991

Atom manipulation

Nanotube discovery

1999

Now and the future

Nanomaterials

Nanoelectromechanical device

Nanobiology

NanoSensor

Fig. 1.2 Important milestones in the development of nanotechnology

In 1974, Taniguchi first used nanotechnology to describe precision machining [6]. In 1981, scanning tunneling microscope was invented by IBM, G. Binnig and H. Rohrer of Zurich Research Laboratory, Switzerland, and AFM (atomic force microscope) was invented in 1986 [7]. These two inventions are considered to be significant breakthroughs in the history of nanoscience and technology. They provide technological tools for nanoscience research and make nanoscience research rapidly popularized worldwide. Therefore, STM inventors won the 1986 Nobel Prize in Physics. In 1985, British scientist H. W. Kroto, American R. E. Smalley and R. F. Carl used mass spectrometers to obtain the Baki sphere (C60) [8]. In 1990, researchers at IBM Zurich Laboratory successfully manipulated 35 xenon atoms using STM and arranged them into three letters of “IBM” [9]. This is the first successful operation for a single atom. It not only verifies the feasibility of bottom-up nano-fabrication, but also shows the great development potential of nanotechnology. In 1991, S. Iijima of Japan Electric Corporation found multiple-walled carbon nanotube (MWT) in the procedure of studying Baki sphere molecule [10], and in 1993, single-walled carbon nanotube (SWNT) was found. These two materials are the most promising substitutes for monocrystalline silicon for manufacture of new nano-electronic devices. Until now, they still dominate the research direction of

4

1

Introduction

nano-electronics. In 1996, P. Hoyer synthesized TiO2 nanotubes [11]. At present, scientists are studying application of TiO2 nano-particles or nanotubes in solar cells as photoelectric conversion materials, which are now close to practicality. Since the concept of nanotechnology has been put forward, its development not only opens up new fields for basic scientific research, but also provides broad prospects for applied technology research and development. This is mainly reflected in two aspects: on the one hand, discovering new physical, chemical and biological phenomena at atomic and molecular scale, exploring new nanomaterials, verifying or establishing new models and theories; on the other hand, making use of these nanomaterials to produce functional equipment and devices with novel physical or chemical properties. With the emergence of new concepts and methods in nanotechnology, it has become a new and interdisciplinary field of science and technology, such as physics, chemistry, biology, electronics, machinery, materials, manufacturing and measurement, which will promote the revolutionary development of current science and technology. In this book, the application of nanotechnology in life science, information and communication, environment and energy, materials and basic disciplines are described, and how to use AFM as the research tool is emphasized. 1. Application of Nanotechnology in Life Science The space size of proteins, DNA, RNA and viruses are all in the range of 0.1–100 nm, so nanostructures are also the basic forms of life phenomenon. Cellular organelles and other structural units are “nano-devices” that perform certain functions. Cells are like “nano-workshops”. Photosynthesis in plants cell is a typical example of “nano-factories”. The self-assembled arrangement of genetic sequences achieves precise atomic structure, and the information transmission and feedback of nervous system are perfect examples of nanotechnology. Biosynthesis and biological processes have become the source of inspiration and manufacture of new nanostructures. Researchers are imitating biological characteristics to achieve nano-scale control and manipulation in technology. At the same time, the size of nanoparticles is often smaller than that of red blood cells in organisms, which provides a new opportunity for medical research. At present, there are some good examples of application: using SiO2 nano-particles to achieve cell separation, using Au nano-particles to dye the inside of cells, using new drugs coated with magnetic nano-particles for local targeted treatment, and so on. The invention of AFM provides the feasibility for studying the nano-morphology and structure of living biological samples. The main advantage of AFM is that it can acquire sub-nano image of biological samples in physiological environment [12], and it does not need pre-processing such as dyeing and fixing biological samples [13]. Therefore, AFM observation has shown great advantages in biology, especially in characterizing the morphology and the structure of single cell [14, 15] and single molecule [16, 17], which has been widely used in life science. This new observation technology is an effective complement to optical microscopy, X-ray crystallography and electronic crystallography [13], which provides a new method for high-resolution morphology research in biomedical

1.1 Introduction of Nanotechnology

5

science, and enables people to have a new understanding of physiological activities at the cellular and molecular level. AFM is not only an imaging tool with atomic resolution, but also a multi-functional toolbox [13]. The mechanical properties of the sample can be measured by controlling indentation of the AFM tip on sample surface. Measuring the mechanical properties of a single cell by using AFM is a hot topic in the application of AFM in biology [18]. Mechanical properties play an important role in key cellular activities (migration, division, deformation, etc.) [13]. The mechanical properties of cells have been considered as a new biomarker in recent years. The study of these mechanical properties will help to better understand the physiological mechanism of diseases and provide new methods for early detection, diagnosis and treatment of diseases. 2. Application of Nanotechnology in the Field of Information and Communication Since the first integrated circuit (IC) came out in 1958, micro/nano electronic devices manufacturing, ultra-high density information storage technology and micro/nano electromechanical systems have developed rapidly, which have become the basic and core technology in research field of information and communication [19]. The integration of electronic devices has been improved continuously according to Moore’s law. At present, the line width of commercial integrated circuits has been reduced to 7 nm. However, the current machining dimension is limited by technological conditions, so there is a limit size. Additionally, when the circuit structure exceeds the limit size, the working state of the circuit will not follow the traditional physical law, but show obvious quantum effect and statistical fluctuation characteristics [20, 21], which will restrict the improvement of electronic device integration. The emerging nanotechnology provides an effective way to break this problem. It is found that electronic devices based on new nanomaterials, such as single electron transistors, can work under quantum size effects such as coulomb blocking and ballistic transport. This will hopefully break through the physical limits of integrated circuits, and make electronic devices more integrated, less power consumption and faster. AFM can be used to measure the linewidth of insulators and conductors at nano-scale [22–30]. With the continuous improvement of circuit integration in IC manufacturing industry, the requirement for the minimum linewidth measurement range and accuracy of semiconductor grooves is also increasing correspondingly. The measurement accuracy of AFM will help to further improve the process level of circuit integration. On the basis of observation, we can also use the mechanical force between the tip and the sample surface to develop prototypes for assembly of nano-devices, such as single electron transistor [31], plasma waveguide [32], quantum cellular automata control [33], antioxidant mask [34], DNA biochemical circuit [35], assembly DNA, protein microarray [36–38]. In addition, AFM can also be used for nanostructure fabrication. At present, mechanical and electric field processing has been realized on various materials, such as polymer [39–44], metal, insulator [45–51] and semiconductor [52–55].

6

1

Introduction

3. Application of Nanotechnology in the Field of Environment and Energy The cross-integration of nanotechnology with traditional environment and energy technology can improve the efficiency of traditional energy and greatly reduce pollutant emissions. For example, organic solar cells coated with nano-layers or nano-rods can greatly increase the amount of solar energy generated. It is found that the working organic solar cells can be “seen” by the new AFM, and their three-dimensional nanostructures and properties can be linked [56]. The surface state [56–58] and photoelectric potential [59], photocurrent [60–62] can be measured at the same time by AFM tip, and the relationship between nano-scale structure and current change of organic photovoltaic materials can be established, which is of great significance for optimizing the structure of organic photovoltaic materials and improving the performance of organic cells. Many environmental problems such as air pollution, sewage treatment and municipal garbage will be solved by further organic combination of nanotechnology and environmental protection and environmental governance [63]. At present, environmental protection, agencies are developing a kind of nano-membrane with unique functions. The membrane can detect contamination caused by chemical and biological agents and filter them to eliminate contamination. Nanotechnology will strengthen the ability of human beings to protect the environment, thoroughly improve the relevant measures of environmental protection, and even change people’s traditional environmental protection concepts. In the research of environmental protection and environmental treatment, AFM has studied nanomaterials, such as nano-alumina nano-membranes [64, 65], which have unique functions. These nano-membranes can detect the pollution of heavy metals to water sources, and filter these heavy metals to eliminate pollution. 4. Application of Nanotechnology in Materials and Basic Disciplines Nanomaterials refer to the transition region that the size of their structural units is between atomic clusters and macroscopic objects, with surface effect, small size effect and macroscopic quantum tunneling effect. It will show many strange properties, that is, its optical, thermal, electrical, magnetic, mechanical and chemical properties will be significantly different from the macroscopic state. Therefore, on the basis of nanometer observation and manipulation, it is necessary to analyze these characteristics and carry out applied research in order to develop unique and excellent nanomaterials. In recent years, graphene and other nanomaterials have been extensively studied as new materials [66]. AFM is the main tool to measure the mechanical and electrical properties of these materials, and through nano-indentation, electric field cutting and other processing operations [67–69] to reveal the conversion law of material structure and properties in nano-manufacturing process. The lateral force imaging by AFM can also be used to study friction at nano-scale [70–74], which provides an effective technical tool and research approach for observing and analyzing friction at nano-scale. In addition, AFM can also fabricate ion traps and carry out quantum dot manipulation [75, 76], which provides a feasible method for study

1.1 Introduction of Nanotechnology

7

of electronics and quantum mechanics at nano-scale. In a word, these AFM applications have important practical significance in improving the research level of new nanomaterials and mesoscopic physics, promoting the development of mechanical and physical disciplines in basic and frontier.

1.1.2

Characteristics of Nanotechnology

Nanometer is a unit of length, 1 nm = 10−9m, that is, 1 nm equals one billionth of a meter. Through the following introduction, we will have an intuitive understanding of nano-scale. Atom is basic unit of matter. In the nature, the diameter of hydrogen atom is 0.16 nm, and the diameter of carbon atom is about 0.172 nm. Therefore, 1 nm is roughly equivalent to the sum of the diameters of several carbon atoms. Atomic clusters consisting of several to hundreds atoms in size are called “clusters”. At present, fullerenes can be synthesized in large quantities. The diameter of C60 is 0.7 nm. The size of red blood cells in blood is 6–8 lm. The length of common bacteria, such as E. coli, is 2–3 lm. The viruses causing human diseases are usually tens of nanometers. Therefore, the nanoparticles are smaller than red blood cells and bacteria, and are comparable to or slightly smaller than viruses, as shown in Fig. 1.3. The goal of nanotechnology is to fabricate devices

Billion nanometers Million nanometers A black dot on the thumb is about a million The diameter of red 12 hydrogen nanometers blood cells is about atoms are about 1 nanometer and the several thousand nanometers DNA molecule is about 2.0 nm. Nanometer

Less than 1 nm The atomic diameter is usually several angstroms

Thousand nanometers

Fig. 1.3 Relative size of different structures

The height of 2 meters is 2 billion nanometers.

8

1

Introduction

with specific functions based on novel physical, chemical and biological characteristics of atoms, molecules at nano-scale. According to the National Science Foundation of the United States and the National Initiative on Nanotechnology, nanotechnology refers to the ability to understand, control and manipulate nano-materials (0.1–100 nm) in order to create materials, devices and systems with novel characteristics and functions; nanotechnology includes nanoscience, engineering and technology, mainly involving imaging, measurement, modeling and operation of nanomaterials. Although the precise definition of nanotechnology is not yet fully unified, they all have three characteristics: • The range of material structure is between the diameter of a single atom or molecule and 100 molecules or about 100 nm; • The ability to detect and manipulate at nano-scale; • Develop unique properties and functions of materials, devices and systems at nano-scale.

1.1.3

The Key Nanotechnology: Nano-observation and Manipulation

Observation and manipulation on nano-scale is the key technology to carry out nanoscience research, discovering and manufacturing new characteristics of things at nano-scale. Only by observing at nano-scale can we effectively study the micro/ nano structure and characteristics, and accurately provide the state of the operating environment, material structure and tool position. On the foundation of observation, meaningful assembly and fabrication are realized by means of control methods and manipulation tools, and purposeful scientific experimental research and monitoring of manufacturing process can be realized. Therefore, observation methods and manipulation techniques at nano-scale can be summarized as follows. Nano-observation—On-line and quantitative observation of material structure and properties at nano-scale is realized by means of scanning electron microscopy (SEM), transmission electron microscopy (TEM) and scanning probe microscopy (SPM). Nano-manipulation—Using self-assembly, S/TEM, optical tweezers and SPM to move nanoparticles/nanotubes and construct nanostructures. At the same time, it provides scientists with a “micro laboratory” to study new phenomena and put forward new theories at nano-scale.

1.2 Primary Nano-observation Methods

1.2

9

Primary Nano-observation Methods

Nano-observation technology is applied to characterize nanomaterials and structures in order to comprehensively understand the surface morphology and microstructures of nanomaterials and their related properties at atomic level. Because the size and structure of nanomaterials have a great influence on their physical properties, nano-observation technology is the premise and foundation of nanotechnology. Now we begin with traditional optical microscopy to introduce various observation methods and tools at nano-scale, including SEM/TEM, STM and AFM.

1.2.1

Optical Microscopic Observation

Currently, observations are mainly performed using laser confocal microscopy and laser interference microscopy. Laser confocal microscopy, as an important improvement of optical microscopy, has many unique advantages compared with traditional field illumination microscopy: it can control the depth of focus, illumination intensity, and reduce the interference of non-focal plane light noise. Also it can obtain optical slices from a certain thickness specimen, and observe very clear high quality without changing the method of ordinary fluorescence microscopy slicing. Meanwhile, it can measure images and observe living cells or tissues very conveniently. In fact, confocal technology has become one of the most important technological breakthroughs in optical microscopy research. Compared with the traditional field microscopy and laser scanning confocal microscopy, the traditional microscopy images are shown in Fig. 1.4a–c. Correspondingly, the confocal microscopy image (d)–(f) are more clear, delicate and hierarchical as shown in Fig. 1.4. Laser interferometric microscopy can generate topographic height data from the coherent light reflected from the sample surface and the reference reflection. The interferometric objective is scanned in the vertical direction and the evolution of interference fringes is recorded by Charge-Coupled Device (CCD). By analyzing the intensity change in the process of fringe evolution, the computer can accurately determine the height of the sample morphology. Figure 1.4g is a laser interference microscope image of the surface structure of a silicon solar cell.

1.2.2

SEM/TEM Based Observation

The principle of SEM is that an electron beam with a diameter of 20–30 nm is emitted from the cathode of an electron gun. Under the action of accelerating voltage between the cathode and anode, the electron beam is projected to the barrel of the electron gun and reduced to an electron tip with a diameter of several

10

1

(a)

(b)

(c)

(d)

(e)

(f)

(a), (d) Same specimens: mouse hippocampus (b), (e) Same specimens: rat smooth muscle; (c), (f) Same specimens: sunflower pollen

Introduction

(g) (g) Laser interference microscope sample image of the surface structure of a silicon solar cell

Fig. 1.4 Sample images of laser confocal scanning microscopy and laser interference microscope

nanometers by convergence of the concentrator and the objective lens. With the action of the scanning coil on the upper part of the objective, the electron tip scans the sample surface and emits various electronic signals. These electronic signals are detected by corresponding detectors, amplified and converted into voltage signals, and finally sent to the gate of the kinescope to modulate its brightness. The electron beam in the image tube scans with grating on the fluorescent screen, and the scanning motion is strictly synchronized with the scanning motion of the electron beam on sample surface, so that the scanning electron image corresponding to the received signal intensity can be obtained, which reflects the morphological characteristics of the sample surface. The shining point of Fig. 1.5 is the SEM image of gold nanoparticles. TEM projects an accelerated and concentrated electron beam onto a very thin sample, the electron collides with atoms in the sample to change direction, thereby forming solid angle scattering. The scattering angle is related to the density and thickness of the sample, so different images can be formed. After zooming and

Fig. 1.5 Sample images of SEM and TEM

1.2 Primary Nano-observation Methods

11

focusing, the images can be displayed on imaging devices (such as fluorescent screen, film and photosensitive coupling module). The right image of Fig. 1.5 is a TEM image of gold nanoparticles. The resolution of TEM is much higher than that of optical microscope due to the very short De Broglie wavelength of electrons, which can reach 0.1–0.2 nm with magnification of tens of thousands to millions of times. Therefore, TEM can be used to observe the fine structure of the sample and even a row of atomic structures, which is tens of thousands of times smaller than the smallest scale observed by optical microscopy. TEM is widely used in many scientific research fields related to physics and biology, such as cancer research, virology, materials science, nanotechnology and semiconductor research. When TEM is imaged at low magnification, its contrast is mainly related to the different degree of electron absorption related to the different thickness and composition of the material. When the magnification is high, the complex fluctuation will cause different image brightness, so it needs professional knowledge to analyze the obtained image. By using different scanning modes of TEM, the samples are imaged according to the chemical properties, crystal orientation, electronic structure and electronic phase shift caused by the samples.

1.2.3

STM Based Observation

The principle of STM is to use the tunneling effect of the quantum theory to treat the atomic-scale tip and the surface of the studied material as two electrodes. When the distance between the sample and the tip is very close (usually less than 1 nm), under the action of an applied electric field, electrons will pass through the barrier between the two electrodes to the other electrode, which is called the tunneling effect. Tunnel current intensity is very sensitive to the distance between the tip and the sample surface. If the distance is reduced by 0.1 nm, the tunnel current will increase by an order of magnitude. Therefore, the constant tunneling current is controlled by an electronic feedback circuit and scanning of the tip on the sample surface is controlled by a piezoelectric ceramic material. The change of the tip height in vertical direction reflects the fluctuation of the sample surface. Figure 1.6 is a STM scan image of graphene sample. From the working principle of STM, it can be seen that the tip structure and morphology play an important role in the process of STM observation on sample surface. The curvature radius of the tip is the key factor affecting the transverse resolution. The size, shape and chemical identity of the tip affect not only the resolution of STM image, but also measurement of sample structure. Therefore, the accurate description of the geometry and electronic characteristics of the tip is of great reference value for evaluation of the experimental quality.

12

1

Introduction

Fig. 1.6 STM image of graphene

1.2.4

AFM Based Observation

The biggest difference between AFM and STM is that the former uses the Van Der Waals Force interaction between atoms to present the surface properties of samples. It is assumed that there are two atoms, one is at the tip of a cantilever and the other is at the sample surface, the force between them varies with the distance. When atoms are very close to atoms, the repulsion force of each other’s electron cloud is greater than the attraction force between the nucleus and the electron cloud, so the whole resultant force is repulsion force. Otherwise, if two atoms are separated at a certain distance, the repulsion force of the electron cloud is less than the attraction force between each other’s nuclei and the electron cloud, and so the whole resultant force is the action of gravitation. Violita Navarro of the University of Madrid in Spain captured gold crystals by using AFM, producing the clearest picture of the world’s tiny objects. The tiny AFM cantilever moved back and forth on sample surface. The laser interferometer captured the slight movement of the cantilever as it passed the inflection point of atomic size, thus generating high information. Figure 1.7 shows the AFM images of porous aluminum, diamond and nerve cell surfaces from left to right. AFM can represent the surface microstructure of materials with high resolution, and it can also be used to characterize the mechanical and physical properties of local micro-areas on material surface, such as quantitative measurement of elastic modulus, hardness, adsorption and viscoelastic properties. Early attention was paid to the excellent imaging function of AFM and the detection of mechanical properties in micro-regions. In recent years, with the rapid development of AFM, its application fields have been extended to the relocation of atoms and molecules, mechanical processing at micro-and nano-scale, material surface modification and high-density storage in information technology. In the past decades, traditional

1.2 Primary Nano-observation Methods

13

Fig. 1.7 AFM images of porous aluminum, diamond and cell surface

electron microscopy (including SEM/TEM) has been the main method to study the microstructure of materials by imaging. In recent years, the application of AFM has developed more rapidly. The main reasons for attracting people to apply atomic force microscopy are as follows [77]. ① AFM can detect samples under atmospheric condition or in liquid phase environment, unlike SEM/TEMs, by which samples must be observed under high vacuum condition. This allows the AFM to have real-time physical observation that can be used for online observations such as on-line monitoring of biological organism and on-line monitoring of physical and chemical reaction of polymers, as well as product quality control on industrial production lines. ② AFM can measure the size of the sample structure unit at three-dimensional scale with a high vertical resolution. SEM/TEM can only perform detection on the transverse scale (two-dimensional), but not in the longitudinal direction. ③ AFM has the function of measuring the mechanical and physical characteristics of local micro-regions on material surface which SEM/TEM does not possess. ④ Some materials are sensitive to electron beam bombardment, which often results in damage or microstructural changes of samples, it can reach an unobservable level in severe cases. The abovementioned problems can be avoided by AFM. ⑤ Sample preparation of AFM is very simple, and TEM usually requires rich experience and skilled skills, as well as long preparation time. ⑥ The structure of the AFM is compact and easy to operate, while the SEM/TEM equipment is huge, the operation is complex and the price is expensive. The comparative analysis of the main performance indicators of AFM and SEM/ TEM is as follows (Table 1.1). (1) Research on High-Speed Scanning AFM From the view of AFM usability, a data-driven-based parameters self-tuning control and imaging method of AFM is proposed in reference [78, 79]. The model parameters of the AFM imaging system will be changed when the sample, scanner, tip or scanning parameters are changed, and so the gain of the controller needs to be adjusted again. In order to increase the usability of the system, the AFM control and

14

1

Introduction

Table 1.1 Main performance indicators of AFM, TEM and SEM Index

AFM

TEM

SEM

Spatial resolution

Atomic level: 0.1 nm

3–6 nm

Sample ambient

Real environment, atmosphere, liquid, vacuum Room temperature, high temperature, low temperature Almost no

Point resolution: 0. 3–0.5 nm Line resolution: 0.1–0.2 nm High vacuum

High vacuum

Room temperature, high temperature, low temperature Damage to electron beam sensitive materials No

Room temperature, high temperature, low temperature Damage to electron beam sensitive materials No

Yes

Yes

Temperature

Sample damage Mechanical properties Elemental analysis

Local micro-region mechanical properties No

imaging method based on the data-driven that can automatically adjust parameters is designed, thereby reducing the threshold of AFM employment. Firstly, the parameter model of controlled auto-regressive and moving-average (CARMA) is introduced to represent the local dynamic linearization model of the AFM system, and the parameters of the model are obtained by data-driven identification method. Then, the Proportional-Integral (PI) controller parameters are calculated online based on the generalized predictive control (GPC) optimization method, and an AFM imaging method with automatic adjustment of control parameters is obtained. The simulation and experimental data show that when the scanning speed of AFM is changed or the PI parameters of the controller are not properly selected, the method can adjust the PI parameters in time, reduce the control error and improve the imaging accuracy. To further illustrate the good performance of this method, the cross-sectional topographic curves of the same location are compared in Fig. 1.8. The dotted line represents the height curve obtained when PI parameters are fixed, and the solid line is the height curve obtained by this method. Statistical calculation shows that the average step height (defined as the difference between adjacent peaks) of the former is 162.3 nm, while the average step height of the latter is 104.6 nm. Obviously, the latter is closer to the real sample morphology. Compared with the existing methods, this method can significantly reduce the imaging error caused by the inappropriate parameters of PI controller, thereby improving the quality of AFM scanning image. Under the circumstances of different scanning speed of AFM or inappropriate selection of PI parameters, this method can automatically adjust the parameters of the controller, thereby reducing the control error and improving the imaging accuracy, thus reducing the operation complexity of the AFM and improving the intelligent level of the system.

1.2 Primary Nano-observation Methods Fig. 1.8 Contrasted curves of imaging results for a scan line

15

Morphological height (nm)

100

50

0

-50

-100

Imaging method proposed in the reference [79] Method for controlling image with fixed PI parameters

-150

0

50

100

150

200

250

300

350

Scan direction (pixel)

(2) AFM Scanning Imaging Learning Control System Based on Online Adjustment In the AFM system, the main dynamic characteristics come from piezoelectric scanners. As for the general AFM system, the response time of the scanners is relatively long. It is necessary to compensate for them in order to make the system fast and stable at each scanning point. Therefore, in the design of the controller, the dynamic characteristics of the scanner are mainly considered, while the dynamic characteristics of other parts are neglected [80]. The learning control system shown in Fig. 1.9 is composed of two parts in series, learning controller and dynamic compensator, which constitute an AFM scanning imaging learning control system based on online adjustment. The main function of the dynamic compensator in the system is to shorten the response time of the piezoelectric scanner as much as possible and lay the foundation for fast scanning. The main purpose of the learning controller is to effectively control the error changes caused by the fluctuation of the sample surface. The

d(t)

Vsp -

Learning controller

u1(t)

Dynamic compensator

u(t)

Fig. 1.9 Structural diagram of an AFM learning control system

AFM System

16

1

Introduction

system treats the fluctuation d(t) of sample surface as an unknown disturbance acting on the system, and compensates the signal d(t) by using the interference suppression link u1(t) 2 R1 based on repetitive learning control method. When using the fast scanning mode of the learning control system to scan soft samples with periodic fluctuation characteristics, it can significantly improve the scanning speed of AFM, and control the distance between the sample and the tip, effectively avoiding the damage of the sample or tip. In the fields of material science and life science, a large number of samples have a fixed period. Therefore, the fast scanning mode of AFM based on learning control will obviously improve the imaging performance of AFM and further broaden its application scope. (3) Measurement of Mechanical Properties of Samples Using AFM The emergence of AFM provides a new technique for obtaining the physical properties of sample surface at nano-scale. At present, most application is the test of the cell mechanical properties under physiological condition. By comparing and analyzing the difference of surface hardness and adhesion of surface elastic modulus between pathological cells and normal cells, we can dynamically analyze and diagnose pathological cells. Compared with the method of cytochemical characterization, this method has the advantages of label-free, direct detection of living cells in natural state and simple preparation of sample, which can detect the dynamic changes of cell physical properties during the physiological activities of cells or external factors [81]. The principle of measuring the physiological characteristics of the cell surface using AFM is to load nano-scale force on AFM tip, so that the AFM tip can press into the cell surface without causing damage. Cells are induced to stimulate, and the body produces certain deformation, then record the force curve, and transforms the force curve into the force indentation curve, as shown in Fig. 1.10, so as to obtain the mechanical properties of biological sample.

Laser

Cantilever

Probe z

Contact point

Contact point

Deformation

PCTD

Four quadrant position sensor

Deformation

Signal processing and feedback control circuit

Distance

Tip

Distance

Cell

y

Substrate

x (a) principle of AFM mechanical characteristic measurement

(b) Mechanical properties measurement of the standard sample

(c) Mechanical properties measurement of the cell surface

Fig. 1.10 Diagram for the detection of cellular physiological characteristics by using AFM

1.3 Primary Nano-manipulation Methods

1.3

17

Primary Nano-manipulation Methods

Nano-manipulation is the key technology to carry out nano-science research, discovering new characteristics of nano-scale things and realizing fabrication and manufacture. Purposeful manipulation to achieve meaningful nano-scale observation, assembly, and fabrication and so on is the necessary methods to carry out scientific experimental research, fabrication and manufacture. Nano-manipulation is the premise of exploring and studying the properties and structure of nanomaterial. It is of great significance to the development of materials, information, biology and many other scientific fields. Therefore, it has become a research direction of common concern of scientists in different fields, and has developed a variety of nano-manipulation methods based on different mechanisms. Among them, the following five methods are widely used: self-assembly based nano-manipulation [82–84]; optical tweezers-based nano-manipulation [85–91]; dielectrophoresis-based nano-manipulation [92–99]; SEM based nano-manipulation [100–102]; and AFM-based nano-manipulation [45, 101, 103–110].

1.3.1

Self-assembly Based Nano-manipulation

The self-assembly refers to the procedure in which basic structural units (molecules, nanomaterials, micron or larger scale substances) spontaneously organize or aggregate into a thermodynamically stable, structurally geometrically stable, and functionally specific aggregate through non-covalent-bond based interactions (such as space matching, electrostatic force, hydrophobic interaction, hydrogen bond, etc.) under equilibrium condition. The process of self-assembly is not a simple superposition of weak forces among a large number of atoms, ions and molecules, but a spontaneous association among several individuals for forming a close and orderly ensemble, which is a comprehensive and complex synergy. In the procedure of self-assembly, the designer sets the control process by choosing parameters, but after the process begins, the outside world can no longer intervene in the process. Figure 1.11 shows the experimental results of the directional alignment of large-area single-walled carbon nanotubes (SWCNTs) by self-assembly of LeMieux et al. [82].

1.3.2

Optical Tweezer Based Nano-manipulation

Optical tweezers make full use of the gradient force of light to drive particles to move towards the position where the maximum light intensity is. Figure 1.12 shows the basic principle of optical tweezers manipulation [87]. Since its appearance in 1986, it has been widely used in biology, physical chemistry and condensed matter physics as a micro/nano manipulation tool. The size of optical tweezers

18

1

A

NH2

Si O O Si O

NH2

NH2

Introduction

Surface B

Surface A

Si O Si O O Si O Si

B

O

Si

O

O

Si

O

Si

O

Si

100

O

80

Si

12

FWHM 24.3°

60 40

0

8 6 4

20

2 -80 -60 -40 -20 0 20 40 60 80

Θ (°)

SiO2

FWHM

10 Counts

O

16 14

Counts

Si

140 120

0

-80 -60 -40 -20 0 20 40 60 80

Θ (°) (b) Spin assembly of single walled carbon nanotubes

Si

S

(a) Functionalized surface

D SiO2

Si

(c) Deposition electode

Θ: orientation angle of nanotubes after assembly; S: source electrode; D: drain electrode

Fig. 1.11 Single-walled carbon nanotubes alignment using self-assembly method. “From LeMieux et al. [82]. Reprinted with permission from AAAS”

manipulating object is from tens of nanometers to tens of micrometers, which can realize various operations such as particle capture, movement and rotation, as shown in Fig. 1.12. The main problem of optical tweezers used in nano-manipulation is the low resolution, which makes it difficult to directly manipulate a single nano-object. Gradient force control

Optical axis

Wave front λ

Colloidal particles Gradient force control

Laser beam Optical tweezer

Various operations of optical tweezer particle capture, movement and rotation

Fig. 1.12 Diagram for movement and rotation of optical tweezer. “Reprinted by permission from Springer: Springer, Nat Photon Grier D G. A revolution in optical manipulation, [Copyright], 2019”

1.3 Primary Nano-manipulation Methods

1.3.3

19

DEP Based Nano-manipulation

DEP based nano-manipulation is a manipulation technology on the foundation of Maxwell’s classical electromagnetic field theory. The controllable electric field is generated by using the pre-set electrode structure to realize the directional control of material transportation, capture, rotation, separation, orientation or localization arrangement and assembly, as shown in Fig. 1.13a. The object of DEP based nano-manipulation mainly includes zero-dimensional and one-dimensional nanomaterials. These materials include gold/silver nanoparticles, carbon nanotubes, nanowires, DNA molecules and so on. Carbon nanotube is the research focus of nanomaterials. Because of its unique electrical, mechanical, photoelectric and chemical properties, which has received extensive attention and research by scientists and technicians in many fields. Since Yamamoto took the lead in separating and purifying carbon nanotubes by using DC [93] and AC [95] electrophoresis, and Chen et al. assembled single-walled carbon nanotubes [96] by using AC electric field for the first time, great progress has been made in the research of using DEP technology to manipulate carbon nanotubes, including separating carbon nanotubes with different electrical properties to purify carbon nanotubes materials; using carbon nanotubes to assemble carbon nanotube probes. Carbon nanotubes (CNTs) are assembled into micro/nano electrodes for constructing micro/nano devices. The main problem of DEP is that the resolution and control accuracy of DEP force are restricted by the size of micro-electric field. It is difficult to achieve quantitative and precise motion control and localization control of micro/nano objects through external electric field, so it is more suitable for batch qualitative control of micro/ nano materials. Figure 1.13b is the result of the operation of multi-walled carbon nanotube (MWCNT) by DEP [94]. The MWCNT clusters are clustered near the electrodes. This method lacks the ability of quantitative and fixed-point manipulation of nanomaterials, and it is difficult to meet the requirements of precise manipulation of nanomaterials and construction of nanostructures.

Medium spherical particle Uneven alternating electric field + +

+

F - DEP

-

vacuum

-

+

Au electrode

(a) Nano-manipulation schematic of SEM

Clustered multiwalled carbon nanotubes

Au electrode

(b) Manipulation result of MWCNT by using DEP

Fig. 1.13 DEP based Nano-manipulation. a Nano-manipulation schematic of SEM. b Manipulation result of MWCNT by using DEP. “Reprinted (adapted) with permission from Fiedler et al. [95]. Copyright (2019) American Chemical Society.”

20

1.3.4

1

Introduction

SEM Based Nano-manipulation

SEM based nano-manipulation refers to embedding nano-manipulators in the vacuum chamber of SEM, and then completing specific nano-assembly tasks with help of real-time images of SEM. Figure 1.14 shows the procedure in which researchers at Oldenburg University in Germany embedded nano-manipulators into a scanning cavity of SEM to manipulate a single CNT [102]. Although the specific nano-manipulation tasks can be effectively accomplished with help of SEM visual feedback, the limitations of this method are as follows: (1) the imaging resolution of SEM is limited (the limit is 1–3 nm, usually more than 10 nm), and some typical nanomaterials such as single-walled carbon nanotubes, DNA molecules are about 1 nm in diameter, which will lead to loss of observation and manipulation function of SEM. (2) SEM requires that the observed objects have certain conductivity, and gold plating is generally required on the surface of non-conducting materials when imaging, which may damage the structure and properties of non-conducting materials; (3) SEM usually requires working in a vacuum environment, so it is impossible to observe and manipulate some active biological samples, and the high-energy electron beam of SEM is easy to destroy biological samples; (4) Lack of force/tactile feedback information during SEM based manipulation makes operators unable to rely on force/tactile feedback information to assist accurate manipulation or assembly. It is easy to cause end-effectors (such as micro-tips) or nanomaterials to be damaged due to excessive force during manipulation. These limitations make it difficult for SEM based nano-manipulation to be widely used in practice.

Holder

1

2

3

4

CNT Substrate

Triaxial micromanipulator Nano -positioning platform

(a) Nano-manipulator structure

(b) Manipulationof a single CNT

Fig. 1.14 Single CNT operation using SEM. “Reprinted from Kim et al. [102], with the permission of IEEE.”

1.3 Primary Nano-manipulation Methods

1.3.5

21

AFM Based Nano-manipulation

AFM is a tool for observing the interaction force between the tip and the sample at nano-scale. Based on the observation, some researchers began to make use of the force between the tip and the sample to change the micro-morphology of the material and manipulate atoms, molecules, biological DNA and nanoparticles. Figure 1.15a shows the principle of nano-manipulation using AFM, which performs mechanical manipulations such as push, pull, cut, etch, press, contact and stretch through the interaction between the terminal tip and the object. Compared with the previous nano-manipulation methods, AFM has many advantages, such as high resolution and motion accuracy, controllable and repeatable operation mode, mechanical operation mechanism, and adaptable to any environment. It overcomes the principle defects of the above-mentioned methods in nano-manipulation, such as weak controllability, poor repeatability and low operation resolution, which becomes the most potential nano-manipulation tool at present. Numerous researches have been applied in many fields. As early as 1995, Junno et al. used AFM to manipulate GaAs particles on the surface of GaAs substrate, forming “nm” character shapes, as shown in Fig. 1.15b [103]; Schaefer et al. used AFM tip to manipulate gold nanoparticles on the surface

Pushing/Pulling

Indenting

Cutting

Lithography

Touching

Fiber Pulling

(a) AFM nanomanipulation schematic diagram

(b) Manipulation of nanoparticle to construct "nm" pattern

z/Å

y/Å x/Å (c) Mechanical nanolithography results of mica surface

Fig. 1.15 AFM based Nano-manipulation. “Reprinted from Junno et al. [103], with the permission of AIP Publishing.”

22

1

Introduction

of highly oriented pyrolytic graphite (HOPG) to form nanostructures [45]; Martin et al. used non-contact mode AFM to form nanostructures on SiO2 surface in 1998. In 2003, Decossas used AFM to bend carbon nanotubes [106], Yang reported the effect of the AFM force on the transport of copper nanoparticles [107]; In 2004, Muller used AFM to achieve a 3 nm gap nano-scale characterization on mica surface [108]; In 2005, Pumarol et al. deposited gold nanoparticles on InP substrates by applying electrical pulses on AFM tip [109]. Sugimoto et al. also used AFM to push Sn atoms on Ge substrates at room temperature, arranged the Sn atoms to form the letter “Sn” and so on [110]. The application research based on AFM has also been carried out in China. The Center for Precision Engineering of Harbin Institute of Technology has installed diamond tips on the AFM to machine nano-materials such as monocrystalline silicon and monocrystalline aluminium [111, 112]. The mechanical cutting experiments of single crystal copper were also carried out by using diamond tip in the College of Mechanical Engineering and Automation of HuaQiao University [113]. The stretching and shearing experiments of nanotubes were carried out in the State Key Laboratory of Tribology, Tsinghua University [114]. DNA molecule was cut and picked up by Shanghai Jiaotong University [115]. The micro/nano group of Shenyang Institute of Automation, Chinese Academy of Sciences has carried out precise manipulation and assembly experiments on single/multi-walled carbon nanotubes [116, 117], and realized controllable manipulation of single-molecule viruses in three-dimensional space [118]. In addition, aiming at the problem that manipulation and observation of AFM cannot work at the same time, the State Key Laboratory of Robotics and Systems of Harbin Institute of Technology has proposed a dual-tip AFM operating system, which can realize the mode of manipulation while observing as shown in Fig. 1.16. When the system works, it needs to calibrate the position of each tip. On this basis, the two tips calibrate each other to achieve collaborative work [119–121].

Fig. 1.16 Principle diagram of dual-tip scanning for AFM

1.4 Application Characteristics and Problems of AFM Based Nano-manipulation

1.4

23

Application Characteristics and Problems of AFM Based Nano-manipulation

Through the abovementioned representation, we know that AFM compared with the manipulation methods based on SEM/TEM, DEP, optical tweezers and so on, has the characteristics of motion accuracy, controllable and repeatable operation mode, unique mechanical mechanism, and the ability of observation and manipulation, which has attracted wide attention of researchers. It overcomes the shortcomings of the abovementioned methods in assembling nano-electronic devices and equipment, and carries out high-precision nano-manipulation on the basis of high-resolution observation, which is the most potential nano-manipulation tool at present. However, due to the lack of information feedback in nano-manipulation, the real-time manipulation results cannot be displayed. The manipulation results can only be obtained by re-imaging after maneuvering, so the manipulation efficiency is not high. At present, there are many uncertainties in AFM nano-manipulation, such as thermal drift of the system, the tip broadening effect, tip position error and control instability, which make it difficult for the AFM tip to achieve accurate localization and stable manipulation in task space. The broadening effect of the tip makes the feature width of the image larger than its true width, which results in the error of the scanning image [122], as shown in Fig. 1.17. In view of the errors existing in the control of PZT, two methods are mainly adopted: model compensation and closed-loop control of PZT. The main problem of model compensation is that its performance depends on the accuracy of parameters. Accurate calibration of model parameters is a very difficult and time-consuming task, and the robustness of model compensation is weak. Because AFM usually works in arbitrary environment, the anti-interference ability of model compensation is not strong due to the change of environmental conditions, and it is not easy to control the error in real time. Closed-loop control can obtain better compensation effect, but this method can only improve the localization accuracy of the tip relative to the central axis of the PZT (robot joint space), and it is difficult to eliminate the influence of system thermal drift and relative random drift.

(a) SEM image of nanostructure Fig. 1.17 AFM tip broaden effect

(b) AFM image of nanostructure

24

1

Fig. 1.18 Traditional nano-manipulation for target location. © [2019] IEEE. Reprinted, with permission, from Xu et al. [123]

Introduction

Initial push line Pushing line Base line x t , yt

xi , yi

x 3, y3

x2 , y2

x1 , y1

TOP

x0 , y0

Although some researchers have proposed the model-based thermal drift compensation methods, such as Kalman filter and neural network, the effect of these methods depends on the accuracy of model parameters. However, it is difficult to obtain accurate model parameters in practice, so it is impossible to accurately compensate thermal drift. The traditional AFM nano-manipulation method is target oriented pushing (TOP) [123], that is, the manipulation direction is always from the current position to the target position, as shown in Fig. 1.18. TOP operation requires the tip to act on the center of the nanoparticles and push the nanoparticles to the target position according to the established trajectory. However, due to many uncertainties, it is difficult to meet the manipulation requirements of TOP in practice. The maneuvering results always deviate from the predetermined designed trajectory, or lead to the separation of the tip and the nano-particle. In order to solve the uncertainty and poor stability of contact manipulation caused by inaccuracy of force model and position control at nano-scale, some researchers have proposed local scan observation method to realize real-time feedback in order to realize online correction. Other researchers have proposed an observing method while manipulating the double tip mechanism to improve the manipulation efficiency. However, it is still difficult to overcome many

1.4 Application Characteristics and Problems of AFM Based Nano-manipulation

25

uncertainties, such as the position of the action point and the magnitude of the action force involved in the tip point contact manipulation, which cannot meet the requirements of efficient and stable nano-manipulation. The main technical challenges for stable and accurate AFM based nano-manipulation are as follows: • It is difficult to localize the tip accurately in task space; • Lack of real-time feedback information on manipulation status; • The tip manipulation point is uncertain. In view of the uncertainties and instability in AFM nano-manipulation, this book combines stochastic idea control method with landmark localization in macro-robot, and carries out theoretical methods and experimental research on precise localization of the tip using the landmark and concurrent manipulation of virtual hands without precise sensors. The main contents of this book include the following aspects: (1) The AFM working mode and the progress of AFM based robotic nano-manipulation methods. (2) Aiming at the problem of thermal drift in the system, the AFM image reconstruction algorithm based on thermal drift compensation model is studied. Combining the scanning images from two different angles of the same sample, the thermal drift model is established by using the relatively high accuracy of its fast scanning direction, and the offset vector is defined to achieve reconstruction of thermal drift image, reduce the influence of thermal drift in the system and improve the image accuracy. (3) As for the problem of tip morphology effect (such as broadening effect) in nano-manipulation environment maps, efficient tip modeling method and precise reconstructing method of environmental map based on tip morphology are studied to establish a more realistic manipulation environment model. (4) Aiming at the uncertainty of tip position caused by non-linearity and system error in the AFM based manipulation, the PI model, creep model, thermal drift model and landmark observation error model are established. On the foundation of probabilistic robotics theory, the environmental landmark map and local scan observation are constructed, and the probabilistic statistics-based tip motion modelling method is studied. Meanwhile, tip location technology based on stochastic method is implemented. (5) In view of the uncertainty of tip manipulation state, based on the landmark location, the virtual nano-hand manipulation method and monitoring interface are constructed by using the theory of robot manipulation trajectory planning. The instability of single tip manipulation is effectively solved, and a realizable technical approach is provided for the robotic nano-manipulation with fixed gesture. (6) On the foundation of the theoretical research of tip localization and virtual hand nano-manipulation, an AFM nano-manipulation platform is designed and constructed. The calibration scheme of model parameters is designed

26

1

Introduction

systematically, and a lot of related experimental research and validation of the method are carried out. The experimental results illustrate that the abovementioned research methods can effectively improve the efficiency and accuracy of nano-manipulation. The theoretical and experimental research work of this book has certain guiding significance for the theoretical research method of AFM based nano-manipulation, and lays a foundation for the development of AFM based robotic nano-manipulation technology.

References 1. Bharat, B.: Springer Handbook of Nanotechnology. Springer, GmbH Germany (2017) 2. Shi, L.Y.: Foundation of Nanotechnology. East China University of Science and Technology Press, Shanghai (2005) 3. Binning, G., Rohrer, H.: Scanning tunneling microscopy. Helv. Phys. Acta 55(6), 725–726 (1982) 4. Feynman, R.P.: There’s Plenty of Room at the Bottom: An Invitation to Enter a New Field of Physics[EB/OL]. In: Lecture at an 1959 APS meeting, The Archives, California Institute of Technology, pp. 60–66. See www.its.caltech.edu/feynman/plenty.html (1959) 5. Kubo, R.: Generalized cumulant expansion method. J. Phys. Soc. Jpn. 17(7), 1100–1120 (1962) 6. Taniguchi, N.: On the basic concept of nanotechnology. In: Proceedings of ICPE Tokyo, pp. 18–23 (1974) 7. Binning, G., Quate, C.F., Gerber, C.: Atomic force microscopy. Phys. Rev. Lett. 56, 930– 933 (1986) 8. Kroto, H.W., Heath, J.R., O’Brien, S.C., et al.: C60: Buckminsterfullerene. Nature 318 (6042), 162–163 (1985) 9. Eigler, D.M., Schweizer, E.K.: Positioning single atoms with a scanning tunnelling microscope. Nature 344(6266), 524–526 (1990) 10. Iijima, S.: Helical microtubules of graphitic carbon. Nature 354(6348), 56–58 (1991) 11. Hoyer, P.: Formation of a titanium dioxide nanotube array. Langmuir 12(6), 1411–1413 (1996) 12. Dufrene, Y.F.: Using nanotechniques to explore microbial surfaces. Nat. Rev. Micro. 2(6), 451–460 (2004) 13. Muller, D.J., Dufrene, Y.F.: Atomic force microscopy as a multifunctional molecular toolbox in nanobiotechnology. Nat Nano 3(5), 261–269 (2008) 14. Dupres, V., Alsteens, D., Wilk, S., et al.: The yeast Wsc1 cell surface sensor behaves like a nanospring in vivo. Nat. Chem. Biol. 5(11), 857–862 (2009) 15. Dague, E., Alsteens, D., Latgé, J.P., et al.: High-resolution cell surface dynamics of germinating aspergillus fumigatus conidia. Biophys. J. 94(2), 656–660 (2008) 16. Scheuring, S., Sturgis, J.N.: Chromatic adaptation of photosynthetic membranes. Science 309(5733), 484–487 (2005) 17. Engel, A., Muller, D.J.: Observing single biomolecules at work with the atomic force microscope. Nat. Struct. Mol. Biol. 7(9), 715–718 (2000) 18. Kirmizis, D., Logothetidis, S.: Atomic force microscopy probing in the measurement of cell mechanics. Int. J. Nanomed. 5(1), 137–145 (2010) 19. Lüth, H.: Semiconductor nanostructures: a new impact on electronics. Appl. Surf. Sci. 130, 855–865 (1998)

References

27

20. Rohrer, H.: Nanoengineering beyond nanoelectronics. Microelectr. Eng. s41–42(2), 31–36 (1998) 21. Rossum, M.V.: The future of microelectronics: evolution or revolution. Microelectron. Eng. 34(1), 125–134 (1996) 22. Dixson, R.G., Koening, R.G., Vorburger, T.V., et al.: Measurement of pitch and width samples with the NIST calibrated atomic force microscope. Proc. SPIE. 3332(420), 420–432 (1998) 23. Nelson-Thomas, C.M., Palmateer, S.C., Lyszczarz, T.M.: Metrology methods for the quantification of edge roughness. Proc. SPIE 3332(19), 19–29 (1998) 24. Zhao, X.Z., Chu, W., Theodore, et al.: Research on nanoscale linewidth measurement model and its algorithm based on AFM. J. Mechan. Eng. 40(4), 50–57 (2004) 25. Chu, W., Zhao, X.Z., Wang, L., et al.: Uncertainty in measurement of nanoscale linewidth using AFM. Acta Metrologicala Sinica 26(2), 120–124 (2005) 26. Xiao, Z.W., Zhao, X.Z.: AFM measurement of nanoscale linewidth by nanotube tips. J. Harb. Univ. Sci. Technol. 10(4), 65–68 (2005) 27. Xiao, Z.W., Zhao, X.Z., Chu, W., et al.: Study on measurement of line width by curve matching method in top line width. Metrol. Technol. 6, 11–14 (2006) 28. Li, H.B., Zhao, X.Z., et al.: Measurement of edge roughness, top surface and bottom surface roughness of AFM single crystal silicon step line. J. Nanjing Univ. Sci. Technol. (Nat. Sci.), 31(4), 478–481 (2007) 29. Li, H.B., Zhao, X.Z.: Measurement of linewidth roughness based on AFM. J. Mech. Eng. 44 (8), 227–232 (2008) 30. Li, H.B., Zhao, X.Z., Zhao, W.Q.: Measurement of line width and line width roughness based on equivalent area method. Acta Metrologicala Sinica 32(1), 16–19 (2011) 31. Carlsson, S.B., Junno, T., Montelius, L., et al.: Mechanical tuning of tunnel gaps for the assembly of single-electron transistors. Appl. Phys. Lett. 75(10), 1461–1463 (1999) 32. Maier, S.A., Kik, P.G., Atwater, H.A., et al.: Local detection of electromagnetic energy transport below the diffraction limit in metal nanoparticle plasmon waveguides. Nat. Mater. 2(4), 229–232 (2003) 33. Mokaberi, B., Requicha, A.A.G.: Drift compensation for automatic nanomanipulation with scanning probe microscopes. IEEE Trans. Autom. Sci. Eng. 3(3), 199–207 (2006) 34. Zheng, J., Zhucheng Chen, A., Liu, Z.: Atomic force microscopy-based nanolithography on silicon using colloidal Au nanoparticles as a nanooxidation mask. Langmuir 16(24), 9673– 9676 (2004) 35. Lü, J.H.: Nanomanipulation of extended single-DNA molecules on modified mica surfaces using the atomic force microscopy. Colloids Surf B Biointerfaces 39(4), 177–180 (2004) 36. Demers, L.M., Ginger, D.S., Park, S.J., et al.: Direct patterning of modified oligonucleotides on metals and insulators by dip-pen nanolithography. Science 296(5574), 1836–1838 (2002) 37. Lee, K.B., Lim, J.H., Mirkin, C.A.: Protein nanostructures formed via direct-write dip-pen nanolithography. J. Am. Chem. Soc. 125(19), 5588–5589 (2003) 38. Lee, K.B., Park, S.J., Mirkin, C.A., et al.: Protein nanoarrays generated by dip-pen nanolithography. Science 295(5560), 1702–1705 (2002) 39. Jin, X., Unertl, W.N.: Submicrometer modification of polymer surfaces with a surface force microscope. Appl. Phys. Lett. 61(6), 657–659 (1992) 40. Jung, T.A., Moser, A., Hug, H.J., et al.: The atomic force microscope used as a powerful tool for machining surfaces. Ultramicroscopy s42–44(92), 1446–1451 (1992) 41. Nie, H.Y., Motomatsu, M., Mizutani, W., et al.: Local modification of elastic properties of polystyrene-polyethyleneoxide blend surfaces. J. Vacuum Sci. Technol. B Microelectr. Nanometer Struct. Process. Measurement Phenomena 13(3), 1163–1166 (1995) 42. Heyde, M., Rademann, K., Cappella, B., et al.: Dynamic plowing nanolithography on polymethylmethacrylate using an atomic force microscope. Rev. Sci. Instrum. 72(1), 136– 141 (2001) 43. Brown, C.: Comparison between dynamic plowing lithography and nanoindentation methods. J. Appl. Phys. 91(1), 506–512 (2002)

28

1

Introduction

44. Cappella, B., Sturm, H., Weidner, S.M.: Breaking polymer chains by dynamic plowing lithography. Polymer 43(16), 4461–4466 (2002) 45. Schaefer, D.M., Reifenberger, R., Patil, A., et al.: Fabrication of two-dimensional arrays of nanometer-size clusters with the atomic force microscope. Appl. Phys. Lett. 66(8), 1012– 1014 (1995) 46. Irmer, B., Blick, R.H., Simmel, F., et al.: Josephson junctions defined by a nanoplough. Appl. Phys. Lett. 73(14), 2051–2053 (1998) 47. Gnecco, E., Bennewitz, R., Meyer, E.: Abrasive wear on the atomic scale. Phys. Rev. Lett. 88(21), 215501–215504 (2002) 48. Fang, T.H., Chang, W.J.: Effects of AFM-based nanomachining process on aluminum surface. J. Phys. Chem. Solids 64(6), 913–918 (2003) 49. Kato, Z., Sakairi, M., Takahashi, H.: Nanopatterning on aluminum surfaces with AFM probe. Surf. Coat. Technol. 169(22), 195–198 (2003) 50. Filho, H.D.F., Maurício, M.H.P., Ponciano, C.R., et al.: Metal layer mask patterning by force microscopy lithography. Mater. Sci. Eng., B 112(2–3), 194–199 (2004) 51. Li, X., Nardi, P., Baek, C.W., et al.: Direct nanomechanical machining of gold nanowires using a nanoindenter and an atomic force microscope. J. Micromech. Microeng. 15(3), 551– 556 (2004) 52. Cortes Rosa, J., Wendel, M., Lorenz, H., et al.: Direct patterning of surface quantum wells with an atomic force microscope. Appl. Phys. Lett. 73(18), 2684–2686 (1998) 53. Hyon, C.K., Choi, S.C., Hwang, S.W., et al.: Direct nanometer-scale patterning by the cantilever oscillation of an atomic force microscope. Appl. Phys. Lett. 75(2), 292–294 (1999) 54. Schumacher, H.W., Keyser, U.F., Zeitler, U., et al.: Nanomachining of mesoscopic electronic devices using an atomic force microscope. Appl. Phys. Lett. 75(8), 1107–1109 (1999) 55. Regul, J., Keyser, U.F., Paesler, M., et al.: Fabrication of quantum point contacts by engraving GaAs/AlGaAs-heterostructures with a diamond tip. Appl. Phys. Lett. 81(11), 2023–2025 (2002) 56. Coffey, D.C., Reid, O.G., Rodovsky, D.B., et al.: Mapping local photocurrents in polymer/ fullerene solar cells with photoconductive atomic force microscopy. Nano Lett. 7(3), 738– 744 (2007) 57. Douhéret, O., Swinnen, A., Bertho, S., et al.: High-resolution morphological and electrical characterisation of organic bulk heterojunction solar cells by scanning probe microscopy. Prog. Photovoltaics Res. Appl. 15(8), 713–726 (2007) 58. Moreno, F.S., Toca-Herrera, J.L.: The new future of scanning probe microscopy: Combining atomic force microscopy with other surface-sensitive techniques, optical microscopy and fluorescence techniques. Nanoscale 1(1), 40–49 (2009) 59. Jaquith, M., Muller, E.M., Marohn, J.A.: Time-resolved electric force microscopy of charge trapping in polycrystalline pentacene. J. Phys. Chem. B 111(27), 7711–7714 (2007) 60. Dante, M., Peet, J., Nguyen, T.Q.: Nanoscale charge transport and internal structure of bulk heterojunction conjugated polymer/fullerene solar cells by scanning probe microscopy. J. Phys. Chem. C 112(112), 7241–7249 (2008) 61. Douhéret, O., Lutsen, L., Swinnen, A., et al.: Nanoscale electrical characterization of organic photovoltaic blends by conductive atomic force microscopy. Appl. Phys. Lett. 89(3), 032107 (2006) 62. Han, D.H., Park, S.M.: Electrochemistry of conductive polymers. 32. Nanoscopic examination of conductivities of polyaniline films. J. Phys. Chem. B 108(37), 13921 (2004) 63. Yang, J.S.: Development Status and Prospects of Nano Environmental Protection Technology. Bull. Sci. Technol. 18(4), 340–343 (2002) 64. Sui, Y.C., Saniger, J.M.: Characterization of anodic porous alumina by AFM. Mater. Lett. 48(3), 127–136 (2001) 65. Lee, W., Ji, R., Gösele, U., et al.: Fast fabrication of long-range ordered porous alumina membranes by hard anodization. Nat. Mater. 5(9), 741–747 (2006)

References

29

66. Geim, A.K., Novoselov, K.S.: The rise of graphene. Nanosci. Technol. Collect. Rev. Nat. J. 11–19 (2009) 67. Giesbers, A.J.M., Zeitler, U., Neubeck, S., et al.: Nanolithography and manipulation of graphene using an atomic force microscope. Solid State Commun. 147(9–10), 366–369 (2008) 68. Tapasztó, L., Dobrik, G., Lambin, P., et al.: Tailoring the atomic structure of graphene nanoribbons by scanning tunnelling microscope lithography. Nat. Nanotechnol. 3(7), 397– 401 (2008) 69. Weng, L., Zhang, L., Chen, Y.P., Rokhinson, L.P.: Atomic force microscope local oxidation nanolithography of graphene. Appl. Phys. Lett. 93(9), 093107 (2008) 70. Gnecco, E., Bennewitz, R., Gyalog, T., et al.: Friction experiments on the nanometre scale. J. Phys.: Condens. Matter 13(31), 619–642 (2001) 71. Krylov, S.Y., Jinesh, K.B., Valk, H., Dienwiebel, M., Frenken, J.W.M.: Thermally induced suppression of friction at the atomic scale. Phys. Rev. E 71(6), 065101 (2005) 72. Pillai, J.K.B.: Atomic-Scale Friction: Thermal Effects and Capillary Condensation. The Kamerlingh Onnes Laboratory, Leiden University, Kerala, India (2006) 73. Mo, Y., Turner, K.T., Szlufarska, I.: Friction laws at the nanoscale. Nature 457(7233), 1116– 1119 (2009) 74. Knippenberg, M.T., Mikulski, P.T., Harrison, J.A.: Effects of tip geometry on interfacial contact forces. Modell. Simul. Mater. Sci. Eng. 18(3), 034002 (2010) 75. Kielpinski, D., Monroe, C., Wineland, D.J.: Architecture for a large-scale ion-trap quantum computer. Nature 417(6890), 709–711 (2002) 76. Pioda, A., Kicin, S., Ihn, T., et al.: Spatially resolved manipulation of single electrons in quantum dots using a scanned probe. Phys. Rev. Lett. 93(21), 216801 (2004) 77. Yang, X.G., Yang, X.: Atomic force microscopy. Chemical Industry Press, Beijing (2012) 78. Dong, X.K.: Research on Imaging Methods of High Speed Atomic Force Microscopy. Nankai University, Tianjin (2012) 79. Dong, X.K., Fang, Y.C., Zhang, X.B.: Generalized predictive control and imaging of atomic force microscope system. Control Theory Appl. 32(08), 1058–1063 (2015) 80. Fang, Y.C., Zhang, Y.D., Jia, N.: Learning control system for advanced scanning mode of atomic force microscope. Control Theory Appl. 27(05), 557–562 (2010) 81. Li, M., Liu, L.Q., Xi, N., Wang, Y.: Advances in cell surface ultrastructural imaging and mechanical properties measurement based on AFM. Biochem. Biochem. Progr. Phys. Phys. 42(8), 697–712 (2015) 82. LeMieux, M.C., Roberts, M., Barman, S., et al.: Self-sorted, aligned nanotube networks for thin-film transistors. Science 321(5885), 101–104 (2008) 83. Whitesides, G.M., Mathias, J.P., Seto, C.T.: Molecular self-assembly and nanochemistry: a chemical strategy for the synthesis of nanostructures. Science 254(5036), 1312–1319 (1991) 84. Whitesides, G.M., Grzybowski, B.: Self-assembly at all scales. Science 295(5564), 2418– 2421 (2002) 85. Arai, Y., Yasuda, R., Akashi, K., et al.: Tying a molecular knot with optical tweezers. Nature 399(6735), 446–448 (1999) 86. Ashkin, A., Dziedzic, J.M.: Optical trapping and manipulation of viruses and bacteria. Science 235(4795), 1517–1520 (1987) 87. Balijepalli, A., LeBrun, T.W., Gupta, S.K.: A flexible system framework for a nanoassembly cell using optical tweezers. ASME 2006 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Philadelphia, Pennsylvania, USA, pp. 333–342 (2006) 88. Grier, D.G.: A revolution in optical manipulation. Nat. Photon 424(6950), 810–816 (2003) 89. Ichikawa, M., Kubo, K., Murata, S., et al.: Single cell manipulation by using tilt controlled optical tweezers. International Symposium on Micro-Nano Mechatronics and Human Science pp. 316–321 (2007) 90. Moffitt, J.R., Chemla, Y.R., Smith, S.B., et al.: Recent advances in optical tweezers. Annu. Rev. Biochem. 77(1), 205–228 (2008)

30

1

Introduction

91. Liu Y.X., Yu M.: Fiber Optical Tweezers for Cell Manipulation and Force Sensing. Conference on Lasers and Electro-Optics, pp. 1–2 (2007) 92. Suzuki, S., Yamanashi, T., Tazawa, S., et al.: Quantitative analysis of DNA orientation in stationary AC electric field using fluorescence anisotropy. Industry Applications Conference, Thirtieth IAS Annual Meeting, vol. 2, pp. 1374–1382 (1995) 93. Yamamoto K., Akita S., Nakayama Y.: Orientation of carbon nanotubes using electrophoresis. Japan. J. Appl. Phys. 35(35), 917–918 (1996) 94. Fiedler, S., Shirley, S.G., Schnelle, T., et al.: Dielectrophoretic sorting of particles and cells in a microsystem. Anal. Chem. 70(9), 1909–1915 (1998) 95. Yamamoto, K., Akita, S., Nakayama, Y.: Orientation and purification of carbon nanotubes using ac electrophoresis. J. Phys. D Appl. Phys. 31(8), L34–L36 (1998) 96. Chen, X.Q., Saito, T., Yamada, H., et al.: Aligning single-wall carbon nanotubes with an alternating-current electric field. Appl. Phys. Lett. 78(23), 3714–3716 (2001) 97. Dewarrat, F., Calame, M., Schönenberger, C.: Orientation and positioning of DNA molecules with an electric field technique. Single Molecules 3(4), 189–193 (2002) 98. Yu, C.P., Hao, C.Z., Wu, M.C.: A novel optoelectronic tweezer using light induced dielectrophoresis. In: IEEE/LEOS International Conference on Optical MEMS, pp. 8–9 (2003) 99. Fung, C.K.M., Wong, V.T.S., Chan, R.H.M., et al.: Dielectrophoretic batch fabrication of bundled carbon nanotube thermal sensors. IEEE Trans. Nanotechnol. 3(3), 395–403 (2004) 100. Dong, L.X., Arai, F., Fukuda, T.: 3-D nanorobotic manipulation of nanometer-scale objects. J. Robot. Mechatron. 13(2), 146–153 (2001) 101. Eichhorn, V., Carlson, K., Andersen, K.N., et al.: Nanorobotic manipulation setup for pick-and-place handling and nondestructive characterization of carbon nanotubes. In: IEEE/ RSJ International Conference on Intelligent Robots and Systems, San Diego, USA, pp. 291– 296 (2007) 102. Kim, K.S., Lim, S.C., Lee, I.B., et al.: In situ manipulation and characterizations using nanomanipulators inside a field emission-scanning electron microscope. Rev. Sci. Instrum. 74(9), 4021–4025 (2003) 103. Junno, T., Deppert, K., Montelius, L., et al.: Controlled manipulation of nanoparticles with an atomic force microscope. Appl. Phys. Lett. 66(26), 3627–3629 (1998) 104. Martin, M., Roschier, L., Hakonen, P., et al.: Manipulation of Ag nanoparticles utilizing noncontact atomic force microscopy. Appl. Phys. Lett. 73(11), 1505–1507 (1998) 105. Resch, R., Bugacov, A., Baur, C., et al.: Manipulation of nanoparticles using dynamic force microscopy: simulation and experiments. Appl. Phys. A Mater. Sci. Process. 67(3), 265–271 (1998) 106. Decossas, S., Patrone, L., Bonnot, A.M., et al.: Nanomanipulation by atomic force microscopy of carbon nanotubes on a nanostructured surface. Surf. Sci. 543(1–3), 57–62 (2003) 107. Yang, D.Q., Sacher, E.: Local surface cleaning and cluster assembly using contact mode atomic force microscopy. Appl. Surf. Sci. 210(3–4), 158–164 (2003) 108. Müller, M., Fiedler, T., Gröger, R., et al.: Controlled structuring of mica surfaces with the tip of an atomic force microscope by mechanically induced local etching. Surf. Interface Anal. 36(2), 189–192 (2004) 109. Pumarol, M.E., Miyahara, Y., Gagnon, R., et al.: Controlled deposition of gold nanodots using non-contact atomic force microscopy. Nanotechnology 16(8), 1083–1088 (2005) 110. Sugimoto, Y., Abe, M., Hirayama, S., et al.: Atom inlays performed at room temperature using atomic force microscopy. Nat. Mater. 4(2), 156–159 (2005) 111. Zhao, Q.L., Dong, S., Zhang, X.L., et al.: Nanomachining Based on AFM. J. Mech. Eng. 36 (11), 64–69 (2000) 112. Luo, X.C., Liang, Y.C., Dong, S.: Molecular dynamics simulation technology of single crystal aluminum nano-cutting process. China Mech. Eng. 11(8), 860–862 (2000) 113. Fu, H.N.: Experimental research on nano-cutting. China Mech. Eng. 12(8), 965–968 (2001)

References

31

114. Lv, J.H., Wu, S.Y., Wang, G.H., et al.: Nanopositioning and cutting of DNA single molecules. J. Chin. Electr. Microsc. Soc. 22(3), 185–188 (2003) 115. Zhang, Y.J., Li, P., Hu, Y.Z., et al.: Manipulation and shearing of carbon nanotubes. Chin. Sci. Bull. 47(14), 1066–1070 (2002) 116. Tian, X.J., Wang, Y.C., Xi, N., et al.: Accurate assembly and electrical connection of single carbon nanotubes prepared by nanoelectronic devices. J. Chin. Electr. Microsc. 25(6), 490– 493 (2006) 117. Tian, X.J., Wang, Y.C., Xi, N., et al.: Pulsed airflow alignment and AFM operation of SWCNT. Chin. Sci. Bull. 53(2), 251–256 (2008) 118. Zhang, C.L., Liu, L.Q., Wang, Y.C., et al.: Three-dimensional controllable operation method of single molecule virus. Chin. Sci. Bull. 58(15), 1456–1462 (2013) 119. Xie, H., Onal, C., Rgnier, S., et al.: Atomic force microscopy based nanorobotics: modelling, simulation, setup building and experiments. Springer Publishing Company, Incorporated, Berlin, Heidelberg (2011) 120. Xie, H., Régnier, S.: High-efficiency automated nanomanipulation with parallel imaging/ manipulation force microscopy. IEEE Trans. Nanotechnol. 11(1), 21–33 (2012) 121. Xie, H., Onal, C., Régnier, S., et al.: Nanomechanics of AFM based nanomanipulation. In: Atomic Force Microscopy Based Nanorobotics, pp. 87–143 (Springer, Berlin, Heidelberg, 2011) 122. Seeger, A.: Surface Reconstruction from AFM and SEM Images. University of North Carolina, Chapel Hill, USA (2004) 123. Xu, K.M., Kalantari, A., Qian, X.P.: Efficient AFM-based nanoparticle manipulation via sequential parallel pushing. IEEE Trans. Nanotechnol. 11(4), 666–675 (2012)

Chapter 2

AFM Based Robotic Nano-manipulation

Abstract AFM is a tool for observation and manipulation using the interaction force between the tip and the sample on the nano-scale. It has the advantages of high resolution and motion precision, controllable and repeatable manipulation mode, mechanical manipulation mechanism, and can be used to various environment, which is currently the most promising nano-manipulation tool. It has gained a lot of practical applications in the field of nano research. This chapter introduces the basic structure, imaging principle and working mode of AFM. It also introduces several kinds of AFM robotic based nano-manipulation, and represents stochastic approach based nano-manipulation method.







Keywords AFM Static image Augmented reality location Stochastic method Virtual nano-hand




2.1





Local scan
Landmark

AFM Introduction

The experimental platform of this book is the scanning probe microscope Veeco Dimension 3100, which can work in a variety of modes, such as STM mode, AFM mode, EFM mode and so on. The basic structure and working principle of AFM are introduced below. The working principle of AFM is shown in Fig. 2.1a, b. A laser beam is irradiated on the cantilever of the AFM scanning probe, and then reflected on the four-quadrant photodetector. When the tip scans the sample surface, the atoms on the tip interact with the atoms on the sample surface (repulsion or attraction), which makes the cantilever deflect upward and downward along with the contour of the sample surface, and the lateral torsional motion (contact mode) or the cantilever amplitude change (tapping mode). These movements of the cantilever shift the position of the laser spot which is signal on the four-quadrant photodetector, thereby altering the output electrical signal strength of the four-quadrant photode-

Authors: Shuai Yuan, Lianqing Liu, Jing Hou, Lixue Yin. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_2

33

34

2 AFM Based Robotic Nano-manipulation

Feedback Loop Maintains Constant Cantilever Deflection

Split-diode photodetector

Controller Electronics D

Adjustable mirror

Repulsive regime

Force

Lazer diode, collimator & lens

A Non-contact regime

0 Lazer path

C

Fixed Mirror

B Attractive regime

Probe Distance from Sample (z distance) (c) AFM tip force-distance curve

Lens Mirror

Lens

y z

Camera objective lens

Sample

x

xyz PZT tube scanner

Cantilever holder

Motorized stage

(b) Magnification of the partial area in the left image

(a) Diagram of AFM structure

Fig. 2.1 Working principle of AFM

tector. The fluctuation of the surface topography can be detected by this variation, and then the surface topography of the sample can be obtained.

2.1.1

Analysis of AFM Atomic Force-Distance Curve

AFM is used to measure the surface topography of the sample. The relationship between atomic force applied to the tip and the distance between the tip and the substrate is represented as follows. Atomic force, also known as van der Waals force, depend on the distance between the tip and the sample. As shown in Fig. 2.1c, region A indicates that the distance between the tip and the sample surface is large, and the interaction force is almost zero; as the tip approaches the sample surface, a weak attraction appears between the tip and the sample when it enters region B. This attraction increases gradually until the tip approaches the sample to point C, which is due to the electron cloud between the atoms approaching a certain degree, and the mutual exclusion will occur, thus weakening the attractiveness. When the distance between the tip and the sample is close to several angstroms (10−10m), the length of the bond is the same order of magnitude as that of the atom, the attraction and repulsion between atoms are balanced, and the interaction force between the tip and the sample is zero. As the tip continues to

2.1 AFM Introduction

35

approach the sample, the van der Waals force becomes repulsive when it enters the D region, and the atoms begin to contact theoretically. In the repulsive region, van der Waals force is extremely sensitive to distance, a small change in distance will cause a larger change in van der Waals force, which will lead to severe bending deformation of the cantilever. If the tip keeps approaching the sample, the distance between the tip atom and the sample atom will not decrease continuously, but will change the surface morphology of the sample. This is also the basic principle of mechanical nano-lithography on the sample surface using AFM.

2.1.2

Three Work Modes of AFM

The scanning mode of AFM includes constant force mode and constant height mode. Constant force mode adjusts the deflection degree of cantilever by feedback control loop, so as to ensure that the force between the sample and the tip is constant. When scanning along X and Y directions, the moving distance of the scanner in Z direction is recorded to obtain the surface image of the sample. In this mode, the force between the tip and the sample surface is regulated by the up-down motion of the tip, so the measured height of the sample is accurate and suitable for the surface analysis of the material. In the constant height mode, the height of piezoelectric ceramics is fixed, and the surface image is obtained by directly measuring the deflection signal of the cantilever beam in scanning. This model is sensitive to the change of sample height, which can scan samples quickly. It is suitable for the observation of molecules and atoms. However, it requires the surface of samples to be flat and without big bumps. The constant force mode of AFM is a common scanning mode, which includes contact mode, non-contact mode and tapping mode. The contact mode is characterized by slight contact between the tip and the sample surface and sliding on the surface. The interaction force between the tip and the sample is the repulsive force between the two contacting atoms, which is about 10−11 − 10−8 N. Contact mode usually relies on this repulsive force to obtain stable and high resolution surface topography images. However, due to the sliding of the tip on sample surface, the tip is vulnerable to wear and tear, and the soft sample (such as biological cells) will be damaged, which is the shortcoming of the contact mode. In the non-contact mode, the tip never contacts the sample surface. The tip scans along the sample at a height of 5–20 nm above the sample surface. The distance between the tip and the sample is controlled by keeping the resonant frequency or amplitude of the cantilever beam constant. In this model, the interaction between the sample and the tip is van der Waals force. Because the attraction is greater than the repulsion, the sensitivity is higher than that of the contact mode, but the resolution is lower than that of the contact mode. The non-contact mode usually works in the extremely dry condition of the sample surface. If the water membrane adsorbed naturally on the sample surface is thicker, the needle tip will easily be adsorbed to the water membrane, resulting in unstable feedback control and the possibility of scratching the sample. Therefore, this mode has been rarely used at present.

36

2 AFM Based Robotic Nano-manipulation

The tapping mode is driven by a piezoelectric ceramic plate on the tip cantilever to make the cantilever resonate in Z direction with a certain high frequency and tens of nanometers amplitude. The feedback loop keeps the Root Mean Square (RMS) value of the resonance signal constant, which is detected by the four quadrant photoelectric detector, and the cantilever scans the sample at a constant amplitude. The amplitude and phase of the cantilever are controlled by adjusting the distance between the sample and the tip, and the surface topography of the sample is obtained by recording the movement of the scanner in Z direction. Because of the high frequency vibration of the cantilever, the time of frequent contact between the tip and the sample is quite short, and there is enough amplitude to overcome the adhesion force between the sample and the tip. Tapping mode is the most commonly used working modes of AFM, which can be used in any environment (vacuum, atmosphere and liquid). This model is especially suitable for soft, brittle and adhesive samples without breaking them. Therefore, it is widely used in the structural study of polymer and biological macromolecules.

2.2

AFM Based Robotic Nano-manipulation

In the late 1990s, some scholars began to introduce robotic technology (such as real-time feedback, master-slave operation technology) into nano-manipulation. Robotic nano-manipulation method and system implementation technology have made some progress, which can be roughly divided into three stages as follows.

2.2.1

Static Image Based Offline Nano-manipulation

Requicha et al. of the University of Southern California, USA, used the idea of interactive control in robotics to redevelop AFM. The nano-manipulation system is shown in Fig. 2.2a [1]. The system uses mouse click to set the tip trajectory on a static AFM image to perform nano-manipulation, and reflects the real-time deformation information of the cantilever during manipulation procedure, which makes it difficult to guide the next nano-manipulation on this basis. In order to improve the efficiency of AFM nano-manipulation, Sitti Research Group [2, 3] of Tokyo University, Japan, established a VR operating interface and introduced a force feedback device with a degree of freedom to let the operator feel the nano-manipulation force on the Z direction of the tip. The system structure is shown in Fig. 2.2b and the manipulation results are shown in Fig. 2.2c, d. Sitti classifies the system as a teleoperation robot at nano-scale. The mechanical model of bi-directional transmission from macro to micro is analyzed, and the force feedback teleoperation control is studied. The visual feedback and force feedback generation in nano-manipulation are discussed preliminarily. Meanwhile, the Guthold Research Group of the University of North Carolina [4] provides a three-freedom-degree force feedback handle and a

2.2 AFM Based Robotic Nano-manipulation

37 Microscope Image

(a)

(b)

Haptic Manipulator

AFM System Stereo vision

Real time 3D graphics

Visual feedback

(c)

Microscope controller

Graphics Engine and Host processor

Phantom controller

Atomic force microscope Tip and sample

(d)

(e)

Touch & manipulations

Fig. 2.2 Early nano-manipulation system based on AFM. © [2019] IEEE. Reprinted, with permission, from Requicha et al. [1], Sitti et al. [3] and Guthold et al. [4]

three-dimensional visual graphics feedback interface for the operator, as shown in Fig. 2.2e. In nano-manipulation, the three-dimensional visual feedback graphics can be tilted and rotated by themselves, allowing the operator to observe the nano-manipulated object from different perspectives. At the same time, the real-time nano-manipulation force in two-dimensional space can be sensed by the force feedback handle. Problems in the first stage of development: in the procedure of nano-manipulation, because the contact area between the tip and the object being manipulated is very small, it can be considered as a point. Therefore, the relative position of the tip and the object being manipulated, the magnitude and direction of the manipulation force, the manipulation procedure and other unknown factors have a great impact on the manipulation results. Meanwhile, the human-computer interaction interface provided in this stage is only superimposed on the static image. The dynamic tip does not provide real-time visual feedback. The force signal of the force feedback handle to the tip is simply magnified, which cannot reflect the real manipulation information. Therefore, it is necessary to re-scan an image to verify the manipulation result. The control flow of the nano-manipulation is shown in Fig. 2.3. The operation process is

38

2 AFM Based Robotic Nano-manipulation

Enviroment model

Force information

Nanomanipulation Environment

Real-time operation result Static image

+

Dynamic tip

Force feedback device

AFM system

Offline Operation command

Fig. 2.3 Previous control flow chart of AFM nano-manipulation system

scanning ! planning ! manipulation ! scanning, which is inefficient and difficult to complete complex nano-manipulation tasks.

2.2.2

Augmented Reality Based Robotic Nano-manipulation

In order to provide real-time state information of nano-manipulation, researchers at Michigan State University and Shenyang Institute of Automation, Chinese Academy of Sciences [5–10] established a real-time nano-manipulation feedback interface based on model compensation and force analysis of nano-manipulation, and realized real-time control of three-dimensional motion of tip by means of force feedback handle. The nano-manipulation system is shown in Fig. 2.4. Because optical microscopy cannot be used to observe the nano-manipulation process in real-time at nano-scale, and AFM cannot complete manipulation and imaging simultaneously, it is impossible to realize real-time visual feedback based on AFM scanning image. Augmented reality environment is a graphical simulation of nano-manipulation results through off-line model. It is a visual feedback interface based on the pre-acquired environment model, real-time manipulation information and kinematics model of nano-objects. In the second stage, the control flow of nano-manipulation is shown in Fig. 2.5. There are many uncertainties in the model compensation, such as the non-linearity of PZT, the thermal drift of the system and other uncertainties, which lead to errors in tip localization, the inaccurate model of nano-scale forces in the nano-manipulation procedure, and thus the credibility of visual feedback is not high. So it is necessary to re-scan an image for manipulation verification. Although the verification reliability of scanning image is relatively high, the efficiency is low.

2.2 AFM Based Robotic Nano-manipulation

39

Fig. 2.4 Augment reality based nano-manipulation system using AFM

Environment model

Real-time operation result

Object Kinematics model

Force information

Augment reality environment

Force feedback device

AFM system

Offline operation command

Fig. 2.5 Control flow chart of real-time visual feedback for Augment reality interface

40

2.2.3

2 AFM Based Robotic Nano-manipulation

Local Scan Based Nano-manipulation Using Landmark Observation

In order to solve low reliability of visual feedback and thermal drift of the system, real-time visual feedback based on local scan and tip localization method based on landmark observation are developed in this stage [11]. The nano-operation system is shown in Fig. 2.6. The system ensures the high reliability of real-time visual feedback and the accuracy of absolute localization of tip, and improves the efficiency of nano-manipulation. Real-time visual feedback based on local scan is a fast observation of the manipulation area in actual manipulation of the AFM. Online correction of visual feedback can be realized without interrupting the nano-manipulation, and the nano- manipulation interface is updated in real time. The feature morphology on the sample surface is marked as a landmark for tip localization. The coordinate system representing the tip position is changed from the Descartes coordinate system based on distance to the landmark coordinate system based on feature. The drift distance between the tip and the sample surface caused by the thermal drift of the real-time monitoring system can effectively overcome the influence of thermal drift on the precise localization of the tip. The third stage of the nano-manipulation control flow of AFM is shown in Fig. 2.7. The visual feedback of the local scan still has the uncertainty of the landmark position in the environmental map. The broaden effect exists when the tip locally scanning the landmark [12], and the deflection of the scanning line will also

Main controller

User interface imaging AFM controller

Linux PC

SAM

DAQ

AFM control scanner

Active tip

Windows PC Tactile

User interface

Fig. 2.6 AFM based manipulation system with real-time task space feedback

2.2 AFM Based Robotic Nano-manipulation

Environmental model

Partial scan Kalman diagnostic

41

Real-time force information

Active probe controller

AFM system Real-time visual feedback interface

Augmented reality environment

Force feedback device

Operation command

Fig. 2.7 Control flow chart of real-time feedback in task space

cause the error of the real-time feedback result when the scanning line observes different areas of the landmark. Meanwhile, real-time visual feedback can only show the position of the maneuvered object, but the controllability of manipulation results is not high. It is difficult to control the moving path of the object according to the user’s requirements. As shown in Fig. 2.8, multiple operations are needed to meet the requirement, which obviously cannot satisfy the requirement of high-efficiency nano-manipulation. The local area in Fig. 2.8b is enlarged. A represents the real position of the particle obtained from the local scan, and B represents the position of the nanoparticle displayed on the interface.

A

(a) Manipulating the nanoparticle in the

(b) Real-time display of

direction of the arrow

nano-manipulation result

B

Fig. 2.8 Manipulation of nano-particle with radius of 175 nm on polycarbonate surface

42

2 AFM Based Robotic Nano-manipulation

(1) Real-Time Visual Feedback An online visual feedback error correction method based on local scan is proposed. In nano-manipulation, in order to improve the reliability of visual feedback, the traditional method is to interrupt nano-manipulation and re-scan the image, and use the imaging results to repair the false visual feedback, which often takes several minutes and reduces the efficiency of AFM nano-manipulation. In the practical AFM manipulation, only the surface topography of the local area will change because of the interaction of the tip, so it is not necessary to scan the whole image. Based on this, it is proposed that online correction of visual feedback can be achieved only by scanning the local operation area quickly without interrupting nano-manipulation. In the process of operation, Kalman filter is used to determine when local scan is needed according to force feedback information. As shown in Fig. 2.7, when force feedback information exceeds the set threshold, fast local scan is started and the nano- manipulation interface is updated in real time. (2) Tip Localization Based on Landmark Thermal drift of the system causes drift between the tip and the sample surface, and it is difficult to detect this change using sensors. Aiming at this situation, a task space position feedback method based on landmark is proposed. The core idea of this method is to mark the feature morphology on the sample surface as a landmark, and change the coordinate system describing the position of the tip from Descartes coordinate system based on distance to the feature-based landmark coordinate system. This localization method avoids the nonlinear mapping calculation from PZT voltage to displacement, and because the landmark is in the same coordinate system as the nano-object is manipulated (thermal drift will cause the landmark to have the same positional drift as the object being operated), it can realize servo of AFM tip to thermal drift, effectively overcome the influence of thermal drift on the precise tip localization.

2.3

Stochastic Approach for AFM Based Robotic Nano-manipulation

In AFM based nano-manipulation, there are many uncertainties, including PZT non-linearity and system thermal drift. In addition, there are other uncertainties, such as the electronic noise of the control system and the vibration of the working environment, which also affect the localization and manipulation result of the tip. Compared with the macro-environment, the effect of these uncertainties on the micro/nano scale is more significant, and the realization of detection compensation is more challenging. In view of the probabilistic and statistical characteristics of these uncertainties, a system framework based on the idea of probability is firstly proposed in this book, which adds a new tip localization and manipulation module based on stochastic approach. The module realizes real-time feedback control of the

2.3 Stochastic Approach for AFM Based Robotic Nano-manipulation

43

tip position in task space, and reduces the thermal drift and other uncertain factors in the system due to PZT non-linearity. On the foundation of this, the concept of virtual nano-hand is proposed to realize multi-point concurrent operation of tips based on probability, which improves the efficiency of nanometer observation and manipulation.

2.3.1

Precision Analysis of AFM Tip Driver

The control modes of the PZT driver of the AFM tip are divided into two categories: closed-loop control and model-based feed-forward control. Commonly used closed-loop sensors are strain gauges, capacitive sensors or optical sensors as displacement feedback sensors. The advantage of closed-loop control method is that it has high absolute localization accuracy and can reach nanometer level. The disadvantages are: phase lag may occur in dynamic localization; hysteresis of PZT actuator and noise of system may lead to instability of feedback control system and affect localization accuracy; high cost is caused due to use of high-precision micro-displacement sensor. The principle of model-based feedforward control is to use a mathematical model to represent the hysteresis characteristics of PZT actuator, calculate and solve the inverse model, and then apply the model as a feedforward controller to PZT actuator for realizing the linear output of the actuator. The advantages of this control method are good robustness, fast response, suitable for dynamic localization and tracking, but low localization accuracy. Because the AFM tip is dynamic localized in scanning image and nano-manipulation, the model-based feedforward control is more suitable. Aiming at the problem of low localization accuracy, this system adopts the landmark observation strategy based on stochastic approach (introduced later) to improve the accuracy of tip localization. This method has the advantages of low cost, strong anti-interference ability and easy realization. Based on the existing research on PZT control, aiming at the asymmetric characteristics of PZT motion, this book establishes PZT asymmetric model by improving PI operator to improve the accuracy of tip position control [13, 14].

2.3.2

Real-Time Tip Localization Analysis in Task Space

In the tip localization of AFM nano-manipulation, the tip localization strategy based on stochastic approach is a feedback control method [15–17], which is based on the framework of new technology. The control block diagram is shown in Fig. 2.9. Compared with common feedback control systems, this method has two differences:

44 Fig. 2.9 Control flow chart of nano-manipulator based on stochastic approach for tip location and path planning

2 AFM Based Robotic Nano-manipulation Manipulation task Tip path planning

AFM Tip motion Tip motion controller

AFM actuator

Tip Localization

Tip position estimation filter

Tip motion model Landmark observation model

Local scan observation

(1) The Data Source of the Tip Localization Feedback Loop is Divided Into Two Parts These two parts include position estimation of tip motion model and location estimation based on landmark observation. The tip motion model can estimate the position of the tip in real time (high frequency response), and its error will increase with time and tip motion in the feedback loop. The uncertainty distribution of the tip position can be reduced by compensating the optimal estimation of Kalman filter based on landmark observation. However, the frequency response of location estimation based on landmark observation is low because of the time spent on local scan when the tip is used for landmark observation. (2) Tip Trajectory Planning Module is a Part of Tip Localization Feedback Loop The function of the tip trajectory planning module is to control the uncertainty of the tip position within the allowable range so as to realize the effective AFM nano-manipulation. Since the landmark observation depends on tip motion, the trajectory planning module of the tip motion is needed for the landmark observation before the tip moves to the target position. This method marks the feature on sample surface as a landmark and estimates the position of the tip in real time in task space coordinate system using the landmark. It represents the position uncertainty of the tip in task space with probability distribution. By establishing the motion model of the tip and combining the observation model based on local scan, the optimal position of the tip is estimated by Kalman filter for performing real-time tip localization with high precision and high resolution in manipulation.

2.3 Stochastic Approach for AFM Based Robotic Nano-manipulation

2.3.3

45

AFM Based Nano-manipulation Using Virtual Nano-hand

On the basis of tip localization, nano-manipulation is carried out by virtual nano-hand. The method is to plan the single push manipulation of the tip into multiple manipulations, and form a virtual nano-hand to realize the stable operation of the controlled object. Figure 2.10a, b show that the tip is in point contact with the object to be operated. The manipulation results are unstable because of the uncertainty of the tip position relative to the nanoparticle in a single operation. However, a new strategy is proposed in this book, which is to form a virtual nano-hand after several concurrent operations to obtain a relatively stable manipulation result [18, 19], as shown in Fig. 2.10c, d. Figure 2.11 is a schematic diagram of stable operation result obtained by traditional nano-rod operation method and virtual nano-hand manipulation method. On the basis of tip localization, the tip operation flow based on virtual nano-hand is shown in Fig. 2.12. Figure 2.12 shows that the tip needs to be positioned before nano-manipulation, and then the AFM based nano-manipulation is performed through path planning design based on virtual nano-hand.

Position set of nanoparticle after manipulation Nanoparticle initial position Tip AFM tip Nanoparticle

Nanoparticle Tip position with uncertainty

(a) Diagram of tip manipulation of the nanoparticle (b) Diagram of traditional tip manipulation of the nanoparticle Position set of nanoparticle after manipulation Nanoparticle initial position

Virtual Nano-hand

Nanoparticle

(c) The tip Manipulates the nanoparticle with multiple times concurrently

(d) Diagram of tip manipulation of the nanoparticle with forming a virtual nano-hand

Fig. 2.10 Schematic diagram of virtual nano-hand manipulation

46

2 AFM Based Robotic Nano-manipulation Nanorod final position set Nano-rod

AFM tip Initial position Tip position with uncertainty (a) Diagram of traditional tip manipulation of the nano-rod Initial position

Nanorods are constrained within a smaller region after manipulation

Virtual nano-hand

(b) Manipulation of the nano-rods with multiple times

(c) Diagram of virtual nano-hand manipulation

Fig. 2.11 Diagram of nano-rod manipulation using virtual nano-hand

Nano-manipulation task Virtual nano-hand composition and motion trajectory planning

Position estimation of the operated object based on probability filter

Tip position estimation based on probability filter

Nano-object motion Virtual nano-hand motion controller

AFM

Nanoobject

Tip motion model Observation model

observation based on local scan

Fig. 2.12 Flow chart of nano-manipulation control using virtual nano-hand

References 1. Requicha, A.A.G., Baur, C., Bugacov, A., et al.: Nanorobotic assembly of two-dimensional structures. In: IEEE International Conference on Robotics and Automation, 1998. Proceedings. IEEE, Los Angeles, USA, pp. 3368–3374 (1998) 2. Sitti, M., Hashimoto, H.: Tele-nanorobotics using atomic force microscope. In: Intelligent Robots and Systems, IEEE/RSJ International Conference, Victoria, Canada, pp. 1739–1746 (1998)

References

47

3. Sitti, M., Aruk, B., Shintani, H., et al.: Development of a scaled teleoperation system for nano scale interaction and manipulation. In: Robotics and Automation, IEEE International Conference, Seoul, Korea, pp. 860–867 (2001) 4. Guthold, M., Falvo, M.R., Matthews, W.G., et al.: Controlled manipulation of molecular samples with the nano manipulator. IEEE/ASME Trans. Mechatr. 5(2), 189–198 (2000) 5. Li, G.Y., Xi, N., Yu, M.M., et al.: 3D nanomanipulation using atomic force microscopy. In: IEEE International Conference on Robotics and Automation, Taipei, pp. 3642–3647 (2003) 6. Li, G.Y., Xi, N., Yu, M.M., et al.: Augmented reality system for real-time nanomanipulation. In: Third IEEE Conference on Nanotechnology, San Francisco, USA, pp. 64–67 (2003) 7. Li, G.Y., Xi, N., Chen, H.P., et al.: Nano-assembly of DNA based electronic devices using atomic force microscopy. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan, pp. 583–588 (2004) 8. Li, G.Y., Xi, N., Yu, M.M.: Calibration of AFM based nanomanipulation system. IEEE International Conference on Robotics and Automation, New Orleans, USA, pp. 422–427 (2004) 9. Li, G.Y., Xi, N., Yu, M.M., et al.: Development of augmented reality system for AFM-based nanomanipulation. IEEE/ASME Trans. Mechatron. 9(2): 358–365 (2004) 10. Li, G.Y., Xi, N., Yu, M.M., et al.: Planning and control of 3-D nanomanipulation. Acta. Mechanica. Sinica. 20(2), 117–124 (2004) 11. Liu, L.Q., Luo, Y.H., Xi, N., et al.: Sensor referenced real-time videolization of atomic force microscopy for nanomanipulations. IEEE/ASME Trans. Mechatron. 13(1), 76–85 (2008) 12. Seeger, A.: Surface Reconstruction from AFM and SEM Images. University of North Carolina, Chapel Hill, USA (2004) 13. Wang, D., Dong, Z.L., Jiao, N.D., et al.: An asymmetric PI hysteresis model for piezoceramics in nanoscale AFM imaging. IEEE International Conference on Nano/Micro Engineered and Molecular Systems. IEEE, pp. 1075–1079 (2010) 14. Wang, Z.Y., Liu, L.Q., Wang, Z.D., et al.: An extended PI model for hysteresis and creep compensation in AFM based nanomanipulation. In: IEEE International Conference on Robotics and Biomimetics, pp. 992–997 (2011) 15. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: AFM tip on-line positioning by using the landmark in nano-manipulation. In: Proceedings of the Nanotechnology Materials and Devices Conference, pp. 75–80 (2010) 16. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: A probabilistic approach for on-line positioning in nano manipulations. In: Proceedings of the Intelligent Control and Automation, 8th World Congress on 2010, pp. 450–455 (2010) 17. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: Feature referenced tip localization enhanced by probability motion model for AFM based nanomanipulations. Robio 2011, 1421–1426 (2011) 18. Wang, Z.Y., Liu, L.Q., Hou, J., et al.: Virtual nano-hand: A stable pushing strategy in AFM based sensorless nanomanipulation. IEEE International Conference on Robotics and Biomimetics, pp. 1409–1414 (2011) 19. Hou, J., Liu, L.Q., Wang, Z.D., et al.: AFM-based robotic nano-hand for stable manipulation at nanoscale. IEEE Trans. Autom. Sci. Eng. 10(2), 285–295 (2013)

Chapter 3

AFM Image Reconstruction Using Compensation Model of Thermal Drift

Abstract The system thermal drift is an uncertain factor in the scanning of the AFM image. Thermal drift is difficult to be detected by using traditional sensor. Therefore, digital image method is adopted to perform correction. Currently, the correction accuracy is not high and this correction method cannot be widely used to solve the system thermal drift. In this chapter, according to the scanning characteristics of the tip driver, the influence of thermal drift on the scanning process is analyzed, and the thermal drift deformation model of the AFM image is established. The characteristic region offset vectors in the AFM image are solved by mathematical method. The offset vectors of other non-characteristic regions in the image are calculated according to the characteristic region offset vectors. The whole scanning image is reconstructed to improve the global image accuracy.













Keywords AFM image System thermal drift Tip driver Offset vector Image reconstruction

3.1

Reconstruction Theory of AFM Thermal-Drift Image

AFM is the most promising tool for nano-scale observation and operation, and plays an important role in nano-scale research. System thermal drift is the important factor affecting image accuracy in scanning process. Therefore, it is important to study system thermal drift and reconstruct the AFM image. In this chapter, the Newton iteration method and the current dominant image interpolation algorithm are introduced. Then, the AFM image reconstruction algorithm is represented in detail. Finally, the effectiveness of the method is verified by simulation and experiment.

Authors: Fangjun Luan, Ye Zhang, Shuai Yuan, Tianshu Chu. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_3

49

3 AFM Image Reconstruction Using …

50

3.1.1

Newton Iteration Method

Newton iteration method [1–3], also known as Newton Raphson method, is an iterative method for solving approximate roots of the equation by linearizing the non-linear equation step by step. Newton iteration method uses the first few terms of Taylor series expansion to solve the equation f(x) = 0. Newton iteration method is an important method for solving equation, the greatest advantage of Newton iteration method for solving equation is that it has the characteristic of square convergence near the single root of f(x) = 0. It can quickly find the root of the equation, so it has been widely used in computer programming. If the root of function y = f(x) is r, assuming that x0 is the initial approximation of r, and making a tangent L of curve y = f(x) which crosses (x0, f(x0)), then L equation is y = f(x0) + f′(x0) (x − x0). According to L equation, the transverse coordinate x1 of intersection point of L and x axis can be obtained: x1 ¼ x0 

f ð x0 Þ f 0 ð x0 Þ

ð3:1Þ

In formula (3.1), x1 is the first approximation of r, and then x1 is brought into the function. A tangent of the curve y = f(x) is made at the point (x1, f(x1)), so that the transverse coordinate x2 of the intersection point of the tangent and the x-axis can be obtained: x2 ¼ x1 

f ð x1 Þ f 0 ð x1 Þ

ð3:2Þ

In formula (3.2), x2 is called the quadratic approximation of r. Through, repeating the above solving process, the n + 1 times approximation of the root r of y = f(x) is obtained. This solution is called Newton’s iteration method. The formula is as follows: xn þ 1 ¼ xn 

f ð xn Þ f 0 ð xn Þ

ð3:3Þ

The procedure of Newton iteration [4] is shown in Fig. 3.1. Newton’s iteration method usually needs the following three steps: Firstly, the iteration variables in the problem are determined, i.e. variables that generate new values alternately in Newton’s iteration method. Secondly, the iteration relation is defined according to the problem, that is, how to use the initial variable value to iterate to solve the next approximation. Finally, the iteration control conditions are set. Because the value calculated by Newton’s iteration method is usually approximation, if the approximate condition is not set, the iteration process will be infinite cycle, which is not conducive to find the optimal solution. So when carrying out calculation with Newton’s iteration method,

3.1 Reconstruction Theory of AFM Thermal-Drift Image Fig. 3.1 Procedure of newton iteration

51

y

(x0,f(x0))

x2

x1

x0

x

two kinds of termination iteration conditions are usually set: one is to determine the number of iterations, when the number of iterations reaches the upper limit, the solution is determined to be the optimal solution; the other is that iterative termination is usually performed when the difference between the last solution and the current solution is less than a certain threshold. In this case, iterative termination conditions need to be set according to the sought problem. Newton iteration method is locally convergent, which has the advantages of fast convergence speed, good stability and high accuracy. However, because its convergence speed near the multiple roots can be reduced and the function and its derivatives need to be recalculated every time, the amount of calculation is large. In practical calculation, simple iteration method is often used to calculate several steps to estimate an initial value with better quality, and then the final solution is computed iteratively with a function. If the initial value is given properly, a better solution can be obtained by two to three iterations, so Newton’s iteration method is not optimized in this book. In the work of this chapter, Newton iteration method is used to iteratively calculate the final value of the offset vector P. In the following section, the solution function of the offset vector will be defined and solved.

3.1.2

Image Interpolation Method

Image interpolation method [5–7] is a common method of processing digital images, which can be used to increase or reduce the pixels of digital images. For example, the pixel information of an image is stored in the function f(x, y). Normally, only the pixel values at any integer position in the image can be obtained by the function. However, by means of image interpolation, the pixel values of any position can be calculated, which makes the information of image pixels more abundant.

3 AFM Image Reconstruction Using …

52

3.1.2.1

Application of Image Interpolation

Image interpolation usually aims at enlarging image, increasing image resolution, image restoration or artistic effect processing. Here are some common applications of interpolation in image processing. (1) Image zooming: Using the known position pixels to obtain the unknown position pixel value by interpolation, enlarging or reducing the number of image pixels to achieve the purpose of enlarging or reducing the image in a certain proportion, as shown in Fig. 3.2. (2) Images twisting: Images twisting is widely used in the construction of artistic effects. By using the known integer position pixels, the algorithm can change the position of any pixel in the image to achieve images twisting, as shown in Fig. 3.3. (3) Image repairing: Image repairing is an image processing method to reconstruct, repair or remove redundant pixels from damaged images. It determines the effective pixel value of the image by algorithm, removes the invalid pixels by interpolation method, and calculates the real pixel value of the location to obtain the repaired image, as shown in Fig. 3.4.

The image before proportional scale transformation

The image before non-proportional scale transformation

The image after non-proportional scale transformation The image after proportional scale transformation

(a) Proportional scale transformation of the image

Fig. 3.2 Image scaling

(b) Non-proportional scale transformation of the image

3.1 Reconstruction Theory of AFM Thermal-Drift Image

(a) The image before twist

53

(b) The image after twist

Fig. 3.3 Image twisting

(a) The image before repair

(b) The image after repair

Fig. 3.4 Image repairing

(4) Image stitching: Image stitching is to stitch a large seamless image composed of overlapping images due to different shooting time, angle or sensor, as shown in Fig. 3.5. (5) Image geometric transformation: Image geometric transformation is to process the image and change the geometric shape, geometric position, geometric size and other characteristics of the image. This transformation is to change the spatial position of the pixels in the image, as shown in Fig. 3.6.

3 AFM Image Reconstruction Using …

54

(a) The image before stitch

(b) The image after stitch

Fig. 3.5 Image stitching

Fig. 3.6 Geometric transformation

The image after affine

The image after rotation

The image before transformation

The image after trim

The image after projection

3.1 Reconstruction Theory of AFM Thermal-Drift Image

3.1.2.2

55

Nearest Neighbor Interpolation Method

Nearest Neighbor Interpolation was originally used to analyze the rainfall data of meteorological stations and calculate the average rainfall. When this method is used, the digital image is defined as function F(x, y), where x, y are coordinate positions of each point in the image, and function F(x, y) is grayscale value of coordinate (x, y). Define the target image as G (x, y), then the relationship between the target image and the original image is as follows: F ð xÞ ¼ Gð xÞ  ðFw=GwÞ F ð yÞ ¼ Gð yÞ  ðFh=GhÞ

ð3:4Þ

In formula (3.4), Fw and Fh are the width and height of the original image, and Gw and Gh are the width and height of the target image. The coordinate positions calculated by formula (3.4) are non-integer values. Because the pixel coordinates in the image are all integers, the nearest neighbor interpolation uses rounding method to convert non-integer coordinates into integer coordinates, so as to fill up the coordinate positions in each target image. Although the algorithm is simple and the computation is small, the interpolated image has very bad effect, which easily causes the discontinuity of grayscale value of the image generated by interpolation. Meanwhile, apparent zigzag appears in the image, the mosaic phenomenon is serious after the image is enlarged, and the distortion is serious after the image is reduced.

3.1.2.3

Bilinear Interpolation Method

Bilinear interpolation is widely used in signal, video and digital image processing. Comparatively speaking, this interpolation method has better effect in scaling image. When obtaining the grayscale value of non-integer points in the original image, bilinear interpolation determines the pixel value of the point by using the four pixel values around the solution point in the original image. The specific calculation method is shown in Fig. 3.7. In Fig. 3.7, point (u, v) is the coordinate of the point in the original image. When calculating the grayscale value of point (u, v), bilinear interpolation calculates the gray value of point E and point F in Fig. 3.7 firstly, and then calculates the grayscale value of point (u, v) according to the grayscale value of point E and point F. The calculation formula is as follows: f ðE Þ ¼ ðv  jÞðf ðC Þ  f ð AÞÞ þ f ð AÞ f ðF Þ ¼ ðv  jÞðf ðDÞ  f ðBÞÞ þ f ðBÞ f ðu; vÞ ¼ ðu  iÞðf ðF Þ  f ðE ÞÞ þ f ðEÞ

ð3:5Þ

3 AFM Image Reconstruction Using …

56 Fig. 3.7 Bilinear interpolation method

C(i,j+1)

D(i+1,j+1)

E

F (u,v)

A(i,j)

B(i+1,j)

In formula (3.5), f(E) and f(F) are the grayscale values of E and F points, and f(u, v) is the grayscale values of the points in the original image. Compared with the nearest neighbor interpolation method, bilinear interpolation has a large amount of computations, and there is no grayscale discontinuity in the adjacent points of the image. However, the grayscale smoothing effect of bilinear interpolation will lead to the degradation of image details, especially when the image is enlarged, this shortcoming is particularly obvious.

3.1.2.4

Newton Interpolation Method

Newton interpolation makes use of several function values of given function y = f (x, y) in a certain interval to make appropriate specific functions. Other points in this interval are estimated to get approximate values with this specific function, which is called interpolation polynomial. Newton interpolation method obtains Newton interpolation polynomials by solving the difference quotient of each order. f ð xÞ ¼ f ðx0 Þ þ f ðx0 ; x1 Þðx  x0 Þ þ f ð x0 ; x1 ; x2 Þ ð x  x0 Þ ð x  x1 Þ þ




ð3:6Þ

þ f ðx0 ; x1 ; . . .; xn Þðx  x0 Þ. . .ðx  xn1 Þ Formula (3.6) is the expansion form of Newton’s interpolation polynomial. The basic idea of Newton’s interpolation method is to transform the n times interpolation polynomial to a form with inheritance characteristics. Then according to the conditions of interpolation, the undetermined coefficients are determined and the needed interpolation functions are obtained.

3.1 Reconstruction Theory of AFM Thermal-Drift Image

3.1.2.5

57

Bi-cubic and B-spline Interpolation

There are some problems in the abovementioned image interpolation methods, so in this book, Bi-cubic and B-spline interpolation method [8] is used to obtain the grayscale information of the corresponding position in the reference frame and the grayscale data of the integer position in the restructured image. The equation of n  m B-spline interpolation surface is: pðu; vÞ ¼

n X m X

aij Nin ðuÞMjm ðvÞ

i¼0 j¼0

ð3:7Þ

0  u  1; 0  v  1 In Formula (3.7), aij is the control vertex of n  m B-spline surface, and Nin and Mjm are the basis functions of surface. It is defined as: Nin ðuÞ ¼

ni 1X ð1Þk Cnk ðu þ n  i  k Þn n! k¼0

0u1 1 X Mim ðvÞ ¼ ð1Þk Cmk ðv þ m  j  kÞm m! k¼0 mj

ð3:8Þ

0v1 When n = m = 3 in Eq. (3.8), the function is a Bi-cubic and B-spline surface equation, as follows: pðu; vÞ ¼

3 X 3 X i¼0 j¼0

aij

3j 3i 1X 1X ð1Þk C3k ðu þ 3  i  k Þ3 ð1Þk C3k ðv þ 3  j  k Þ3 3! k¼0 3! k¼0 ð3:9Þ

0  u  1; 0  v  1

Then calculate Bi-cubic and B-spline surface equation, formula (3.9) is rewritten as follows: pðu; vÞ ¼

3 X

ai ðvÞNin ðuÞ

i¼0

ai ðvÞ ¼

3 X j¼0

ð3:10Þ pij Mjm ðvÞ

3 AFM Image Reconstruction Using …

58

Real sampling point Q14

Q24

Q13

Q12

Q23

Interpolation point Q34

u Q11

v

Q33

Q22

Q32

Q21

Interplation sample point

Q44

Q43

Q42

Q31

Q41

Fig. 3.8 Bi-cubic and B-spline interpolation method

Constructing interpolation surface is to solve B-spline surface by using the pixel values of 16 known pixels in the image to solve the pixel values of unknown pixels. At this time, the control vertex in u direction can be calculated first, and then the control vertex in v direction can be calculated from the control vertex in u direction. The specific calculation method is shown in Fig. 3.8. (1) The control vertices in u direction are calculated first. Taking the curves passing through four real sampling points such as Q11, Q12, Q13, Q14, the control vertices at four sampling points are calculated according to the pixel values of these four points, and the control vertices of the remaining 12 sampling points in u direction are similarly calculated. (2) The control vertices in v direction are inversely calculated by 16 known control vertices in u direction, and the control vertices are expressed by BQBT. Then the 16 control vertices obtained are brought into the matrix expression of Bi-cubic and B-spline surface as follows:

3 v3 6 27 6v 7  6 7 u3 A3 BQBT AT3 6 7 6 v7 4 5 2

 pðu; vÞ ¼ 1 u 2

1

6 16 6 3 A3 ¼ 6 6 6 3 4 1

3

3

6

3

0

3

4

1

1

3

2

u2

1

1:2679

7 6 6 07 7 6 0:3393 7B¼6 7 05 6 4 0:0893 0:0179 0

0:3393

0:0893

1:6964

0:4464

0:4464

1:6964

0:0893

0:3393

3

2 Q11 7 6 7 6 0:0893 7 6 Q12 7Q¼6 6 0:3393 7 5 4 Q13 1:2679 Q14 0:0179

Q21

Q31

Q22

Q32

Q33

Q33

Q44

Q34

Q41

3

7 Q42 7 7 7 Q43 7 5 Q44

0  u  1; i; j ¼ 0; 1;


; n  3

ð3:11Þ

3.1 Reconstruction Theory of AFM Thermal-Drift Image

59

(3) The pixel information on Bi-cubic and B-spline surface can be solved by using sampling point around interpolation point in the u direction and v direction, and substitute these known pixels into Eq. (3.11). The disadvantage of Bi-cubic and B-spline interpolation is that the computational complexity and the computational time increase sharply because of the cubic operation. However, the advantage of this method is that the accuracy of the image obtained by this method is higher and the distortion is smaller than that of the previous methods. Therefore, the Bi-cubic and B-spline interpolation method is mainly used to calculate the interpolation point height information of the AFM reconstructed image in this book.

3.1.3

Thermal Drift Correction Method for Scanning Image

As for the problem of system thermal drift, there are two main methods of thermal drift correction: one is based on image registration, that is, image correction after scanning and imaging; the other is real-time correction of system thermal drift based on local scan during scanning. The thermal drift compensation algorithm based on image registration is described in detail below. In 2011, Long Fei et al. proposed a method based on image registration [9] to compensate the thermal drift of the AFM image system. The image registration method is to obtain multiple images from the same sample in many ways, and then correct the images by iteration summation. It can obtain multiple images from different sensors at different angles or from the same sample at the same angle and at different times. In image registration, one image is used as a reference image and the rest are corresponding measurement images. In the reference image, a small area is selected as the identification. The identification area is compared with the reference image, and the point that is most similar to the identification area is identified as the corresponding point. In image registration, the normalized cross-correlation function is used to measure the similarity of the registration image. The formula for calculating similarity is as follows: P x;y

  ½f ðx; yÞ  f  tðx  u; y  vÞ  tu;v

mðu; vÞ ¼ (rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffirffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi) 2 P P 2 ½f ðx; yÞ  f  tðx  u; y  vÞ  tu;v x;y

ð3:12Þ

x;y

In formula (3.12), m (u, v) is the correlation coefficient. When m (u, v) reaches the maximum value, u and v are the thermal drift in x direction and y direction. Long Fei carried out image registration by scanning 50 consecutive AFM images at the same angle and different times. The thermal drift curves of 50 scanned images are shown in Fig. 3.9.

3 AFM Image Reconstruction Using …

60 30

Thermal drift in x-direction Thermal drift in y-direction Thermal drift size (nm)

10

-10

-30

-50 0

20

40

60

80

100

Scan time (minute)

Fig. 3.9 Thermal drift curve of 50 continuous scanning AFM images

(b) Thermal drift direction x-

y-

Slow scan direction

Slow scan direction

(a)

y+

x+ Thermal drift direction

Fig. 3.10 Thermal drift-induced distortion of AFM images

By studying the thermal drift of 50 continuous scanning AFM images and measuring the size of thermal drift, the characteristics of AFM thermal drift are analyzed. The problem of image distortion caused by thermal drift is described by using square to represent objects, as shown in Fig. 3.10. In Fig. 3.10, assuming that the real shape of the object is a standard square, the slow scanning direction is from top to bottom, and if the x-direction thermal drift is x-, and the y-direction thermal drift is y-, the AFM scanning image of the square object is shown in Fig. 3.10a. The image is stretched in y-direction and gradually shifted to the x-direction. If the x-direction thermal drift is x+, and the y-direction thermal drift is y+, the AFM scanning image of the square object is shown in Fig. 3.10b. The image is compressed in the y-direction and gradually shifts to the x-direction.

3.1 Reconstruction Theory of AFM Thermal-Drift Image

61

In the research, as for the thermal drift of image on two-dimensional plane, the AFM images is reconstructed. The thermal drift of image in x-direction and ydirection is corrected. Colloidal gold droplets are used as experimental objects to reconstruct the image in the sample, and the reconstructed effect of the thermal drift AFM image is verified.

3.2

Reconstruction Method for Thermal Drift Image

Thermal drift of the system will cause distortion of the scanned image in the process of AFM scanning. System thermal drift refers to the relative random drift between the tip and the object to be maneuvered due to the scanner drift caused by the change of the ambient temperature in the imaging process. Because this change cannot be real-time controlled during the scanning process, the AFM scanning image are distorted to a certain extent, which seriously reduces the accuracy of AFM scanning image. Although some researchers have proposed the model-based thermal drift compensation methods, such as Kalman filter and neural network, the effect of these methods depends on the accuracy of model parameters. It is not easy to obtain accurate parameters through experiments, so these methods cannot effectively compensate the drift problem caused by thermal drift. According to the scanning characteristics of the driver, this book analyses the influence of thermal drift on the scanning process, describes the thermal drift by establishing the image deformation model, estimates the image deformation parameters using Newton iteration method, and constructs a reconstruction method based on cubic spline curve interpolation to correct the scanning image. On the basis of the abovementioned research, the thermal drift image is simulated and reconstructed by MATLAB to verify the effectiveness of the method. Finally, this method is applied to the AFM scanning image. The experimental results show that this method can effectively reduce the influence of thermal drift on AFM scanning image and improve the quality of the image.

3.2.1

Compensation Model for Thermal Drift

The nanoparticle samples used in this book are regular spheres, and so the two-dimensional plane image of the scanning result should be regular circular. In the following research experiments, this characteristic of nanoparticle can be used to analyze and verify. In order to conveniently understand the influence of thermal drift on the imaging result of sample during AFM scanning process, the same sample is scanned in two different directions, and the scanning imaging result are shown in Fig. 3.11. Figure 3.11a shows the true morphology of the nanoparticle sample, which is a regular sphere. Figures 3.11b, c are the scanning result respectively when the

(b) The AFM image of nanoparticle from top to bottom

System thermal drift direction

System thermal drift direction

Slow scan direction (a) The SEM image of nanoparticle

Slow scan direction

3 AFM Image Reconstruction Using …

62

(c) The AFM image of nanoparticle from bottom to top

Fig. 3.11 AFM scanning image

scanning image pixels are 512  512, the thermal drift direction is upward to right, and the slow scanning direction changes. Figure 3.11b shows the AFM scanning image of the sample in the direction of slow scanning from top to bottom. Figure 3.11c shows that the same sample AFM image is obtained with slow scanning direction from bottom to top. As can be seen in Fig. 3.11b, the original regular nanoparticle sample is compressed in vertical direction, and the whole image shows a gradual shift to the left. In Fig. 3.11c, the regular circular nanoparticles are stretched in vertical direction, and the whole image is gradually shifted to the right. It can be seen that the system thermal drift in scanning process will lead to larger errors in AFM image measurement. If the distortion caused by the system thermal drift during scanning can be corrected, the measurement error of the image can be reduced and the accuracy of the AFM image can be improved. The scanning speed of AFM in the fast scanning direction is very fast, and the scanning time interval between points on the same fast scanning line is very short. Therefore, the influence of thermal drift on fast scanning is very little and the image distortion is small. However, when scanning by AFM, after a round trip scan in the fast scanning direction, it moves a step forward in the slow scanning direction. So the interval of scanning time in the slow scanning direction is longer, and the influence of thermal drift is significant, so the distortion in the slow scanning direction is more obvious. According to the scanning characteristic of AFM, the same sample is scanned in two different angles (namely 0 and 90 directions,) to obtain the scanning result, and according to the result, the displacement and deformation of the image are determined, and the real image information is calculated. When scanning by using AFM, the ideal scanning result should be a standard square grid. However, due to the influence of system thermal drift, the image will be distorted after scanning. Compared with the ideal standard grid, the real scanned image will have a certain degree of offset in both horizontal and vertical directions [10]. The ideal results of the scanning images by AFM and the scanning results in the direction of 0° and 90° in the actual situation are shown in Fig. 3.12. Figure 3.12a is an ideal AFM scanning grid, whose row spacing is s and column spacing is t. It is assumed that the row number of the scanning grid is j and column number is i; Fig. 3.12b is a 0° scanning distorted grid, which is defined as a reference in this book; Fig. 3.12c is a 90° scanning distorted grid, which is defined

3.2 Reconstruction Method for Thermal Drift Image

63

i

i s

s

j

j t

t

s

t

t

t

t

t

s

t

s

s

s

s

(a) Image scanning grid points (b) Drift between scanning lines (c) Drift between scanning columns

urj

vid

uid

vrj

(d) Definition of thermal drift error for line scanning

(e) Definition of thermal drift error for column scanning

Fig. 3.12 Scanning result of the same sample

as a deformation image; Fig. 3.12d shows the horizontal/vertical displacement of the reference image relative to the ideal AFM scan grid on each scan line. urj represents the horizontal offset on the scan line of the j-th line on the reference image, and vrj represents the vertical offset on the line; Fig. 3.12e shows the horizontal/vertical displacement of the deformation image relative to the ideal grid on each scan line. udi represents the horizontal offset of each point on the scan line of the i-th column on the deformation image, and vdi represents the vertical offset on the column.

3.2.2

Thermal Drift Offset Vector

According to the characteristics of AFM scan, there is a certain correspondence between the reference and the deformation image of the same sample in AFM scanning results. If the relationship between the reference and the deformation image can be defined effectively, the real information of each scanning position can be calculated. The book defines this mapping relationship as a mapping function for

3 AFM Image Reconstruction Using …

64

the offset vector p. For example, if the height information at the coordinate point (x, y) in the reference frame is defined as a function g (x, y), and the height information at the corresponding position (x′, y′) in the deformation image is h (x′, y′), there should be a mapping relationship between (x, y) and (x′, y′) about the offset vector p, which is defined as: x0 ¼ x þ uðx; y; pÞ y0 ¼ y þ vðx; y; pÞ

ð3:13Þ

In this book, the first-order offset vector and the second-order offset vector [11] is studied, and the correspondence between the reference image and deformation image is also researched.

3.2.2.1

First-Order Offset Vector

When the offset vector is defined as the first-order partial derivative, there is a mapping relationship represented as the first-order offset vector p1-order between the reference and deformation image. At this time, the offset vector p1-order has six parameters. The first-order offset vector p1-order (p0, p1, p2, p3, p4, p5) is defined as: p1order

  @u1order  @v1order  u1order ðx0 ; y0 Þ; v1order ðx0 ; y0 Þ; ; ; @x x0 ;y0 @x x0 ;y0   ! @u1order  @v1order  ; @y x0 ;y0 @y x0 ;y0

ð3:14Þ

In formula (3.14), p0 and p1 are the horizontal and vertical offsets of the two images at the image center, p2 represents the horizontal offset on the row scan line, p3 represents the horizontal offset on the column scan line, p4 represents the vertical offset on the row scan line, and p5 represents the vertical offset on the column scan line. The mapping relationship between the reference and the deformation image regarding the offset vector p1-order is as follows: u1order ðx; y; p1order Þ ¼ p0 þ p2 ðx  x0 Þ þ p4 ðy  y0 Þ v1order ðx; y; p1order Þ ¼ p1 þ p3 ðx  x0 Þ þ p5 ðy  y0 Þ

ð3:15Þ

In formula (3.15), u1-order (x, y, p1-order) and v1-order (x, y, p1-order) are the first-order displacement functions related to the corresponding relationship between the reference and the deformation image, indicating the mapping relationship between the deformation image and the reference at the position of the reference point (x, y). (x0, y0) denotes the central point coordinate of the reference image, and point (x, y) denotes the position coordinate of the points in the reference.

3.2 Reconstruction Method for Thermal Drift Image

3.2.2.2

65

Second-Order Offset Vector

When the offset vector is defined as the second-order derivative, there is a mapping relationship defined as the second-order offset vector p2-order between the reference and the deformation image. At this time, the offset vector p2-order has twelve parameters. The second-order offset vector p2-order (p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11) is defined as:     1 @u2order  @v2order  @u2order  @v2order  ð x ; y Þ; v ð x ; y Þ; ; ; ; u 2order 0 0 2order 0 0     B C @x @x @y @y x ;y x ;y x ;y x ;y B   0 0 0 0 0 0 0 0C p2order ¼B C @ @u2order  A @v2order  @u2order  @v2order  @u2order  @v2order  ; ; ; ; ; ; @x@x  @x@x  @y@y  @y@y  @x@y  @x@y  0

x0 ;y0

x0 ;y0

x0 ;y0

x0 ;y0

x0 ;y0

x0 ;y0

ð3:16Þ In formula (3.16), p0 and p1 are the offsets of the two images in the horizontal and vertical directions of the center point of the image. p2, p3, p4, p5 represent the first-order offset vectors of the mapping function, p6, p7, p8, p9, p10, p11 represent the second-order offset vectors of the mapping function. There is a mapping relationship defined as the offset vector p2-order between the reference and the deformation image as follows: 1 u2order ðx; y; p2order Þ ¼ p0 þ p2 Dx þ p4 Dy þ p6 Dx2 þ 2 1 v2order ðx; y; p2order Þ ¼ p1 þ p3 Dx þ p5 Dy þ p7 Dx2 þ 2

1 p8 Dy2 þ p10 DxDy 2 1 p9 Dy2 þ p11 DxDy 2 ð3:17Þ

In formula (3.17), Δx = x − x0, Δy = y − y0, u2-order (x, y, p2-order) and v2-order (x, y, p2-order) are second-order displacement functions related to the corresponding relationship between the reference and the deformation image, representing the mapping relationship between the deformation image and the reference point (x, y) position. (x0, y0) denotes the central point coordinate of the reference, (x, y) denotes the position coordinate of in the reference. The effect of each p in formula (3.17) on the offset of each point in the image is shown in Fig. 3.13. As can be seen from Fig. 3.13, the change of any parameter in the second-order displacement vector has a great influence on the image morphology. Therefore, applying the second-order displacement vector to the calculation of the displacement vector, more kinds of image morphology can be calculated. At the same time, the second-order displacement vector can also represent a wider range of deformation, and the obtained displacement vector is more accurate. Because the system thermal drift is mostly horizontal and vertical components, and when scanning the same image, the velocity and direction of thermal drift are relatively stable, so only the first-order offset vector need to be used to calculate the displacement of each point in the image effectively in this book. Therefore, in the

3 AFM Image Reconstruction Using …

66 y

x

Normal image without transformation

P5=0.3

P9=0.1

P2=0.3

P6=0.1

P10=0.1

P4=0.3

P3=0.3

P8=0.1

P7=0.1

P11=0.1

P2=-0.1, P3=-0.1, P4=0.1, P5=-0.15, P6=0.05,P8=-0.05,P10=0.1,P11=0.125

Fig. 3.13 The effect of each p component on AFM image

image reconstruction of this book, only the first-order offset vector is used to calculate the displacement vector.

3.2.3

Offset Vector Calculation

In the abovementioned content of this chapter, the AFM image thermal drift model has been described, and the first and second order offset vectors have been defined. In this book, only the first order offset vectors need to be used to correct the thermal drift image. As described in the previous sections, the first-order offset vector calculation formula, i.e. the mapping relationship between the reference and the deformation image is as follows: uðx; y; pÞ ¼ p0 þ p2 ðx  x0 Þ þ p4 ðy  y0 Þ vðx; y; pÞ ¼ p1 þ p3 ðx  x0 Þ þ p5 ðy  y0 Þ

ð3:18Þ

In this book, the thermal drift correction method is adopted for the same region obtained by scanning at two different angles, and the height information of the same position is compensated. These scanned images not only obtain the nanoparticle

3.2 Reconstruction Method for Thermal Drift Image

67

height data with significant regular information in the nanoparticle region, but also contain some noise information in the non-nanoparticle region. These significant height information of nanoparticles can be used correct the nanoparticle regional image. As for the non-nanoparticle region with irregular noise signals, it is difficult to identify and compare the corresponding region in the reference image and deformation image for correcting them. Because the offset vectors calculated from non-characteristic regions are inaccurate, in order to avoid unnecessary errors, the image is divided into characteristic regions and non-characteristic regions when calculating offset vectors. The former is the area where nanoparticles are located, and the latter is the area without nanoparticles. When calculating the offset vector p, the significant height information of nanoparticles is used first to correlate the two images of the same sample scanned from different angles; secondly, we calculate the offset vectors of the characteristic regions, i.e., the scanning lines of the positions of nanoparticles in the sample; Finally, the offset vector of the non-feature region in the sample is calculated using the offset vector of the nanoparticles around the non-characteristic region.

3.2.3.1

Calculation of Offset Vectors in Characteristic Regions

In this book, 0° scanning image is used as the reference image, and 90° scanning image is called deformation image. When calculating the offset vectors of characteristic regions, the maximum cross-correlation coefficient method is used to calculate the similarity of characteristic regions by combining the reference and deformation images. The calculation method of similarity is as follows [10]: 0 C ð pÞ ¼ @

y0 X þ N=2

x0 X þ M=2

gðx; yÞhðx þ uðx; y; pÞ; y þ vðx; y; pÞÞÞ

y¼y0 N=2 x¼x0 M=2

0

@

y0 X þ N=2

x0 X þ M=2

y¼y0 N=2 x¼x0 M=2

g2 ðx; yÞ

y0 X þ N=2

x0 X þ M=2

y¼y0 N=2 x¼x0 M=2

h2

x þ uðx; y; pÞ; y þ vðx; y; pÞ

!1 A ð3:19Þ

Formula (3.19), (x0,y0) denotes the central point of the corresponding region with size M  N in the reference image, and g(x,y) denotes the height of (x,y) in the reference. h(x + u(x,y,p),y + v(x,y,p)) denotes the height of (x,y) in the deformation image. The offset vector p value is calculated by using the similarity calculation formula. The calculation method is as follows: (1) Assuming an Initial Value of p as pold According to the deformation law, a better initial value of p is estimated, which can reduce the number of iterations and save the algorithm time.

3 AFM Image Reconstruction Using …

68

(2) Solving the Height of Corresponding Points in Deformation Image with Substituted p The initial value pold of p is substituted into the deformation image function h (x + u (x,y,p),y + v (x,y,p)). The height information of (x, y) points in the deformation image is solved. Because only integer coordinate position and height information is collected in deformation image, Bi-cubic and B-spline interpolation method is needed to calculate the height information of the points to be solved. (3) Iterative Calculation of pnew Values The height information of the reference and the corresponding points in the deformation image are substituted into the similarity function, and the new p value is solved iteratively by Newton iteration method. The iteration formula is as follows: rrC ðpold Þðpnew  pold Þ ¼ rC ðpold Þ

ð3:20Þ

In formula (3.20), ∇C (pold) and ∇∇C (pold) are the first and second order derivatives of similarity function, respectively. The first derivatives are as follows:  rC ¼

@C @pi



X 2 @hðx; y; pÞ ðgðx; yÞ  hðx; y; pÞÞ ¼P 2 @pi g ðx; yÞ i¼0:5 i¼0:5 ð3:21Þ

In formula (3.21), pi is the component of offset vector p, the second derivative formulas of p0, p1, p2, p3, p4, p5 as follows: 9 8 P @hðx;y;pÞ @hðx;y;pÞ > >  2  P 2 = < 2 @pj @pi g ðx;yÞ @ C rrC ¼ ¼ ð3:22Þ P 2 @ hðx;y;pÞ @pi @pj i¼0:5 > ðgðx;yÞhðx;y;pÞÞ @p @p > : þ P2 i j ; g2 ðx;yÞ j¼0:5 i¼0:5 j¼0:5

In formula (3.22), both pi and pj are components of offset vector p. In calculation of Newton iteration method, when the difference between pold and pnew is less than the threshold or the iteration number reaches the upper limit of iteration number, it is determined that the pnew obtained at this time is the best p value for the next image correction. Newton iteration method converges faster, if the given initial value is appropriate, the appropriate p value can be obtained by several iterations.

3.2.3.2

Calculation of Offset Vectors in Non-characteristic Regions

The p-value calculation of non-characteristic region is based on the offset vector of the center point of the characteristic region near the non-characteristic region. When calculating the offset vector p of non-characteristic region, the whole image is divided according to the location of the characteristic region, and then the offset

3.2 Reconstruction Method for Thermal Drift Image

69 x

Region 1 1

2

Judgement line 1 Judgement line 2

3

Segment line 1

Region 2 4

5

Judgement line 3 Segment line 2

6

Region 3

y

Fig. 3.14 Reconstructed area partition method for integral image

vectors of non-characteristic region in each region are calculated according to the known p value in this region. When dividing regions, the vertical distance between any two adjacent nanoparticles is taken as the basis for dividing them. If the vertical coordinates of any two nanoparticles intersect, the two nanoparticles will be classified into the same region. Otherwise, the two nanoparticles will be divided into two different regions. The dividing method is shown in Fig. 3.14. The detailed description of the specific regional division method is as follows: (1) Calculation for Judging the Longitudinal Intersection of Nanoparticles As shown in Fig. 3.14, nanoparticle 1, 2 and 3 intersect in the longitudinal direction, such as decision line 1 and 2 in the figure; nanoparticle 4 and 5 intersect in the longitudinal direction, such as decision line 3 in the figure; nanoparticle 3 and 4 are far away from each other in the longitudinal direction; nanoparticle 5 and 6 do not intersect in the longitudinal direction; (2) Dividing the region According to Intersection Since the nanoparticles 3 and 4 do not intersect in the longitudinal direction, the center points of the two nanoparticles are connected in the longitudinal direction, and a horizontal line is formed along the midpoint of the line, and the horizontal line is the dividing line 1. The nanoparticle 3 and the nanoparticle 4 are divided into two regions; the nanoparticle 5 and 6 do not intersect in the longitudinal direction, and the same is divided into two regions along the dividing line 2. According to this division method, the area in Fig. 3.14 is divided into three regions.

3 AFM Image Reconstruction Using …

70

(3) Calculating p-Value of Non-characteristic Region When calculating the p values in three regions, the average of the offset vectors of the nanoparticles in the region is calculated using the offset vectors of the nanoparticles in the region, which is the p value of the region. The calculation method is as follows:

 ppart1 ¼ P0r1 þ P0r2 þ P0r3 =3

 ppart2 ¼ P0r4 þ P0r5 =2 ppart3 ¼

ð3:23Þ

P0r6

In the formula, ppart1, ppart2 and ppart3 are the offset vectors of the three regions in Fig. 3.14, respectively. p′r1, p′r2, p′r3, p′r4, p′r5 and p′r6 are the offset vectors of the six central points of the nanoparticles in Fig. 3.14. This section describes the calculation method of offset vector p in detail. In the next section of image reconstruction, the offset vector calculated in this section is used to reconstruct the thermal drift image.

3.2.4

Integral Image Reconstruction

The AFM image reconstruction of thermal drift in this book mainly depends on the position relationship between reference and deformation image relative to standard grids. Figure 3.12 shows that a point (i,j) on the standard grid is located on the column i of row j. The corresponding position coordinate of the point is (i + uri , j + vrj ) in the reference and (i + udi , j + vdj ) in the deformation image. The difference between the horizontal and vertical coordinates of the corresponding point in the reference and the deformation image is obtained as follows [10]:

 Ui þ urj ;j þ vrj ¼ i þ udi  i þ urj ¼ udi  urj

 Vi þ urj ;j þ vrj ¼ j þ vdi  j þ vrj ¼ vdi  vrj

ð3:24Þ

If there are n rows and m rows of scanning lines in a scanned image (n and m are positive integers which are not zero respectively), in order to reduce the influence of deviation effect, the method of calculating mean is used to minimize random error. First, the distance difference of m points on the same scanning line is summed up and the mean is calculated:

3.2 Reconstruction Method for Thermal Drift Image

urj ¼  vrj

m m 1X 1X Ui þ urj ;j þ vrj þ ud m i¼1 m i¼1 i

m m 1X 1X ¼ Vi þ urj ;j þ vrj þ vd m i¼1 m i¼1 i

71

ð3:25Þ

urj and vrj in formula (3.25) are known image displacement caused by system thermal drift at each location which has been solved in the abovementioned content udi and vdi are unnecessary unknowns, in order to eliminate the two unknowns, udi and vdi can be eliminated by subtracting the displacement of any two scanning lines separated by several lines. If the displacement of line j + k and line j is subtracted, the following results can be obtained: m

1X Ui þ urjþ k ;j þ k þ vrjþ k  Ui þ urj ;j þ vrj m i¼1 m

1X Vi þ urjþ k ;j þ k þ vrjþ k  Vi þ urj ;j þ vrj vrj þ k  vrj ¼  m i¼1

urj þ k  urj ¼ 

ð3:26Þ

Therefore, when reconstructing the reference, the horizontal line c is selected as the reference line, and make urc = 0, vrc = 0. The general expression of the horizontal and vertical displacement increment of each point on the horizontal scanning line relative to the horizontal line c in the reference is as follows: m

1X Ui þ urj ;j þ vrj  Ui;c m i¼1 m

1X Vi þ urj ;j þ vrj  Vi;c Dvrj ¼ vrj  vrc ¼  m i¼1

Durj ¼ urj  urc ¼ 

ð3:27Þ

In formula (3.27), j denotes the line marker of the scanning line, Δurj and Δvrj denote the increment of displacement in horizontal and vertical directions on the horizontal scanning line j, respectively. Then the real coordinate information of each point on the reference image is: xcj ¼ xrj þ Durj ycj ¼ yrj þ Dvrj zcj

¼

ð3:28Þ

zrj

In formula (3.28), xcj , ycj and zcj represent the horizontal and vertical coordinates and height information of each point on the j-th horizontal scanning line in the reconstructed image, while xrj , yrj and zrj represent the horizontal and vertical coordinates and height information of each point on the j-th horizontal scanning line in the reference.

72

3 AFM Image Reconstruction Using …

By using the abovementioned method, the position coordinates and height information of each point in the reconstructed image can be calculated, and a new corrected image can be obtained. However, the coordinate information at this time is not an integer value, so the Bi-cubic and B-spline interpolation method is used to calculate the height information of the integer coordinate position in the reconstructed image to complete the reconstruction of thermal drift image.

3.3

Simulation and Experimental Analysis

In this section, the algorithm in this book is simulated, and the AFM image is reconstructed. The validity of the algorithm in this book is verified by comparing and analyzing the experimental data.

3.3.1

Simulation and Analysis of Thermal Drift Image

In order to validate the effectiveness of the algorithm in this book, before applying the algorithm to the AFM scanning image, firstly, according to the thermal drift characteristics of the scanning process of AFM, the thermal drift effects of 0° and 90° scanning images in the experiment are simulated, and then the algorithm of this book is used to reconstruct the image to validate the effectiveness of the algorithm. According to the thermal drift characteristics of AFM, a standard circular image is constructed. The diameter ratio of the circle is 1, that is, the real morphology of the nanoparticle. According to the data information of the standard circle, two images with the same data information as the 0° scanning image and the 90° scanning image are constructed, respectively. The simulation experiments are carried out using the algorithm in this book. The simulation process is shown in Fig. 3.15. In Fig. 3.15a is a standard circle constructed by simulation, Fig. 3.15b is a 0° scanning image constructed by simulation, and Fig. 3.15c is a 90° scanning image constructed by simulation. Using the simulation data in Fig. 3.15b, c, the algorithm in this book is used to reconstruct the image of Fig. 3.15b. The reconstructed result is shown in Fig. 3.15d. Before reconstruction of the simulated image, the ratio of the horizontal and vertical diameter of the ellipse region in the reference image is 0.8710; after the reconstruction of the simulated image, the ratio of the horizontal and vertical diameter of the ellipse region in the image is 0.9794. The simulation results show that the corrected images are closer to the actual morphology of the samples than the two scanned simulated images, which verifies the effectiveness of the proposed algorithm. In the next section, the algorithm in this book is applied to the real scanned images of AFM for image reconstruction.

3.3 Simulation and Experimental Analysis

73

100

150

80

100

60

y (pixel point)

y (pixel point)

40 20 0 -20 -40 -60

50 0 -50

-100

-80 -100 -100 -80 -60 -40 -20 0 20 40 60 80 100 x (pixel point)

-150 -150

150

150

100

100

50

50

y (pixel point)

y (pixel point)

(a) Scanning result of standard circle

0 -50

-50 50 0 x (pixel point)

100

150

(b) Row scan result under the influence of thermal drift

0 -50

-100

-100 -150 -150

-100

-100

-50 50 0 x (pixel point)

100

(c) Column scan result under the influence of thermal drift

150

-150 -150

-100

-50 50 0 x (pixel point)

100

150

(d) Thermal drift constructed result of the algorithm

Fig. 3.15 Simulation results of AFM scanning images under the influence of thermal drift

3.3.2

Experiment and Analysis of Reconstruction of Thermal Drift Image

The proposed algorithm in this book needs to process the same sample image in the direction of 0° and 90° respectively. The corresponding scanning images of the two directions are shown in Fig. 3.16. There are six independent nanoparticles (marked from P1 to P6) in the sample image, two of nanoparticles are conjoined together (marked as P7).

3 AFM Image Reconstruction Using …

74

Nanoparticles P1

P3

P5

P6

P2

P2

P4

P4 P6

P7

P3 P1

(a) Scanning image of the nanoparticle with row-by-row

P5

P7

(b) Scanning image of the nanoparticle with column-by-column

Fig. 3.16 Scanning images of the same region at 0° and 90°

In Fig. 3.16a shows the sample imaging results at 0° scan, and Fig. 3.16b shows the sample imaging results at 90° scan. Next, two images in Fig. 3.16 are used to carry out an experiment with the algorithm proposed in this book. This book verifies the effectiveness of the thermal drift image reconstruction algorithm mainly from three aspects: first, because nanoparticles are standard circle, two-dimensional planar image can observe the reconstruction effect of the image by measuring the aspect ratio of the diameter of the nanoparticle; secondly, two ways to calculate whether the thermal drift speed is in order of magnitude, that is, calculating the image thermal when reconstructing the image, which aims to verify the correctness of thermal drift correction. Finally, on the basis of local reconstruction of the AFM thermal drift image, the whole scanning image of the AFM is corrected, and the overall reconstruction effect is verified by measuring the change in distance between the two nanoparticles in the horizontal and vertical directions.

3.3.2.1

Morphological Changes of Nanoparticles

After reconstructing the thermal drift image by using the algorithm in this book, the aspect ratio of nanoparticles before and after reconstructing changes significantly. Six nanoparticles from P1 to P6 in the experimental sample of Fig. 3.16 are taken as an example, Fig. 3.17 is the image before and after local reconstructing of each nanoparticle region. Figure 3.17a, c, e, g, i and k are images before reconstruction, Fig. 3.17b, d, f, h, j, l are images after reconstruction. From the coordinate information in Fig. 3.17, it can be seen that the reconstructed image is stretched in the longitudinal coordinate direction, and the reconstructed image is closer to the standard circle.

3.3 Simulation and Experimental Analysis

75

W=22.15

H=21

y (9.8nm)

y (9.8nm)

W=23

H=21.20

x (9.8nm)

x (9.8nm)

(a) Image of the particle P1 before reconstruction

(b) Image of the particle P1 after reconstruction

W=22.15

H=19

y (9.8nm)

y (9.8nm)

W=21

x (9.8nm) (c) Image of the particle P2 before reconstruction

H=21.10

x (9.8nm) (d) Image of the particle P2 after reconstruction

W=25.00

H=22

x (9.8nm) (e) Image of the particle P3 before reconstruction

y (9.8nm)

y (9.8nm)

W=24

H=24.00

x (9.8nm) (f) Image of the particle P3 after reconstruction

Fig. 3.17 Comparison before and after local reconstruction

3 AFM Image Reconstruction Using …

76

W=24.05

H=22

y (9.8nm)

y (9.8nm)

W=24

x (9.8nm) (g) Image of the particle P4 before reconstruction

H=23.26

x (9.8nm) (h) Image of the particle P4 after reconstruction

W=25.10

H=22

y (9.8nm)

y (9.8nm)

W=25

H=23.15

W=23

W=24.08

H=21

x (9.8nm) (k) Image of the particle P3 before reconstruction

Fig. 3.17 (continued)

y (9.8nm)

x (9.8nm) (j) Image of the particle P5 after reconstruction

y (9.8nm)

x (9.8nm) (i) Image of the particle P5 before reconstruction

H=22.90

x (9.8nm) (l) Image of the particle P3 after reconstruction

3.3 Simulation and Experimental Analysis

77

Table 3.1 Nanoparticle aspect ratio before and after reconstruction (Unit: nm) Nanoparticle Aspect ratio

Particle 1 (%)

Particle 2 (%)

Particle 3 (%)

Particle 4 (%)

Particle 5 (%)

Particle 6 (%)

Reference image Reconstructed image Increased proportion (%)

91.30 95.71

90.48 95.48

91.67 96.00

91.67 96.71

88.00 92.23

91.30 95.10

4.41

5.00

4.33

5.04

4.48

3.80

Because the nanoparticles used in the experiment are regular spheres, the image reconstructed results can be effectively observed by measuring the diameter aspect ratio of nanoparticles in the image. It is considered that there are two-dimensional tangent nanoparticles in the experimental image, it is not conducive to calculate the aspect ratio and cannot verify the effect. Therefore, only the single nanoparticle in the sample is compared. After processing, the statistical results of the diameter aspect ratio of each nanoparticle in the two images of reference and restoration are shown in Table 3.1. As can be seen from Table 3.1, the diameter aspect ratio of the reconstructed image corrected by the proposed method is closer to 1 than that of the reference image, that is to say, the reconstructed image is closer to the real morphology of the sample. In addition, compared with the reference diameters, the ratio of reconstructed diameters has been increased, the lowest ratio of diameters to lengths has been increased by 3.80%, the average ratio has been increased by more than 4.51%, and the difference between the lowest and the highest ratio is small, the reconstructed effect is more stable and acceptable. Longfei reconstructed colloidal gold droplets in the reference [9]. This experimental sample is also a regular sphere, and the true shape of the two-dimensional plane is also a regular circle. Therefore, the evaluation criteria for reconstructing effect is the same as this book. Diameter aspect ratio is used to measure the effect of reconstructing. In Longfei’s experimental verification, only a set of reconstructed data is provided to verify the effect of reconstruction. The comparison result before and after reconstructing is shown in Fig. 3.18. In Fig. 3.18, after reconstruction of Longfei algorithm, the aspect ratio of diameter is increased by about 4.21%, which is lower than the mean value of the aspect ratio in the proposed algorithm. Compared with the diameter aspect ratio before and after Longfei reconstruction as shown in Fig. 3.18 and the diameter aspect ratio before and after image reconstruction in Table 3.1, the reconstructed image after the proposed algorithm is closer to the true shape of the sample (i.e. the aspect ratio is 1), and the reconstructed effect of this algorithm is better than that of the thermal drift reconstruction algorithm based on image registration.

3 AFM Image Reconstruction Using …

78

Aspect ratio 0.95

Aspect ratio 0.91

(b) AFM image after reconstruction

(a) AFM image before reconstruction

Fig. 3.18 Comparison before and after reconstructing Longfei algorithm

3.3.2.2

Verification of Thermal Drift Velocity

In the experiment of this book, not only the effectiveness of the algorithm is validated by the morphology of nanoparticles, but also the effect of image reconstruction is validated by the thermal drift velocity. Thermal drift velocity is calculated by using image position change before and after reconstruction, and matched with calculated result using scanning image of the same region with a certain interval time to illustrate the accuracy of the reconstructed image. After image reconstruction, the thermal drift velocity is calculated according to the position change and scanning time of the two nanoparticles in the horizontal and vertical directions. The calculated thermal drift velocity is −0.0320 nm/s in the horizontal direction and −0.0267 nm/s in the vertical direction. In order to verify the accuracy of the reconstruction algorithm, two AFM images are obtained by scanning the samples used in this book with 20 min interval and the same scanning angle. The change of the position of the same nanoparticles in the two images is divided by the interval time for calculating the velocity of thermal drift in the horizontal and vertical directions. Figure 3.19 shows the AFM image obtained by scanning the same sample at different times. In Fig. 3.19, (a) is the first image of the sample, (b) is the second image scanned after 20 min elapsing for the same sample, and (c) is the superposition effect of the two images.

P1

P3

P5

P1

P2

P3

P4

P5

P6

P1

P2

P2

P3

P4

P5

P6

P4 P6

P7

P7

P7

(a) AFM scanning image of nanoparticle at certain time

(c) Two images (a) and (b) are (b) AFM scanning image of nanoparticle after 20 minutes overlapped for representing thermal drift

Fig. 3.19 Comparison of images of the same region at different times

3.3 Simulation and Experimental Analysis

79

Table 3.2 Horizontal/vertical thermal drift speed (Unit: nm/s) Thermal drift speed

Particle 1

Particle 2

Particle 3

Particle 4

Particle 5

Particle 6

Particle 7

Horizontal Vertical

−0.0164 −0.0284

−0.0213 −0.0319

−0.0248 −0.0170

−0.0253 −0.0154

−0.0252 −0.0115

−0.0306 −0.0114

−0.0310 −0.0023

According to the displacement changes of six nanoparticles and one conjoined nanoparticles in the horizontal and vertical directions in Fig. 3.19c, the thermal drift velocities in the two directions are calculated as shown in Table 3.2. From Table 3.2, it can be seen that the thermal drift speed calculated by two different methods before and after reconstruction are consistent in magnitude, which illustrates the validity of the reconstruction algorithm.

3.3.2.3

Experimental Results of Global Reconstruction

At present, the algorithms for reconstruction of AFM thermal drift images only aim at local images, that is, only carry out a reconstruction at the salient feature areas in the scanned samples. Although this reconstruction algorithm can achieve better reconstruction effect in the local scan results, but there is no practical application value for solving the problem of AFM thermal drift images without global image reconstruction. Therefore, a whole-image reconstruction model based on the offset vector of the feature region is established in this book, which can effectively reconstruct the entire scanned sample and greatly improve the application value of the AFM image. The reference image of the scanned sample is reconstructed by the algorithm in this book. After reconstruction, the position coordinate information of the image is interpolated again to obtain the data information of the integer coordinate position. The interpolated image is represented in the form of BMP image as shown in Fig. 3.20. After image reconstruction, because the thermal drift affects any region in the image, the position information of each nanoparticle and the distance between the two adjacent nanoparticles also change with image reconstruction. The horizontal/ vertical distances of the two adjacent nanoparticles before and after reconstruction are shown in Tables 3.3 and 3.4. From the experimental results in Tables 3.3 and 3.4, it can be seen that the reconstruction algorithm in this book not only achieves good results in local reconstruction, but also makes some progress in global reconstruction, which provides more space for the wide application of AFM.

3 AFM Image Reconstruction Using …

80 Fig. 3.20 Entire image reconstruction result

P1

P2

P3 P4 P5

P6

P7

Table 3.3 Changes in horizontal interval of adjacent nanoparticles before and after reconstruction (Unit: nm) Distance Particles P2 and P1 Reconstructed image Reference image

Particles P3 and P2

Particles P4 and P3

Particles P5 and P4

Particles P6 and P5

Particles P7 and P6

1933.8000

−990.8669

634.6978

−1350.3000

3075.9000

−3575.500

1933.6000

−1043.6000

625.6132

−1359.6000

3070.5000

−3609.700

Table 3.4 Variation of vertical interval of adjacent nanoparticles before and after reconstruction Unit: nm

Reconstructed image Reference image

Distance Particles P2 and P1

Particles P3 and P2

Particles P4 and P3

Particles P5 and P4

Particles P6 and P5

Particles P7 and P6

5.0715

1690.5000

291.4714

296.9005

172.9347

1095.0000

4.9397

1646.6000

283.8915

289.1794

168.4374

1066.5000

References

81

References 1. Lv, J.J., Yao, J.J.: Aerial target localization based on least squares and newton iteration method. Microelectr. Comput. 28(9), 108–110 (2011) 2. Li, L.R.: Improvement of newton iterative method. China Water Transp.: Theor. Ed. 4(5), 204–206 (2006) 3. Yao, J.J., Han, Y.: Research on target localization method based on particle swarm optimization and Newton iteration method. J. Comput. Appl. 27(5), 1700–1701 (2010) 4. Deuflhard, P.: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, pp. 1299–1316. Springer Science & Business Media, Berlin, Heidelberg (2011) 5. Fu, X., Guo, B.L.: Overview of image interpolation techniques. Comput. Eng. Des. 30(1), 141–144 (2009) 6. Sheng, M.: Application Research of Nonlinear Interpolation Method in Digital Image Processing. Hefei University of Technology, Hefei (2009) 7. Ji, C.T., He, X.H., Fu, Y., et al.: An edge-oriented interpolation algorithm based on regularization. J. Electr. Inf. Technol. 36(2), 293–297 (2014) 8. Liu, H.C., Feng, Y., Yang, X.Q.: Subpixel image interpolation method based on bicubic B-spline surface. J. Harb. Inst. Technol. 39(7), 1121–1124 (2007) 9. Long, F., Wang, C.M., Sun, J.L., et al.: Thermal drift distortion correction of atomic force microscope image based on image registration. Nuclear Tech. 34(11), 877–880 (2011) 10. Sun, Y.F., Pang, J.H.L.: AFM image reconstruction for deformation measurements by digital image correlation. Nanotechnology 17(4), 933–939 (2006) 11. Lu, H., Cary, P.D.: Deformation measurements by digital image correlation: implementation of a second-order displacement gradient. Exp. Mech. 40(4), 393–400 (2000)

Chapter 4

AFM Image Reconstruction Algorithm Based on Tip Model

Abstract Aiming at the tip broadening effect in the AFM image as nano-manipulation environment map, this chapter introduces the mathematical basis of the AFM tip blind modeling theory, and studies the efficient tip estimation method. Based on the accurate reconstruction of environment map using the tip topography, the manipulation environment map is established closer to the real scene. Finally, the effectiveness of the model is illustrated by using image reconstruction experiments. Keywords Image reconstruction ogy Noise threshold




4.1 4.1.1


Tip blind modeling
Mathematical morphol-

Theoretical Basis of AFM Tip Blind Modeling Reconstruction Basic Concepts of Mathematical Morphology

The AFM scanning process can be described by mathematical morphology [1]. The basis of mathematical morphology is set theory, which has a complete mathematical basis, and lays a solid foundation for morphology in image analysis and processing, feature analysis and system design of morphological filters. When the tip scans the sample, our study range is the morphological information of the sample surface. From the perspective of mathematical morphology, the sample can be represented by the set S(x, y). The relationship satisfied by the elements in set S is the single value function s(x, y), (x, y) represents the horizontal and vertical coordinate of a certain point in the sample surface, and the value of s(x, y) represents the elevation of the sample surface. The point elements in S are included in the spatial shape of the sample. The AFM tip scans the upper boundary of the sample, so set S can be defined as: Authors: Shuai Yuan, Fangjun Luan, Jing Hou, Tianshu Chu. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_4

83

4 AFM Image Reconstruction Algorithm …

84

S ¼ fðx; y; zÞjz  sðx; yÞg

ð4:1Þ

From this definition, we can get that set S is composed of all point elements on the top surface of a set defined by the value of z = s(x, y) and point elements less than s(x, y). All point elements less than s(x, y) value are also called “Umbra of a surface”. Mathematical morphology is composed of a set of algebraic operators of morphology. There are four basic operations: Expansion, Corrosion, Opening and Closing. Before introducing these basic operations, we first introduce the translation, intersection, union, inclusion and mapping operations of sets in mathematical morphology. • The translation of a set: A þ d ¼ fa þ dja 2 Ag

ð4:2Þ

T½A þ dðx; yÞ ¼ sðx  dx ; y  dy Þ þ dz

ð4:3Þ

• The intersection and union of a set: T½A [ Bðx; yÞ ¼ max½aðx; yÞ; bðx; yÞ

ð4:4Þ

T½A \ Bðx; yÞ ¼ min½aðx; yÞ; bðx; yÞ

ð4:5Þ

• The inclusion relation of a set: AB: aðx; yÞ ( bðx; yÞ

ð4:6Þ

^ • The reflection operation of a set: the reflection of A is denoted as A ^ ¼ fxjx ¼ a; a 2 Ag A

ð4:7Þ

^ T½Aðx; yÞ ¼ T½Aðx; yÞ

ð4:8Þ

• The expansion operation of a set: A  B ¼ [ b2B ðA þ bÞ

ð4:9Þ

A  B ¼ max½aðx  u; y  vÞ þ bðu; vÞ

ð4:10Þ

ðu;vÞ

4.1 Theoretical Basis of AFM Tip Blind Modeling Reconstruction

85

The properties of expansion operation: AB¼BA

ð4:11Þ

A  ðB  CÞ ¼ ðA  BÞ  C

ð4:12Þ

ðA þ xÞ  B ¼ A  ðx þ BÞ ¼ ðA  BÞ þ x

ð4:13Þ

if

BC

then

ABAC

ð4:14Þ

• The corrosion operations of a set: AHB ¼ \ b2B ðA  bÞ

ð4:15Þ

AHB ¼ min½aðx þ u; y þ vÞ  bðu; vÞ

ð4:16Þ

ðu;vÞ

Operational properties: ðA þ xÞHB ¼ ðAHBÞ þ x or AHðB þ xÞ ¼ ðAHBÞ  x if

BC

then

A  BA  C

ð4:17Þ ð4:18Þ

• The opening operation of a set: A B ¼ ðAHBÞ  B

ð4:19Þ

Operational properties: A B¼

[

fB þ yjB þ yAg

ð4:20Þ

ðA  BÞ B ¼ A  B

ð4:21Þ

A BA

ð4:22Þ

A B ¼ ðA  BÞHB

ð4:23Þ

ðAHBÞ B ¼ AHB

ð4:24Þ

A BA

ð4:25Þ

• The closing operation of a set:

Operational properties:

4 AFM Image Reconstruction Algorithm …

86

4.1.2

Mathematical Description of Tip Imaging Process

Figure 4.1 shows the scanned image obtained by scanning the sample morphology using AFM. When the tip samples the height information of xi at a certain position on sample surface, the tip contacts with the sample surface and keeps a certain distance (the contact point is C, which depends on the set value of AFM feedback control). The height information of the tip center Ptop marks the height information of the image on the corresponding sample position. As for the sample morphology, scanning image and tip morphology in the process of tip scanning procedure, s, i and t functions are used to describe them respectively. s, i and t are functions of any position x in the planar region of the abovementioned morphology and image, which satisfy the relationship: iðxÞ ¼ max ½sðx0 Þ  tðx0  xÞ 0

ð4:26Þ

x

It can be abbreviated as: I ¼ S  ðTÞ

ð4:27Þ

where I, S, −T are the mappings of scanning image, sample morphology and tip morphology through the tip center. Because the tip modeling is to model the tip morphology (−T), P is used to simplify −T to describe the true morphology of the tip. In order to simplify the calculation, the origin of the tip morphology center is usually set at the apex of the tip (if there are multiple apex and the height is the same, then one of the vertices is chosen as the origin of the tip morphology center). That is: Pð0; 0Þ ¼ 0

ð4:28Þ

When the tip is imaged at a certain position on sample surface, if the tip touches the sample surface, the scanning image will not be distorted; otherwise, if the other xi

Scan Contact point

Tip

Ptop

P scanning at different positions

Fig. 4.1 Mathematical morphological description of the tip scanning the sample

4.1 Theoretical Basis of AFM Tip Blind Modeling Reconstruction

87

part of the tip touches the sample surface, the scanning image will be distorted due to the influence of “tip broadening effect”. According to the abovementioned definition and formulae (4.14) and (4.27), it can get: IS

ð4:29Þ

The above formula shows that the sample morphology is a subset of scanning, or the upper boundary of the scanned image contains the upper boundary of the sample morphology. Meanwhile, according to the description of mathematical morphology, it is deduced from formula (4.21): I P¼I

ð4:30Þ

According to formula (4.20), the following result is obtained: I ¼ [ fP þ yjP þ yI g

ð4:31Þ

According to the definition of formula (4.10), all boundary points in I are included in P and P′ after translation. P needs to satisfy the restriction condition when translation: P′ must be below the image I and cannot move over the boundary to the top of I. As shown in Fig. 4.2a, all boundary points in I can be contacted by P ′. The image of tip topography P scans the lower part of image I and contacts any point of the image, which does not cross the boundary and move to the upper part. In order to obtain the relationship between scanning image I and tip morphology P, now analyze the situation where P′ is in contact with boundary I at a certain position x, as shown in Fig. 4.2b. Assuming that P is in contact with a point in the image, the vector displacement of the contact point and the tip is d. When the contact point is different, d is also different. Therefore, d can be considered as a Scanning I

I Tip

Amplification

Contact point S

dx

P'

S Top of the tip P' at a certain scanning position

dz d

P

Sample contour

(a) Schematic diagram of the tip scanning the sample

Contact point

Sample contour

(b) Partial magnification of the tip at a certain point

Fig. 4.2 Mathematical representation of tip imaging process

4 AFM Image Reconstruction Algorithm …

88

function of x. The corresponding P′ after the tip motion can be expressed as P – d + x, and the condition of tip translation is that P′ below the image I, which is expressed as: 8x 2 I; 9d 2 PjPI þ d  x

ð4:32Þ

The upper bound of tip morphology P is also described, but d, x are based on the known P, and the current P is the variable that needs to be estimated, so d, x is unknown. Blind modeling algorithm deduces P by iteration based on this formula [1]. The algorithm framework is described as follows: (1) Assuming the initial upper bound P of the tip; (2) When estimating tip morphology at a certain position x in scanning image I, a point in P0 is selected to contact x, and d0 is obtained. According to formula (4.32), all possible contact points in P0, i.e. all values of d0 are estimated to estimate the possible upper bound of tip morphology. This process will be described detailly in the next section of the tip morphology estimation algorithm. (3) Performing the tip topography estimation in step 2 on all positions x of the image I for obtaining a new tip topography P1, and this calculation process is called the first iteration calculation; (4) Starting from P1, the process of estimating tip morphology is repeated until Pi converges (Pi does not update when estimating at all position X in image I), and the result of tip morphology estimation is obtained, then the calculation is completed.

4.1.3

Tip Morphology Estimation Algorithm

(1) Algorithmic Principle In the i-th iteration calculation of the algorithm, the upper bound of the tip topography is estimated as Pi, then there is P  Pi . When Pi estimates the tip morphology at a position x in I, because Pi is not the real tip morphology and the dj value is uncertain, it is necessary to assume many points in Pi (each point can be identified by three-dimensional coordinates dj because the tip vertex is defined as (0, 0, 0), dj can also be considered as a moving vector moving from the tip vertex to that point) may contact x. Then, the corresponding tip morphology is calculated with all the values in dj, and the upper bound of the tip morphology estimation at x position is obtained. The value of dj includes two cases, as shown in Fig. 4.3. In the first case, as shown in Fig. 4.3a, the scanning line is a dotted line, the profile of Pi is a solid line and larger than the real tip morphology. If the point corresponding to the current value of dj is a real contact point, then the intersection

4.1 Theoretical Basis of AFM Tip Blind Modeling Reconstruction

Pi

Contact point

dj

89

dj

Pi

Contact point

I-x' +dj

I-x+dj

(a) Estimation of tip morphology when Pi is at x and the tip top is under the image contour

(b) Estimation of tip morphology when Pi is at x' and the tip top is not under the image contour

Fig. 4.3 Estimation of tip possible morphology after P contacts with scanning image I at different points

of Pi and scan line I (shadow part), i.e. (I − x + dj) \ Pi, which describes a possible upper bound of tip morphology. The closest upper bound of tip topography is obtained by the union of the upper bounds of the tip topologies corresponding to all values dj in this case. In the second case, as shown in Fig. 4.3b, if the point corresponding to the current value of dj is the contact point, then P(0, 0) 6¼ 0 is obtained from the intersection of Pi and scan line I (shadow part). The result conflicts with formula (4.28). Therefore, the point corresponding to the current value of dj must not be the contact point. To avoid calculating the value of di in this case, the algorithm defines a constraint condition: Dx

pi

¼ fdjd 2 Pi and 0 2 I  x þ dg

ð4:33Þ

The constraint on d in the formula above is to limit the center P (0, 0) of the tip to the lower part of the scanning image I. Under this constraint, we define the expression for calculating the upper bound of the tip morphology at position x. 8x 2 I; P [ d2Dx

pi

ðI  x þ dÞ \ Pi

ð4:34Þ

According to formula (4.9), the formula on the right side of the formula above is simplified as follows: [ d2Dx

pi

ðI  x þ dÞ \ Pi ¼ ½ðI  xÞ  Dx

pi  \ Pi

ð4:35Þ

The i-th iteration of the algorithm is calculated for all positions x in I, so the formula can be written as:

4 AFM Image Reconstruction Algorithm …

90

P \ x2I ½ðI  xÞ  P0i  \ Pi

ð4:36Þ

According to the above calculation process, the formula of the first iteration is defined as: Pi þ 1 ¼ \ x2I ½ðI  xÞ  P0i  \ Pi

ð4:37Þ

After the tip model operation is repeated multiple times, if Pi−1 is not updated in the i-th operation, the calculation result converges to obtain PR, which is the upper bound of the real tip topography: PR ¼ lim Pi

ð4:38Þ

i!1

When using the above algorithm, it is assumed that the tip morphology is P0 (blue line rectangle), PR is obtained by reconstructing the tip morphology, and then the scanning image is reconstructed. As shown in Fig. 4.4, the surface morphology of the sample can be represented more accurately. (2) Discussion of AFM Tip Morphology Estimation In the process of calculating tip topography using blind modeling algorithm, the local topography of the tip is calculated by using each pixel in scanning image I and its neighborhood topography information, and the local topography of the tip is assembled into an entire tip topography. The algorithm needs to process every pixel in the reference surface image used to build the tip model, so the computation is very heavy. The noise in the scanning image I of calibrating tip morphology also has a great influence on the blind modeling algorithm. When the denoising threshold is not selected properly, the estimation of tip morphology of the algorithm will have a great error, as shown in Fig. 4.5. In Fig. 4.5a, b, the black outline is the true shape of the tip. The tip morphologies estimated by different noise thresholds are different. In the original tip blind modeling algorithm [1, 2], the selection of noise threshold is based on the optimal estimation method of increase volume d. That is, by increasing the noise T P0 PR P

20

0

30

y(nm)

y(nm)

10

-10

Sample surface Reconstruction image

20

Scanning image

10

0

-20 -8 -4

0

4

8

x(nm) (a) Blind modeling estimation of tip morphology T

0

20

40

80

60

100

120

x(nm) (b) Simulation of scanning and reconstruction image of sample surface

Fig. 4.4 Scanning image by using tip topography in the estimation algorithm

140

4.1 Theoretical Basis of AFM Tip Blind Modeling Reconstruction Real tip shape NoiseTH: 0.0 NoiseTH: 0.2 NoiseTH: 0.4 NoiseTH: 0.6 NoiseTH: 0.8 NoiseTH: 1.0 NoiseTH: 1.2 NoiseTH: 1.4 NoiseTH: 1.6 NoiseTH: 1.8 NoiseTH: 2.0 NoiseTH: 2.2 NoiseTH: 2.4 NoiseTH: 2.6 NoiseTH: 2.8

y (nm)

-5

-10

-15

-20 -25

-6

-4

-2

0

2

4

6

-10

-15

-20

-25 -4

35 30 25 20 15 10 5

0

0.5

1

1.5

2

2.5

NTH (nm) (c) Envelope area changing rate of the asymmetric tip contour when the noise reduction threshold is increasing

3

-2

0

2

4

6

x (nm) (b) Estimation of the symmetry tip morphology under different noise reduction thresholds Total area between adjacent curves (nm2)

40

0

Real tip shape Noise TH:0.0 Noise TH:0.2 Noise TH:0.4 Noise TH:0.6 Noise TH:0.8 Noise TH:1.0 Noise TH:1.2 Noise TH:1.4 Noise TH:1.6 Noise TH:1.8 Noise TH:2.0 Noise TH:2.2 Noise TH:2.4 Noise TH:2.6 Noise TH:2.8

-5

-6

x (nm) (a) Estimation of the asymmetry tip under different noise reduction thresholds Total area between adjacent curves (nm2)

-8

0

y (nm)

0

91

40

35 30 25

20 15 10 5

0

0

0.5

1

1.5

2

2.5

3

NTH (nm)

(d) Envelope area changing rate of the symmetric tip contour when the noise reduction threshold is increasing

Fig. 4.5 Optimal estimation of denoising threshold in the principle of blind modeling algorithm

threshold, the enclosure area d between adjacent tip contours corresponding to continuous thresholds is calculated. When the d value is the maximum, the threshold NTH is the optimal estimation value. In the experiment, the threshold value changes from 0 to 2.8, and the step is 0.2. Then the enclosure area change rate of the tip profile is calculated, as shown in Fig. 4.5c, d. Experiments show that this method is effective for asymmetric tip morphology estimation, as shown in Fig. 4.5a. However, it will fail when the tip with symmetrical morphology is estimated. As shown in Fig. 4.5b, when Noise TH = 1.4, d is the maximum, while the actual optimal noise reduction threshold is 2.0. There is no corresponding relationship between them. Therefore, it is necessary to make a new evaluation criterion to determine the optimal denoising threshold. In this book, a new algorithm is proposed to solve the above problems, which not only reduces calculation quantity, but also ensures that its calculation accuracy is the same as that of the original computation.

4 AFM Image Reconstruction Algorithm …

92

4.2

A Method for Improving the Speed of Tip Modeling Calculation

The original tip blind modeling algorithm involves all the pixels in the scanned image to participate in the calculation (The 512 512 pixel matrix is used in this experiment), and the information of the tip topography is calculated at each pixel point, and the calculation process needs to be iterated multiple times, so the calculation is heavy. In order to reduce the computational burden, this book first estimates the range of tip topography in scanning images, i.e. the radius of tip for determining the range of tip topography estimation. Then it accelerates the calculation speed of the tip estimation by ending the calculation in advance through evaluating whether the calculation at the pixel can effectively update the current tip topography.

4.2.1

Pre-estimation of Tip Morphology

In this book, the extreme points of the local area in the scanned image are used as feature points to participate in the tip modeling calculation and estimate the tip morphology. As shown in Fig. 4.6a, the tip morphology can be updated by calculating the intersection of overlaps with coinciding the central origin O of the upper bound of the tip morphology and the extreme point of the scanned image morphology. As shown in Fig. 4.6b, other points (except the central origin O) contacting with the sample surface is against the precondition of the algorithm, which is not allowed to occur. Therefore, the tip morphology can be quickly estimated by using local extremum points.

Feature point

Feature point

Pupperbound Pupperbound Image Image

Fig. 4.6 Estimating feature topography of the tip around the vicinity of the feature point

4.2 A Method for Improving the Speed of Tip Modeling Calculation

93

(1) Selection of Feature Points In the scan image I of the reference surface for calibrating the tip topography, there are a plurality of local topographies reflecting the tip information, and the vertex of the local topography is the feature point. This feature point is defined by the number of closed contours that surround the extremum points of a local region as type 1 or type 2 of feature points, as shown in Fig. 4.7c. Among these feature points, one type of feature points is shown in Fig. 4.7a, which has more tip morphology information. In this method, the type 1 extreme points is used as feature points to estimate the tip morphology. Searching for the type 1 feature points can be divided into two steps: • Based on the Douglas-Peuker curve feature point extraction algorithm (described later), the feature points (including type1 and type 2 of feature points) are determined in the horizontal and vertical direction of the image matrix. • According to the number of contours surrounding the feature points, the type1 feature points is determined in the set of feature points. Douglas-Peuker algorithm is a method for shape simplification of 2-D curves. The feature of this method is to select the main feature points which reflect the overall and Height (nm)

Feature points of class II 50

0 z (nm)

z (nm)

50

Feature points of class I

1000 800

1000

50 0 1000

800

600

1000 800

200

200 00

z (nm)

(b) Reference feature points in the surface image

20

640

0 180

z (nm)

140

z (nm)

50 0 1000 1000

800

800

600

600 400

200

200 00

(d) Contour line of the reference surface

Fig. 4.7 Feature point extraction in image I

200 00

0 40

(a) Reference surface 3D image

400

400

400

400

200

600

600

600

400

800

600

50 0 1000

1000

800

800

600

600 400

400 200

200 00

(e) Feature points I after classification

4 AFM Image Reconstruction Algorithm …

94

local shape of the curve from the complex curve point series by relatively simple global recursive operation. Therefore, this algorithm can be used to extract the feature points in AFM scan line. The basic principles of the algorithm are as follows: • In a series of curve points arranged in sequence (as shown in Fig. 4.8a, the starting point and the end point are connected by a straight line, which is called the baseline (dashed line). • If the midpoint number between the baseline ends is not zero, the distance from the midpoint to the baseline is calculated in turn. The points with the largest vertical distance to the baseline are selected and recorded as feature A. At the same time, these midpoints are divided into left and right regions, which are processed iteratively according to the next step, first left and then right, as shown in Fig. 4.8b. • If the distance is less than or equal to the pre-defined threshold rt, all midpoints between the baselines are deleted (Fig. 4.8c). If the distance is greater than rt, all midpoints are selected and inserted into the selected point list. Processing the next area according to this principle. Feature point A Maximum vertical distance

Start point

End

Feature point A Start point

Maximum vertical distance Feature point B

Start point

End

Maximum vertical distance End

Invalid point C

Fig. 4.8 Diagram of Douglas-Peuker algorithm

4.2 A Method for Improving the Speed of Tip Modeling Calculation

95

Fig. 4.9 Scope of tip morphology estimation

z (nm)

0

O(0,0)

-20 -40 -60

Vyb

200

Vxl

100

Vxr Vyf

y (nm) 0 -100 -200

-200

-100

0

100

200

x (nm)

(2) Estimating the Radius of Tip Morphology The size of the pixel matrix of the initial tip P0 is the same as that of the scanned image. The radius and initial shape of the tip P0 are estimated by formula (4.39) with the center of the matrix as the origin (0, 0). P0 ¼ fP0 ðx0 ; y0 ÞjP0 ðx0 ; y0 ÞIðxa þ x0 ; ya þ y0 Þg

ð4:39Þ

where (xa, ya) 2 [type 1 feature points], x′ 2 [−R, R], y′ 2 [−R, R], R can be defined as the initial value of 500 nm. After P0 is obtained, as shown in Fig. 4.9, the tip center is at O (0, 0). Because the tip is conical, and the spacing between the holes in the porous aluminum film is about 100 nm, which is also the lateral calibration range of the tip, only the lowest trough points Vxl, Vxr, Vyf and Vyb within 50 nm from the tip center are considered as boundary points. Usually the pixel matrix describing the tip is a square matrix, so the average distance between Vxl, Vxr, Vyf and Vyb and the center of the tip is calculated, and then the size of the describing square matrix is estimated as the tip radius R.

4.2.2

Improvement of Algorithm Core

As for the calculation process at the location of non-feature points in the scanned image, as shown in Fig. 4.10, it is assumed that the current tip upper bound is Pupperbound. According to the blind modeling algorithm, when the upper boundary of the tip is updated and calculated at point A in the scan line, since the precise shape of the tip is unknown, it can be assumed that the different positions C1, C2 and C3 of the tip are in contact with point A in the scan line. As shown in Fig. 4.10a, the corresponding series of tip topography are estimated. Because the information contained in the tip morphology can represent the real tip shape,

4 AFM Image Reconstruction Algorithm …

96 C1

C2 C3

Pupperbound I A

(a) The tip upper bound contacts the (b) The sample surface contacts the sample surface at C1, C2, C3 tip upper bound at A I

I

Pupperbound

I

A P upperbound

A

Pupperbound

C1

C3

β

α

γ

(d) The tip contour is estimated (e) The tip contour is estimated as γ at C2 and A as at C1 and A

(c) The tip contour is estimated as α at C3 and A

β α

A

C2

αUβ Uγ

Pupperbound

γ β α

γ

(f) The tip contour is updated using (g) The updated tip contour is zoomed in using α, and γ on the points C1, C2, C3. α, and γ

Fig. 4.10 The process of updating the top boundary P of the tip topography at point A in image I

combination of them is performed to obtain a more accurate upper bound for description of the tip morphology. According to the previous description, when estimating the tip morphology at a pixel point in the scanned image, the conditional judgment expression for judging the possible contact position (point) in the upper boundary Pi (the i-th iteration) of the tip morphology is Eq. (4.32), in which d represents the possible contact point between the upper boundary of the tip morphology and point A in the scan line

4.2 A Method for Improving the Speed of Tip Modeling Calculation

97

I. For all possible contact points in Pi, after calculating the tip morphology at each contact point, the combination of these tip morphologies is calculated to obtain the upper bound of the tip morphology. Figure 4.10c is the estimated tip topography a when the C3 point in the Pupperbound is in contact with point A. At this time, the tip vertex C1 has intersected with the scanning line. C3 point is the rightest contact point of the possible contact area in Pupperbound. At the same time, a represents the upper contour morphology around the tip vertex. Figure 4.10d assumes that when the C1 point in Pupperbound is in contact with point A, the tip morphology b is estimated. At this time, C1 has intersected with the scanning line, so C1 point is the most left contact point of the possible contact area in Pupperbound, and b is also the upper boundary of the contour morphology around the tip vertex. In Fig. 4.10e, it is assumed that when C2 point (C2 < [C1, C3]) in Pupperbound is in contact with point A, a possible morphology of the tip is estimated. Figure 4.10f is the upper boundary of the tip profile after combining a, b and c. Figure 4.11 illustrates the calculation process of updating the upper bound of tip morphology by using blind modeling algorithm at B point position in scanning line. Because the upper bound of the tip morphology cannot be updated when calculating the tip morphology at some points, as shown in Fig. 4.11.

C4 C5 Pupperbound

I

B

(a) The tip upper bound contacts the sample surface at C4, C5. I

(b) The sample surface contacts the tip upper bound at B.

I

Pupperbound Pupperbound B

B

α

(c) The tip contour is estimated as α at C5.

αUβ

C4

C5

β

(d) The tip contour is estimated as at C4.

α β

(e) The tip contour is updated using α and

Fig. 4.11 The upper bound P of tip morphology cannot be updated at point B in image I

4 AFM Image Reconstruction Algorithm …

98

The tip morphology is estimated at B position in scan line I. It is assumed that Pupperbound contacts B point through two points C4 and C5, the upper bounds a, b of tip morphology are estimated, and the combination of them (red line representation) is calculated. This combination coincides with the Pupperbound contour (blue dotted line). It shows that the calculation results at the position of B point in scan line cannot update the tip morphology effectively. These numerous invalid computations will result in time-consuming and inefficient tip modeling. Therefore, when the blind modeling algorithm calculates at a certain point in the scanning line, it is necessary to in real time judge whether the current calculation results can update Pupperbound, and if so, continue to calculate; otherwise, end the calculation at that moment. In this way, the ineffective calculation can be avoided and the calculation speed can be improved. The flow chart of the above calculation process is described as follows, as shown in Fig. 4.12. Pi_x in Fig. 4.12 is the result of updating the tip morphology Pi on the x-pixel of I by using the improved algorithm. For different contact points dj in Dx_pi, the result of one estimate are as follows: d

Pi j ¼ ðI  x þ dj Þ \ Pi

j ¼ 0; . . .; m2  2

ð4:40Þ

In the formula, m is the rank describing the Pi pixel matrix, and the center of the pixel matrix is set to the tip vertex (0, 0, 0). The tip estimation takes into account the values of other pixel points, so it is usually necessary to calculate m2 − 1 times. In

Start

Take dj ( j=1 ) in Dx_ pi , and calculate (I - x - d1) Pi according to formula 4.40

Pi d1

Pi _ x

N

Pi d1

Pi _ x

Pi _ x

Pi Y

Whether dj in Dx_ pi is all involved in calculation

Pi

dj

Pi _ x

Take dj ( j=j+1 ) in Dx_ pi, and calculate d the tip morphology Pi j according to the formula 4.40 N

Y End

Fig. 4.12 Flow chart of the new blind modeling algorithm

4.2 A Method for Improving the Speed of Tip Modeling Calculation

99

formula (4.40), d is the vector identification of the position of Pi at contact point x. In the original algorithm, all d in Dx_pi need to be calculated. In the new algorithm, with the change of x, d in the corresponding Dx_pi may not be all involved in the calculation, which reduces the computational complexity of the algorithm [3].

4.3

Method for Improving the Accuracy of Tip Modeling

The scanning image of AFM can be regarded as the convolution result of the sample and the tip morphology, as described in Eq. (4.26). The AFM image reconstruction based on the tip morphology is a kind of inverse operation of the abovementioned formula, such as Eq. (4.30). From this formula, it can be seen that the reconstructed image I depends on the correctness of the tip morphology model, and the noise in the calibration image of the tip has a decisive influence on the result of the tip modeling, so it is necessary to estimate the optimal denoising threshold in the image. The optimal estimation of noise reduction threshold in the algorithm is studied in the simulation experiment. In the simulation experiment, Gaussian noise with RMS 1 and mean 0 is added to the scan line in Fig. 4.13a, as shown in Fig. 4.13b, c is an enlarged area in the dotted circle.

4.3.1

Definition of Denoising Threshold

As shown in Fig. 4.14, there are some “wrinkles” in the estimation results of tip topography due to the influence of noise in the image. As the number of iterations

y (nm)

30

Scanning profile Sample morphology

20 10 0 0

20

40

60

80

100

120

140

160

180

200

x (nm) 1.0

30

0.5

28

y (nm)

y (nm)

(a) The true sample morphology and scanning profile

0

-1.0

26 24

-0.5

22 0

20

40

60

80

100

120

140

160

180

200

x (nm) (b) Random noise added into the sample scanning profile

48

50

52

54

x (nm) (c) Local contour after noise is added

Fig. 4.13 Adding Gaussian white noise to the scanning curve for tip modeling and estimation of optimal noise reduction threshold

4 AFM Image Reconstruction Algorithm …

100

Pnewbound Pupperbound

(b) Partial enlarged view of the tip contour

(c) Comparison of the updated area with the original upper bound contour

y

(a) The tip contour updation

NTH

Pcomputed

xj xj+1 x xi xi+1 (d) Definition of noise reduction threshold

Pnewbound

(e) Updating the tip profile using the noise reduction threshold

Fig. 4.14 Definition and use of noise threshold in tip modeling

increases, the influence of the noise will be amplified many times, resulting in serious distortion of tip modeling. In order to reduce the influence of noise on tip modeling, the Noise TH threshold is defined. In Fig. 4.14a–d, the dotted broken line is the upper bound of the tip morphology, and the red line is the current calculation results Pnewbound. The updated region of the upper bound are enlarged step by step from xi to xj+1. Since there is noise in the scanning line, the height difference between the Pnewbound contour and the Pupperbound contour is defined as the result of noise interference if its value is smaller than the noise reduction threshold NTH, while the change beyond the range of NTH is defined as the difference caused by the real shape of the tip. Therefore, the region from xi to xi+1 and from xj to xj+1 is defined as the difference caused by noise interference without updating; the region from xi+1 to xj is defined as the effective updating, and the updating contour of the upper bound of tip morphology corresponding to the noise reduction threshold is shown in the figure.

4.3 Method for Improving the Accuracy of Tip Modeling

101

0

Noise TH: 0.0 Noise TH: 0.6 Noise TH: 1.2 Noise TH: 1.8 Noise TH: 2.4 True contour

0 -0.1

-5

y (nm)

y (nm)

-0.2 -10 -15

-0.3 -0.4 -0.5 -0.6

-20

-0.7 -0.8

-5

0 x (nm)

-1

5

14

40 Total area between adjacent curves (nm2)

Number of top vertices

0 x (nm)

12 10

(c)

8 6

4 2

0 0.5

1 1.5

2

2.5

3

NTH (nm) (c) The number of top vertices of the tip contour estimation under different noise reduction thresholds

0.5

1

(b) Top magnified image of tip contour estimation under different noise reduction thresholds

(a) Estimation of the contour of asymmetric tip

0

-0.5

35 30 25 20 15

10 5 0

0

0.5

1

1.5

2

2.5

3

Calculating the total area between the curve and the actual curve (nm2)

-25

70 60 50

40 30

20 10

0

0 0.5

1 1.5 2 2.5 3

NTH (nm) NTH (nm) (d) Change rate of the contour (e) Errors between the estimated contour and the true contour of the tip with envelope area with variation of different noise reduction thresholds the noise reduction thresholds

Fig. 4.15 Estimation of optimal denoising threshold for asymmetric tip topography

4.3.2

Estimation of Denoising Threshold

In the process of establishing tip model by using blind modeling algorithm, noise reduction threshold has a decisive influence on the estimation of tip morphology [5, 6]. The experimental results in Fig. 4.15 show that the method is effective for asymmetric tip morphology estimation, but it will fail for asymmetric tip morphology estimation, as shown in Fig. 4.16a–e. Therefore, it is necessary to study a new evaluation criterion to determine the optimal denoising threshold. When the denoising threshold in the algorithm changes from small to large, the tip morphology changes from sharp to blunt and gradually expands from inside to outside. It can be assumed that there is only one vertex in the real shape of the tip. When the top of the tip model changes from one to more, it is assumed that the current tip model is close to the real tip shape. So in the algorithm, the number of the highest points on the top of the tip is changed from one to multiple denoising thresholds, which are defined as the critical denoising threshold. Figure 4.16a is a tip model calculated when the noise reduction threshold varies from 0 to 2.4. Figure 4.16b is an enlarged image of the top of the tip. Figure 4.16 shows the estimated contour and the real tip morphology of the noise threshold from 0 to 2.4.

4 AFM Image Reconstruction Algorithm …

102 0

-0.2 y (nm)

y (nm)

Noise TH: 0.0 Noise TH: 0.6 Noise TH: 1.2 Noise TH: 1.8 Noise TH: 2.4 True contour

0

-5 -10

-15

-0.4 -0.6 -0.8

-20

-1 -4

-2

0 x(nm)

2

4

-1.5

6

Total area between adjacent curves (nm2)

Number of top vertices

-0.5

0 0.5 x(nm)

40

12 10 8 6 4 2 0 0

-1

1

2

3

NTH (nm) (c) The number of top vertices of the estimated tip contour under different noise reduction thresholds

35

30 25 20 15

10 5 0

0

1

(b) Top magnified image of tip contour estimation under different noise reduction thresholds

(a) Estimation of the contour of symmetric tip

0.5 1

1.5

2

NTH (nm)

2.5

3

(d) Change rate of the tip contour enclosed area under the variation of the noise reduction thresholds

Calculating the total area between the curve and the actual curve (nm2)

-25 -6

100 80 60

40 20 0

0

0.5 1

1.5 2

NTH (nm)

2.5 3

(e) Errors between the estimated contour and the true tip contour with different noise reduction thresholds

Fig. 4.16 Estimation of optimal denoising threshold for symmetric tip topography

When TH = 2.2, this value is the critical denoising threshold (Fig. 4.16c). The optimal denoising threshold is found near the critical denoising threshold. Through the incremental change of the noise threshold, the change rate of surround area between the tip adjacent contours corresponding to the continuous threshold d is calculated. When the maximum value of d is obtained, NTH is the optimal estimate. In Fig. 4.16d, the local maximum of delta is found near the critical threshold, and NTH = 2.0 is obtained. In Fig. 4.16e, when NTH = 2.0, the difference between the tip model and the real tip shape is the smallest, which illustrates the validity of the abovementioned optimal denoising threshold. Figure 4.15 can also draw the same conclusion.

4.4 4.4.1

The Experiment of AFM Image Reconstruction Tip Topography Estimation

Figure 4.17a is the scanning image for calibrating porous aluminum film. The shape information of the tip is contained in the feature points and their adjacent morphologies in the image. It is necessary to use these feature points and their adjacent

Height (nm) 180

0

(a)0esh generation of porous aluminum image

103 Calculation time (×1000s)

4.4 The Experiment of AFM Image Reconstruction

12

Original algorithm

10 8 6 4 2 

Improved algorithm 0

10 20 30 40 Calculation amount (×1600)

Calculation amount (*1600) Calculation time of the original algorithm(s) Calculation time of the improved algorithm(s)

4

9

16

25

1780

2843

6322

8537

141

185

205

225

(b) Comparison of calculation time between improved algorithm and original algorithm

Fig. 4.17 Model of tip topography

information to establish the tip morphology. According to the abovementioned description, the radius of tip morphology is estimated at the feature point, which is about 40 nm. In order to illustrate that the improved blind modeling algorithm can effectively improve the computing speed, the feature points in the image are divided into several regions according to the grid (the number of pixels in each region is 40 40). The more the number of regions involved in calculating the tip morphology, the more accurate the tip morphology is, and the longer the computing time is. This book will gradually increase the computational area. As shown in Fig. 4.17b, the computational time before and after the improvement of Blind Modeling algorithm is compared to illustrated the effectiveness of this method [6]. Figure 4.17b shows that with the increase of the number of pixels in the image, the execution time of the improved algorithm is much less than that of the original tip blind modeling algorithm. When the number of pixels rises to a certain value (16 1600), since the improved algorithm converges quickly and the increase of data has little effect on the calculation time, the improved algorithm runs slowly, while the original algorithm shows a linear growth trend. Figure 4.18 shows the experimental verification results of the tip modeling algorithm. Figure 4.18a is a tip image taken by SEM. The outline of the tip is extracted and compared with the corresponding profile of the tip 3D profile (Fig. 4.18b) established by the abovementioned algorithm. When the threshold TH = 4.53, it is the optimal noise reduction threshold, and the corresponding red thick line is the tip profile. The approximation degree between the line and the contour (* point contour) extracted from the SEM image is the best, which illustrates the correctness and validity of the tip modeling algorithm, as shown in Fig. 4.18c.

4 AFM Image Reconstruction Algorithm …

104

NTH =4.56nm 0

0

NTH =4.53nm -20

z(nm)

z(nm)

-40

-80

NTH =4.51nm

-40

NTH =4.5nm

-120 50

50

y(nm) (a) SEM image of the tip and the extracted contour

-60

0

0

-50

-50

x(nm)

-50

(b) Estimation of the tip morphology by improved blind modeling algorithm, and fetching the tip profile contour

50

0

x(nm)

(c) Comparison between the contour extracted from SEM image and the profile contour of estimation result

Fig. 4.18 Verification of the results of a new tip blind modeling algorithm

4.4.2

Scanning Image Reconstruction of Carbon Nano-tubes and Nano-particles

Carbon nanotubes (CNTs) are regular tubes, and their scanned images can also be used to verify the effectiveness of tip-based scanned image reconstruction. The carbon nanotubes before reconstruction are shown in Fig. 4.19a. The height and Scanning contour Reconstruction of scanning contour

48nm 29nm

2

z (nm)

200 0

150

250 profile (nm)

200

1

300

350

(c) Profile at position 0 0

Scanning contour Reconstruction of scanning contour

(a) Carbon nanotube image before reconstruction

z (nm)

49nm 29nm 200 0 150

300

200 250 profile (nm)

(d) Profile at position 1

2

0 (b) Carbon nanotube image after reconstruction

Scanning contour Reconstruction of scanning contour

48nm 30nm

200

z (nm)

1

0 0

20

40

60

80 100 120 profile (nm)

140 160

180 200

(e) Profile at position 2

Fig. 4.19 Width comparison of scanning and reconstructed carbon nano-tube image

4.4 The Experiment of AFM Image Reconstruction

105

Height (nm) 50

Contour line of the new tip Reconstructed contour line z(nm)

29nm 20 10 0 200

0.0

512.0

(a) Carbon nanotube image scanned by the new tip

250

profile (nm)

300

350

0

(b) Comparison between the reconstructed contour and the profile line extracted from the new tip image

Fig. 4.20 Reconstruction of scanning image based on tip model using new needle scanning carbon nanotube

width of the carbon nanotubes are about 20 nm and 49 nm respectively. Figure 4.19b is the result of reconstructed tip with a width of about 29 nm. The scanning lines before and after reconstruction are compared at three positions of the nanotubes, as shown in Fig. 4.19c–e. In theory, the actual width of carbon nanotubes should be consistent with its height (about 20 nm), but because of the broadening effect of the tip, the width of the scanned image is far from the actual width, so it needs to be reconstructed. In the experiment, the width of the reconstructed image of carbon nanotubes is about 20 nm narrower than that of the original scanned image. By comparing the scanning lines before and after reconstruction with those of the new needle (fine needle tip), we can see that the reconstruction process can improve the quality of the scanned image and make the scanned image closer to the real shape (Fig. 4.20). In the tip localization research and manipulation experiments, polystyrene nanoparticles with a diameter of about 200 nm produced by Polyscience Company were used as landmarks, and the particles presents regular spherical shape [7]. To illustrate the validity of scanning image reconstruction based on the tip. The nanoparticles before and after reconstruction are shown in Fig. 4.21a, d. Comparing the scanning lines before and after reconstruction in the horizontal and vertical directions of nanoparticles, as shown in Figs. 4.21e, f, the reconstructed image can be more approximate to the real particle morphology.

4 AFM Image Reconstruction Algorithm … 200 0

V

225nm H

200 150 100 50 0 -50 400 2000

z (nm)

z (nm)

106

500nm

0

500

1000

1500

x (nm) (b) Three-dimensional image of the nanoparticle before reconstruction

0

Scan contour Reconstruction contour

800

1000

1200

x (nm)

(c) Horizontal cross-section profile of the nanoparticle before and after reconstruction 224nm

0

V

H

500nm (d) Nanoparticle image after reconstruction

600

200

z (nm)

z/nm

(a) Nanoparticle image before reconstruction

175nm

H

500

1000

1500

200 150 100 50 0 -50 800

V

1000

Scan contour Reconstruction contour

176nm

1200

y (nm)

1400

1600

2000

x (nm) (b) Three-dimensional image of the nanoparticle after reconstruction

(f) Horizontal cross-section profile of the nanoparticle before and after reconstruction

Fig. 4.21 Scanning image reconstruction of polystyrene nanoparticles with diameter of about 200 nm

References 1. Villarrubia, J.S.: Morphological estimation of tip geometry for scanned probe microscopy. Surf. Sci. 321(3), 287–300 (1994) 2. Villarrubia, J.S.: Algorithms for scanned probe microscope image simulation, surface reconstruction, and tip estimation. J. Res. Nat. Inst. Stand. Technol. 102(4), 425–454 (1997) 3. Yuan, S., Luan, F.J., Song, X.Y., et al.: Reconstruction of an AFM image based on estimation of the tip shape. Meas. Sci. Technol. 24, 105404 (2013) 4. Dongmo, L.S., Villarrubia, J.S., Jones, S.N., et al.: Experimental test of blind tip reconstruction for scanning probe microscopy. Ultramicroscopy 85, 141–153 (2000) 5. Tranchida, D., Piccarolo, S., Deblieck, R.A.C.: Some experimental issues of AFM tip blind estimation: the effect of noise and resolution. Meas. Sci. Technol. 17, 2630–2636 (2006) 6. Yuan, S., Dong, Z.L., Miao, L. et al: Research on the reconstruction of fast and accurate AFM probe model. Chinese Sci. Bullet. 55(24), 2750–2754 (2010) 7. Yuan, S., Yao, X., Luan, F.J., et al.: Accurate and rapid modelling of AFM tip morphology through scanning sphere nanoparticles. Int. J. Simul. Process Model. 12(3/4), 328–337 (2017)

Chapter 5

Stochastic Approach Based AFM Tip Localization

Abstract In nano-environment, besides the non-linearity and thermal drift of PZT, there are other uncertainties such as noise of control system and vibration of working environment, which also bring the uncertainty into tip localization. Similar to mobile localization in macro robots, the uncertainty of tip position increases with time and tip motion. In this chapter, aiming at the abovementioned problems, a tip localization and path planning method based on stochastic approach is proposed. The feature on the sample surface is marked as a landmark. The real-time tip position is estimated in task space coordinate system using landmark by observing the spatial distance relationship between the tip and the landmarks. In this method, the uncertainty of tip position in task space is represented by probability distribution. The tip motion model is established by PI model, creep model and thermal drift model. Combining with the observation model based on local scan, the optimal estimation of tip position is performed by using Kalman filter, which realizes the real-time determination of tip position with high accuracy, high resolution and low cost in manipulation. For implementation of the algorithm, a statistical experimental scheme is designed to calibrate the model parameters. Finally, the simulation and experimental results illustrate the effectiveness and feasibility of the algorithm, which avoid the influence of PZT nonlinearity and thermal drift, and realize the precise tip localization in task space coordinate system.







Keywords Tip localization Landmark localization Path planning Creep model Thermal drift model Local scan Kalman filter




5.1 5.1.1








PI model


Research of AFM Tip Localization Stochastic Approach Based AFM Tip Localization Strategy

In generally robotic system, closed-loop control based on state observation is the key technology to ensure the control accuracy of terminal actuator. High bandwidth Authors: Shuai Yuan, Lianqing Liu, Zhidong Wang, Ning Xi. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_5

107

5 Stochastic Approach Based AFM …

108 Manipulation task

Path planning

Motion controller

Operational actuator

End effector motion

Sensor perception

Fig. 5.1 Path planning and control system based on closed-loop feedback

frequency response characteristics are the prerequisite and important guarantee for the stability of the control system. Recently, most commercial AFM systems use closed-loop position feedback control to ensure the position accuracy of tip scanning and imaging. The control system is shown in Fig. 5.1. Unlike nano-observation imaging, AFM nano-manipulation usually uses tip to push nano-objects into the target area. Figure 5.2 shows an example of assembling a nano-device in which a tip pushes a nano-rod between electrodes. In the process of nano-manipulation, the traditional position feedback control based on scanning image will no longer be suitable for nano-manipulation, because various uncertainties in the nano-environment will directly affect the distribution area of tip position. Without algorithm compensation, the distribution region of tip position will be larger than that under closed-loop control. If the initial distribution region of the tip position is larger, the tip will not contact the nano-rod at the expected position under the open-loop control, which will make the position and rotation angle of the nano-rod more uncertain after one manipulation [1]. It may lead to the failure of nano-rods to connect between electrodes, then resulting in the failure of nano-assembly tasks, as shown in Fig. 5.2. In this situation, it is necessary to carry out further processing, such as re-imaging and re-execution of push operation, which makes it difficult for current nano-assembly to achieve the efficiency of macro-automatic assembly in factories. Electrode

AFM Tip

Electrode

AFM Tip

Nanotube

Nanotube Initial position uncertainty Nano-assembly task

Nanoassembly task

(a) Diagram of nanorod assembly using AFM (b) Diagram of assembly task failure using AFM Fig. 5.2 Device assembly diagram using general AFM nano-manipulation

5.1 Research of AFM Tip Localization

109

In the macro mobile robot localization method, the position estimation based on landmark observation can reduce the localization error and uncertainty. This method is especially an effective strategy for those uncertainties that increase with time or motion. The method based on landmark observation needs to find some nano-features in task space like nanoparticles. These features cannot be moved directly by the AFM tip in uncertain and changing nano-environment, which can remain static, so they can be used as feature landmarks. In macro environment, mobile robots can assemble some position sensors, such as laser rangefinder and ultrasonic rangefinder for landmark observation, but in nano-environment, it is difficult to install sensors on the tip to detect the nano-feature landmarks in task space due to the limitation of the spatial scale of the tip and the resolution of the sensor. However, we can use the existing method—local scan method to observe the localization of feature landmarks, and estimate the localization of the tip according to the spatial relationship between the tip and the landmark. In nano-manipulation, the tip is firstly moved to the vicinity of the nano-rod (target position), and the tip position needs to be kept in the limited position distribution area, then the nano-rod is pushed between the electrodes for assembly of nano-devices. In order to accomplish this assembly task, the tip is not moved directly to the target position (as shown in Fig. 5.3), but needs path planning. First, the tip is moved to the nanoparticle near the nano-rod for landmark observation (local scan). When the tip position is updated (position uncertainty is reduced), the tip is moved to the target position for push manipulation. This mode of tip motion will effectively improve the success rate of nano-manipulation. There are also errors and position uncertainties in the process of landmark observation, so optimal estimation of tip position will be performed by probability and statistics method. Based on this, a new method for estimating the tip position is proposed, i.e. landmark observation and tip localization based on stochastic approach. In order to improve the accuracy of the tip position, the center of the landmark (nanoparticle) will be observed intermittently, which is similar to the localization method of the macro robot in Fig. 5.4a by observing the spatial relationship between itself and the wall, corner and cylinder. In Fig. 5.4b, the tip position will be

Fig. 5.3 Successful implementation of the nano-assembly task by combining stochastic tip localization and path planning method

AFM Tip

Electrode Nanoparticle Nanotube

Local scan

Nanoassembly task

5 Stochastic Approach Based AFM …

110

lh lv

z (nm)

300

y

200 100 0 3.0

y (µm)

0

xk

3.0

Tip

400

0 3.0

P1

Nanoparticles

Nanoparticles

Thermal drift direction

1.8

y (µm)

x (µm)

lv

lh

P1'

xk+i

1.8

y (µm) 0.6

0.6 0.6

1.8

3.0

2.5

(b) Scanning profile of the nanoparticle as landmark feature

z (nm)

z (nm)

Tip

2.0

2.0 1.5

Cylinder P4 x (a) Macroscopic robotic localization and mapping using environmental feature

400

2.5

3.0

0.6

x (µm)

3.0

t2 = 18 min

t1 = 0 min

(c) Scanned image of the nanoparticle at a certain moment

1.8 x (µm)

(d) Scanned image of nanoparticles after 18 minutes

Fig. 5.4 Tip localization strategy based on stochastic approach

estimated by observing the spatial relationship between itself and the localization of the landmark center based on local scan. Figure 5.4c shows the state of the AFM tip at t1 = 0. After 18 min, affected by system thermal drift, the nanoparticle drift from P1 to P′1, and the tip will move downward and right relative to the sample surface. In order to find the current position of the tip, the algorithm will control the tip to scan the nanoparticle along horizontal direction lh and vertical direction lv, so that it can obtain the central position of the nanoparticle P′1 and the relative distance between the current position of the tip xk+i and the center of nanoparticle. According to the position of P′1 in task space, the current position of the tip xk+i is estimated. The position uncertainty of the AFM tip increases with the increase of motion and time. The current position of the tip is observed in real time by locally scanning the feature morphology (landmark) of the sample surface, and the Kalman filter is used to estimate the optimal position of the tip by combining the input control of the driving tip motion. The algorithm has two distinct characteristics:

5.1 Research of AFM Tip Localization

111

(1) Localization Strategy Based on Landmark Observation As for the position uncertainty between the tip and the sample surface, this algorithm uses the simultaneous localization and mapping (SLAM) algorithm [2–4] of macro robot localization to estimate the current position of the tip in real time by observing landmarks. (2) Stochastic Approach Based Probability Representation Compared with the traditional method, this method uses probability to describe the uncertainty of the tip position, establishes the motion model and observation model of the tip, and then uses the optimal estimation filter to improve the accuracy of the tip position.

5.1.2

Nano-manipulation Coordinate System Defined on AFM

(1) Definition of Coordinate System Three coordinate systems are established in the stochastic approach based tip localization strategy using landmark. As shown in Fig. 5.5a, a task space coordinate system is established on sample surface, and coordinates of the tip motion and manipulation are all defined in this system. The origin definition of task space

Feedback system

y 1

PZT PSD

x

2 3

Task space coordinate system

Tip position

(b) AFM image coordnate system

y x y (nm)

z

Push

200 100

0

(a) AFM manipulation coordinate system

0

0.2

0.4 0.6 x (µm)

0.8

1

(c) Scanning coordinate system

Fig. 5.5 Definition of coordinate system for motion control of AFM tip

5 Stochastic Approach Based AFM …

112

coordinate system is similar to the world coordinate system of macro robot. It can be defined at any position. Generally, it is defined on the feature points related to nano-manipulation. The z-axis of the coordinate system is based on the height of the sample surface, and the x-axis and y-axis are based on the plane where the nano-objects and nano-devices are located. A two-dimensional image coordinate system is established on the scanned image (map) of the AFM. As shown in Fig. 5.5b, the image will be mapped on the xy plane of task space coordinate system to perform tip trajectory planning using local scan landmark observation. In order to simplify the control of the tip motion model, the origin of the coordinate system is defined in the center of the scanned image, and the x-axis is established in the horizontal direction of the scanned image. The scanning coordinate system is established on the height profile of local scan, as shown in Fig. 5.5c. The coordinate system contains the offset distance and height information of each sampling point on the local scan trajectory relative to the starting point. The relative distance between the tip and the landmark needs to be further processed by the observed model when the topographic information of the sample on the local scan line is used as the signal output of the landmark observation. (2) Definition of Tip Position State The position of the AFM tip at time k is expressed by xk. The state is a two-dimensional vector in task space coordinate system and image coordinate system. The set or trajectory of a series of tip positions is expressed as follows: XK ¼ fx0 ; x1 ; x2 ; . . .; xk g

ð5:1Þ

The control quantity of the AFM tip at time k is expressed by uk. The input quantity is the control quantity that transfers the tip state quantity from time k to time k + 1. A set of control variables can be expressed as follows: UK ¼ fu1 ; u2 ; u3 ; . . .; uk g

ð5:2Þ

Each control variable uk in the abovementioned formula, i.e. the displacement of the AFM tip is compensated nonlinearly by PI model, creep model and thermal drift model, because it is difficult to obtain the ideal compensation model and accurate model parameters, the uncertainty of the tip position will gradually increase with the input of the control variable in task space coordinate system. In order to express the uncertainty of tip position, the tip motion model will be presented by probability method. In the following formula, the state xk+1, i.e. the position state of the tip at time k + 1 will be represented by the probability statistical model of the tip motion below, which includes the position state xk and the control quantity uk at the previous time of the tip. xk þ 1 ¼ gðxk ; uk Þ þ wk þ 1

wk þ 1  Nð0; Rk þ 1 Þ

ð5:3Þ

5.1 Research of AFM Tip Localization

113

g(*, *) is the state transition function of the tip motion model. wk+1 is the perturbation variable of the tip position. It obeys the normal distribution. Rk+1 is a two-dimensional covariance matrix, which is similar to the macro mobile robot [5– 8]. The state transition function is the linear superposition result of the position xk and the motion control quantity uk at the previous moment of the tip. In the formula, uk is the control variable compensated by the PZT non-linear hysteresis model. In addition, the creep distance dk of PZT and the thermal drift vk  Δt are added to the formula to correct the tip motion model: gðxk ; uk Þ ¼ xk þ uk þ dk þ vk Dt wk þ 1 ¼ wk

h

þ wk

c

þ wk

d

ð5:4Þ ð5:5Þ

where vk is the velocity of thermal drift, and Dt is the time interval between the tip motion from xk to xk+1. Since the tip works in a small area around the central axis of the PZT tube to perform scanning and manipulation, under the condition that the input voltage varies at a certain rate, vk, dk and uk can be considered to be independent with time change. Therefore, the perturbation of the tip position wk+1 is considered as the linear superposition of the perturbation random variables wk_h, wk_c and wk_d corresponding to the three factors uk, dk and vk. These perturbation factors obey the Gauss distribution, and the corresponding parameters are calibrated by experiments in the latter part.

5.2 5.2.1

Analysis of Landmark Observation Model Based on Kalman Filter Landmark Definition

Before AFM nano-manipulation, it is necessary to image the manipulation area and obtain a prior map, as shown in Fig. 5.6, then mapping the prior map to task space coordinate system. In the prior map, nanoparticles or nano-rods with characteristic morphologies are selected as landmarks, and the position of these landmarks is recorded in set M. As shown in Eq. (5.6), m represents the position of the landmarks, and N is the total number of landmarks. MN ¼ fm1 ; m2 ; m3 ; . . .; mN g mj ¼ mj;xy þ vmap

vmap  Nð0; Qmap Þ

ð5:6Þ

where mj, xy is the position of nanoparticle in task space coordinate system as the jth landmark. vmap is a random error variable of the center of nanoparticle, which obeys zero mean and its covariance is the Gauss error distribution of Qmap.

5 Stochastic Approach Based AFM …

114 Height (nm)

380

Height threshold

m1 Landmark localization uncertainty

Nanoparticles

y

m2

x

0 (a) Priori map

m3

1µm (b) Landmark selection

Fig. 5.6 Definition of landmark in the map

5.2.2

Analysis of Landmark Observation

When observing landmarks, two fast scan observations in different directions and perpendicular to each other can be carried out for nanoparticles, thus updating the distribution of tip position, as shown in Fig. 5.7a. Generally, when observing landmarks, the orientation of two local scans is the same as the horizontal and vertical directions of the image coordinate system, as shown in Fig. 5.7b, which can simplify setting of the motion model. Figure 5.7c is a landmark observation process based on local scan in the horizontal direction of tip localization, and zk+1 is the observation of the tip at time k + 1 after local scanning of nanoparticles. Usually, zk+1 is used to observe the landmark mj in the map by using local scan observation in the position of xk+1. In nano-environment, unlike the detection method of position sensor used by a macro-robot in a certain position, the tip needs to obtain the observed measurement zkp of the landmark position xkp from the current position xk to the next position xk+1 trajectory using local scan. The definition is as follows:

200 0

xk

z (nm)

Scan line

xkp

xk+1

Horizontal scan line

Nanoparticle

(a) Local scan path

Particle center

Max contour

(b) 3D contour of local scan

(c) Horizontal scan contour in local scan

Fig. 5.7 Tip position updating requires horizontal and vertical landmark observation

5.2 Analysis of Landmark Observation Model Based on Kalman Filter

zkp ¼ hðxk ; xk þ 1 ; mj Þ þ vz;kp

vz;kp  Nð0; Qz;kp Þ

115

ð5:7Þ

In the formula, h(*, *, *) is the observation function and vz, kp is the error random variable in the observation. It is assumed that the tip has only one observation in each local scan, that is, only one landmark is scanned, so a set of observations is obtained: ZT ¼ fz1 ; z2 ; . . .; zT g

ð5:8Þ

where T is the number of landmark observations.

5.2.3

Analysis of Horizontal Observation of Landmark

In this section, only the horizontal observation process is analyzed, the vertical observation process can be treated similarly. Figure 5.8 is a horizontal observation process, which is divided into two steps. (1) Firstly, the optimal position distribution of the tip in the center of the nanoparticle xkp is estimated, and then the position distribution of the tip in xk+1 is estimated according to the motion model.

Fig. 5.8 Analysis of the process of observing landmarks horizontally

116

5 Stochastic Approach Based AFM …

(2) Based on Kalman filter, the optimal position distribution of the tip in the center of the nanoparticle xkp is estimated. Figure 5.9 shows the flow chart of the landmark observation made by this method in order to reduce the uncertain distribution of the tip position in the horizontal direction. When observing the landmark, the tip moves to the left side of the nanoparticle at time k, it is assumed that the distribution of the tip at that _ position is xðkjkÞ, then the nanoparticle are scanned horizontally to the right. From _ the motion model, the position distribution of the tip with large error at x ðk þ 1jkÞ is obtained. Meanwhile, according to the observation information on the scan line, the position distribution of the tip moving horizontally to the center of nanoparticles (mj) is estimated. Then the tip is observed by the observation model. After obtaining the observation value hðxkp ; mj Þ, the tip is updated by Kalman filter method at _ position xkp, and the position distribution of the tip xðk þ 1jk þ 1Þ at time k + 1 is estimated. Fig. 5.9 Flow chart for updating tip position for observing landmarks horizontally

Start

The tip moves from xk to xk+1 to observe the nanoparticle

Calculate the relative position of the center of the nanoparticle xkp in the observation line

Calculate the position distribution of the tip at xkp using motion model

Calculate actual observation using the position of the nanoparticle in the map and the scan line

Optimal estimation of tip position distribution at xkp using Kalman filter

Estimate the position distribution of the tip at xk+1 using the distance uk of the tip from xkp to xk+1

End

5.2 Analysis of Landmark Observation Model Based on Kalman Filter

117

Compared with the macro mobile robot, the characteristics of the landmark observation strategy based on local scan are as followed: • In the course of landmark observation without position sensor, the tip needs to scan the landmark quickly in the course of motion, and estimate the position of the landmark according to the observation information on the trajectory, that is, the process of the tip motion is the basis of landmark observation. The macro robot uses position sensor to observe, and its landmark observation is independent of the motion process. • In the course of performing local scan, the method does not update the position of the tip directly at position xk+1, but firstly observes and updates the tip position at the landmark in the scanning trajectory, and then estimates the tip position from the position xkp to the positon of time k + 1 according to the motion model, so that the position distribution after the position uncertainty reduced is obtained.

5.2.4

Optimal Estimation of Tip Position Based on Kalman Filter

Figure 5.10 shows the estimation of tip position at position xkp during the local scan observation. The specific process is as follows: • State model:

xk;kp
x ðkpjkÞ ¼ xk þ lk1


þ wkp
xk;kp


Fig. 5.10 The observation line when scanning the landmark horizontally

y(nm)

_

200 150 100 50 0

wkp  Nð0; Rkp Þ

lka

lkp

ð5:9Þ

lkb Height threshold line

0

100

xk

200

700

300

lk1

800

lk2 Nanoparticle

900 1000 x(nm)

xk+1

5 Stochastic Approach Based AFM …

118

In this formula, lk1 represents the random variable of the scan line length of the tip from xk to xkp in Fig. 5.10, which is the mean of lk1. xk,kp is random vector of the tip from xk to xkp. jjxk;kp jj is the modulus of the random vector mean, and xk;k1 =jjxk;k1 jj represents the unit vectors of local scan directions in task space. wkp is a random variable of error perturbation. The perturbation is a linear superposition of the perturbation wk of the tip at xk and the perturbation wk1 of the tip from xk to xkp. In order to obtain the exact position of the nanoparticle center xkp, lk1 is calculated in the scanning coordinate system, which is calculated from the midpoint of lka and lkb at the intersection of the scan line and the height threshold line. Then xkp is mapped to the task space coordinate system according to the position of the starting point xk of the scan line in the task space coordinate system. Generally speaking, the shape of landmark nanoparticle is spherical or near spherical. The method of determining the center of nanoparticle by calculating the midpoint between lka and lkb is more stable than directly finding the highest point in the scan line as the center of landmark, because the latter lacks robust results in the case of errors near the highest point. lk1 is calculated as follows: 1 lk1 ¼ ðlka þ lkb Þ þ wk1 wk1  Nð0; Rk1 Þ 2 wkp ¼ wk þ wk1 wkp  Nð0; Rkp Þ 1 wk1 ¼ ðwka þ wkb þ rka þ rkb Þ 2

ð5:10Þ

In the formula, wka and wkb are perturbation errors of the tip moving from xk to lka and lkb, respectively. rka and rkb are perturbations introduced in calculating lka and lkb at two intersections, respectively. • Observation model Since the landmark observation at time k + 1 is to observe the tip position at xkp, the observation model is as follows: _

_

z ðkpÞ ¼ hðxðkpjkÞ; mj Þ þ vz;kp

vz;kp  Nð0; Qz;kp Þ

ð5:11Þ

In order to estimate it by Kalman filter, the observed values can be inferred from the state values: _

hðxðkpjkÞ; mj Þ ¼ xðkpjkÞ

ð5:12Þ

In the process of local scanning landmark, the actual observation of the tip at xkp is the updated value of the current position of the tip at position mj,xy of the landmark in the map:

5.2 Analysis of Landmark Observation Model Based on Kalman Filter

h0 ðxkp ; mj Þ ¼ RTh ðSx Rh mj;xy þ Sy Rh xkp Þ

119

ð5:13Þ

In the formula, Rq 2 R22 is the rotation matrix in the local scan direction, Sx and Sy are the selection matrices.  Sx ¼

1 0

 0 ; 0

 Sy ¼

0 0

0 1

 ð5:14Þ

From the description of the abovementioned observation model, it can be seen that a local scan observation can only provide the updating of the tip position in one direction, while this method provides the updating of the tip position in the lateral direction (including x and y directions) of task space coordinate system. Therefore, two fast scan observations in different directions and perpendicular to each other can be carried out for updating the lateral distribution of tip position in the landmark observation. Generally, we use a two-dimensional prior image to define the landmark position. When observing the landmark, the direction of two local scans is the same as the horizontal and vertical directions of the image coordinate system, which can simplify the representation of rotation matrix Rh in the observation function. The uncertainty in local scan landmark observation comes from three parts: the error vmap of landmark position mj, xy; the error vz_kp caused by different positions of fast scan line; and the error vz_h caused by the deviation of fast scan line direction. In this way, the errors of landmark observation can be obtained: vz;kp ¼ vmap þ vz

kp

þ vz

h

ð5:15Þ

• Optimal estimation based on Kalman filter At the position xkp, based on the tip motion model and observation model, Kalman filter is used to estimate the optimal position. It is assumed that the input control of the tip motion is u(k), the predicted value from the motion model at the position xkp is: _

_

xðkpjkÞ ¼ gðxðkjkÞ; uðkÞÞ

PðkpjkÞ ¼ rg PðkjkÞrgT þ RðkpÞ

ð5:16Þ

where P is the covariance of the tip at position x. If the landmark observation is not performed by local scan in tip motion, the predicted value of the tip position _ _ xðk þ 1jkÞ is considered as a posterior estimate xðk þ 1jk þ 1Þ. The observed values and residuals of the tip at position xkp are calculated as follows:

5 Stochastic Approach Based AFM …

120 _

_

z i ðkpÞ ¼ hðx ðkpjkÞ; mj Þ i ¼ 1; . . .; N h i _ vij ðkpÞ ¼ zj ðkpÞ  z i ðkpÞ h i _ ¼ zj ðkpÞ  hðx ðkpjkÞ; mj Þ

ð5:17Þ

h i Sij ðkpÞ ¼ E vij ðkpÞvTij ðkpÞ

ð5:18Þ

¼ rh PðkpjkÞ rhT þ Qj ðkpÞ

S is the covariance matrix after observing landmarks by local scan. By calculating the 2 Mahalanobis distance vTij ðkpÞS1 ij ðkpÞvij ðkpÞ  g , a threshold g is set to perform real-time landmark observation. The optimal estimations at position xkp are as follows: WðkpÞ ¼ PðkpjkÞ rhT S1 ij ðkpÞ _

ð5:19Þ

_

x ðkpjkpÞ ¼ xðkpjkÞ þ WðkpÞvðkpÞ

PðkpjkpÞ ¼ PðkpjkÞ  WðkpÞSðkpÞW T ðkpÞ After the optimal estimation at position xkp based on the Kalman filter, the calculation at xk+1 based on the motion model is: xk þ 1 ¼ xkp þ ukp

ð5:20Þ

Pðxk þ 1 Þ ¼ Pðxkp Þ þ Rk þ 1

ð5:21Þ

In a landmark observation including horizontal and vertical, the change of the tip motion state is shown in Fig. 5.11. A full observation of its position is completed from xk to xk+4, where m represents the landmark, xk represents the position of the tip at time k, uk represents the control quantity at time k, and zk represents the observation at time k.

uk xk+3 xk

xk+2

uk+1 xk+1

xk

uk+3

uk+2 xk+2

uk+4 xk+3

xk+4

xk+1 mj

xkp

Localization for horizontal scan

(a) Local scan path

x'kp zkp

xk+4

z'kp mj

Localization for vertical scan

(b) State transition diagram during the local scan process

Fig. 5.11 Tip localization diagram based on landmark observation

5.3 Establishment of Tip Motion Model

5.3

121

Establishment of Tip Motion Model

In AFM based nano-manipulation, the input voltage of PZT, which controls the tip motion, changes gradually after non-linear compensation, and the tip moves gradually according to a fixed step size. Every time after the tip moves a fixed step, the AFM system will sample the height information of the tip position. When the tip moves along the planned path, a contour scan line is obtained to monitor the spatial relationship between the tip and the landmark. Therefore, the tip motion model is the basis of the observation model. In the program, the “fixed step” of the tip motion is called the basic step, which represents not only the basic unit of the tip motion, but also the resolution of the observation. The tip motion model consists of three parts, PI hysteresis compensation model, creep model and thermal drift compensation model. Then the following three parts will be introduced.

5.3.1

PI Based Motion Model

As for the problem of non-linear hysteresis of PZT controller, PI model is widely used in PZT-driven feedforward controller because of its simple principle and easy solution of inverse model. PI model is superimposed by basic hysteresis operators. As shown in Fig. 5.12a, PI model is established by fitting the sampling points of PZT hysteresis loop, and then the compensation value of input control voltage is obtained by inverse model, it can be described by the following formula: yh ðtÞ ¼ H ½ xðtÞ ¼ wT Hr ½x; z0 ðtÞ

ð5:22Þ

Hr ½x; z0 ðtÞT ¼ ðHr0 ½x; z00 ðtÞ; . . .; Hrn ½x; z0n ðtÞÞ

ð5:23Þ

Inverse model formula: H 1 ½yðtÞ ¼ w0T Hr0 ½y; z00 ðtÞ

ð5:24Þ

The inverse model parameters are as follows: ri0 ¼

i X

wj ðri  rj Þ

ð5:25Þ

j¼0

w00 ¼

1 w0

w i  w0i ¼  Pi P w0 þ j¼1 wj w0 þ i1 j¼1 wj

ð5:26Þ ð5:27Þ

5 Stochastic Approach Based AFM …

122

0.08

20 10 Right boundary point

0 -10

PI operator -20

Creep trajectory (µm)

Output trajectory (µm)

30 Sampling point PI model

Left boundary Inverse model point

-30 -30

-20

0 10 -10 Input signal (V)

20

30

0.06

0.04 0.02

0

0

10

20 Time (s)

30

Fig. 5.12 PI motion model

w00i ¼

i X j¼0

wj z0i þ

n X

wj z0j

ð5:28Þ

j¼i þ 1

In the formula, i = 1, …, N, N is the cardinal number of the hysteresis operator.

5.3.2

Creep Model of PZT

The input control voltage of PZT varies step by step according to the sampling step. The displacement of PZT at a certain time is related not only to the current input control voltage, but also to the historical change of input control voltage. When the input control voltage remains unchanged at a certain value, the displacement of PZT will gradually increase for a period of time and then reach a stable value. Figure 5.12b is a curve showing the change process (Creep process). The curve has a larger change rate in the early stage and tends to a stable value quickly. At the later stage, the curve reaches a stable value smoothly. As for the early change process of PZT creep, a creep model is established to improve the accuracy of PZT driving tip positioning. The creep model is composed of the superposition result of the first order differential operator and the input voltage multiplied by a proportional coefficient. It is described as follows: 1 x_ i ðtÞ þ xi ðtÞ ¼ uðtÞ k

ð5:29Þ

5.3 Establishment of Tip Motion Model

xi ðkÞ ¼ e

ki T

123

xi ðk  1Þ þ ð1  e

yc ðkÞ ¼

N X

ki T

Þuðk  1Þ

ci xi ðkÞ þ auðkÞ

ð5:30Þ ð5:31Þ

i¼1

where xi is creep factor, yc is creep offset, u is input control voltage, ki , T, ci and a are creep time constant, sampling period, creep parameter and control parameter respectively.

5.3.3

System Thermal Drift Model

The offset caused by system thermal drift mainly depends on system thermal drift, the mechanical structure of the system and the thermal expansion coefficient of the components of the system [10, 11]. Thermal drift can cause spacing changes between nanoparticles P1 and P2 in continuous scan as shown in Fig. 5.13a, b. These continuous images are obtained by alternating Frame up and Frame down scan modes. The thermal drift speed can be estimated by the following strategy. The nanoparticles in Fig. 5.13 drift to the right and up simultaneously in a continuous scanning image. The tip scans the image in a left-to-right mode. In the process of scanning image by Frame up mode, the nanoparticle P2 is first imaged. Before imaging, P1 will be drifted away from P2 due to thermal drift, making the vertical distance du_y between them larger than the real distance. However, in the process of scanning image in Frame down scanning mode, the nanoparticle P1 is first imaged, and P2 will drift towards the direction near P2 before imaging, making the vertical distance dd_y between them smaller than the real distance. According to the time interval of nanoparticles under different scanning modes, the thermal drift velocities vdrift_x and vdrift_y in the x and y directions are calculated as follows: Tu0 ¼ Tu  Dt ¼ Td0 ¼ Td þ Dt ¼ vscan vscan vdrift

y

y y

dy y  vdrift

y

vscan

dy y þ vdrift

y

þ vdrift  vdrift

du

y

vdrift

x

¼

vscan

 dd

¼

dd

y y

¼

ð5:32Þ ð5:33Þ

Tu  Dt Td þ Dt

y þ 2
Dt
vscan Tu þ Td x  du Tu þ Td

x

ð5:34Þ y

ð5:35Þ ð5:36Þ

P1

P1

P2

1µm

t

P1 P2

1µm

Frame up

(a)t=15:43:30s

P1'

Tu '

Td'

Frame down

P2 (c) Calculate the temperature drift speed in the vertical direction

(b)t=15:46:30s

Vdrift_x

Vdrift_v

5 Stochastic Approach Based AFM …

124

Vdrift_x P1

P1 P1 '

P1 dd_x-du_x

Td Tu P2

P2

P2 '

(e)In Frame down mode, the (d) In the Frame up mode, the nanoparticles P1 moving to P1' nanoparticles P2 moving to P2' in in the image are horizontal the image are the horizontal drift temperature drift drift in the horizontal direction.

P2

P2'

(f) Calculate the temperature drift speed in the horizontal direction

Fig. 5.13 Estimation of system thermal drift based on AFM scanning image

In the formula, dy is the actual distance between nanoparticles P1 and P2 in the vertical direction, vscan_y is the scan speed of tip in vertical direction, vdrift_y is the thermal drift speed of sample in the vertical direction, and T′u is the scanning time interval between nanoparticles P1 and P2 in Frame up scanning mode. In Fig. 5.13c, the tip first scans to P2, then T′u is difference of the time Tu of finding assumed nanoparticle P′1 in the vertical direction and the exact time Dt of finding P′1 when scanning left to right horizontally. T′d is the scan time interval between the nanoparticles P1 and P2 in the Frame down mode, which is the sum of the time finding P′1 in horizontal direction Dt and the time finding P2 in vertical direction. Formula (5.34) can get from formula (5.32) dividing (5.33), then the formula (5.35) is obtained, it can be used to estimate the system thermal drift in the vertical direction. The similar method can be used to estimate the thermal drift in the horizontal direction, which is as shown in formula (5.36).

5.4 Simulation Experiment of Tip Localization Based on Landmark Observation

5.4

125

Simulation Experiment of Tip Localization Based on Landmark Observation

In order to illustrate the validity of the tip localization method based on landmark observation, we compare the simulation results of the tip localization method based on landmark observation with that based on direct motion. In the simulation, the planning steps of the tip motion path are as follows: first, the tip is moved to the initial point x0; then the tip is controlled to move from the starting point x0 to the target point x8 by using landmark observation method and direct motion mode. These abovementioned steps can be used to illustrate the validity of the proposed method by comparing the accuracy of tip localization. In this algorithm, the parameters of motion model and observation model are calibrated using the parameter calibration method described in Chap. 7. Figure 5.14 shows the motion control diagram of the tip from the starting point x0 to the target point x8. The initial distribution of the tip at the starting point x0 is assumed, without observing the landmark, the tip moves directly to x8 by xd_1 (the path is marked by dashed lines), and the position error of the tip at x8 will exceed the predetermined threshold. In order to improve the position accuracy of the tip, the nanoparticle near the target position are used as landmarks for tip position observation and estimation. The trajectory of the tip is marked by a solid line. The simulation results based on the two tip control modes are shown in Fig. 5.15, and the tip positions are shown in Tables 5.1 and 5.2. Table 5.1 is the simulation result of the tip moving directly from x0 to x8 without the landmark localization strategy; Table 5.2 is the simulation result of the tip moving from x0 to x8 under the landmark localization strategy.

Fig. 5.14 AFM tip motion planning path using landmark localization experiment

5 Stochastic Approach Based AFM …

126 1.5

1.5

x0

Tip localization based on direct motion

x0 1.0

0.5

y (µm)

0.5

y (µm)

Tip localization based on landmark observation

1.0

0

0

x5 -0.5

-0.5

x4

x2

x1

x3 x6,kp

-1.0

x6

-1.0

xd_1

x8

-1.5 -1.5

-1.0

-0.5

0

0.5

1.0

1.5

x (µm) (a) The tip moves directly from the initial position to the target position without the landmark localization strategy

x8

x7

-1.5 -1.5

-1.0

-0.5

0

0.5

1.0

Fig. 5.15 Simulation results diagram based on two tips control modes

Table 5.1 Tip motion simulation result based on direct motion (lm) x0 xd_1 x8

Table 5.2 Simulation result of tip localization based on SAFLP algorithm (lm) x0 x1 x2 x3,kp x3 x4 x5 x6,kp x6 x7 x8

1.5

x (µm) (b) The tip moves from the initial position to the target position with the landmark localization strategy

Simulation data ly lx

rx

ry

−1.245 −1.245 1.031

0.014 0.014 0.019

0.016 0.021 0.022

1.316 −1.381 −1.434

Simulation data ly lx

rx

ry

−1.245 −1.245 0.239 0.802 1.220 1.254 0.804 0.771 0.764 0.761 1.101

0.014 0.014 0.017 0.006 0.008 0.008 0.009 0.009 0.009 0.009 0.010

0.016 0.020 0.020 0.020 0.020 0.021 0.021 0.006 0.008 0.009 0.010

1.316 −0.645 −0.700 −0.701 −0.701 −0.407 −0.371 −0.682 −0.986 −1.431 −1.471

5.4 Simulation Experiment of Tip Localization Based on Landmark Observation

127

Table 5.1 shows that the tip moves directly from x0 to x8 over a long distance xd_1, and the variance of its position error increases to about 20 nm. Table 5.2 shows that by observing nanoparticle, the tip moves from x0 to x8, and the variance of its position error is reduced to about 10 nm. The calibration and experiment of the model parameters related to simulation will be described in Chap. 7.

References 1. Li, G.Y., Xi, N., Yu, M.M., et al.: Development of augmented reality system for AFM-based nanomanipulation. IEEE/ASME Trans. Mech. 9(2), 358–365 (2004) 2. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT Press, London, UK (2005) 3. Thrun, S.: Probabilistic algorithms in robotics. Am. Assoc. Artif. Intell. 21(4), 93–109 (2000) 4. Leonard, J.J., Durrant-Whyte, H.F.: Mobile robot localization by tracking grometric beacons. IEEE T. Robot. Atom. 7(3), 376–382 (1991) 5. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: A Probabilistic Approach for On-Line Positioning in Nano Manipulations, pp. 450–455. IEEE WCICA 8th, JiNan, China (2010) 6. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: AFM Tip On-Line Positioning by Using the Landmark, pp. 75–80. IEEE NMDC 10th, CA, USA (2010) 7. Yuan, S., Wang, Z.D., Liu, L.Q., et al.: Stochastic approach for feature-based localization and planning in nano-manipulations. IEEE Trans. Autom. Sci. Eng. 14(4), 1643–1654 (2017) 8. Yuan, S., Wang, Z.D., Xi, N., et al.: AFM Tip Position Control in situ for Effective Nanomanipulation. IEEE/ASME Trans. Mechatron. 23(6), 2825–2836 (2018) 9. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: Feature referenced tip localization enhanced by probability motion model for AFM based nanomanipulations. IEEE ROBIO 2011, Phuket, Thailand, pp. 1421–1426 (2011) 10. Liu, L.Q., Luo, Y.L., et al.: Sensor referenced real-time videolization of atomic force microscopy for nanomanipulations. IEEE/ASME Trans. Mechatron. 13(1), 76–85 (2008) 11. Krohs, F., Onal, C., et al.: Towards automated nanoassembly with the atomic force microscope: a versatile drift compensation procedure. J. Dyn. Syst. Measur. Control 131(3), 061106 (2009)

Chapter 6

Path Planning of Nano-Robot Using Probability Distribution Region

Abstract The landmark observation method using local scan can significantly improve the accuracy of tip localization. However, in order to reduce the time and distance of tip motion and achieve effective AFM nano-manipulation, it is necessary to carry out path planning. As for the abovementioned problems, this book refers to the strategy of macro-robot using landmark localization. Firstly, a tip localization model using landmark observation is established. Then, a path planning method for landmark observation based on probability distribution region is established. In order to solve the problem of one-scan and multiple-scan during local scan, according to the criterion of the shortest moving path and the fastest time of scanning the nanoparticle (landmark), the moving path of tip in a single landmark environment is planed based on the analysis of the single landmark localization accuracy of the tip in different initial positions and scanning modes. Meanwhile, in view of the complex environment of multiple landmarks in the task space where the tip is localized, the observation distance between adjacent landmarks is defined and the adjacent matrix is established to plan the tip path. Then, Dijkstra method and ant colony algorithm [1] are used to plan the shortest path between the landmarks. The simulation and experiment have illustrated the effectiveness of this method. Finally, aiming at the environment where there is no pre-set landmark near the task point or far away from the landmark, the concept of active configuration of landmark and its domain is proposed, and virtual nano-hand is used to realize active configuration of landmark. The method proposed in this book will be helpful to promote the practical application of AFM nano-manipulation in assembly of micro/nano-devices.








Keywords Path planning Landmark observation Probability distribution region Adjacent matrix Dijkstra method Ant colony algorithm Virtual nano-hand










Authors: Shuai Yuan, Lianqing Liu, Zhidong Wang, Ning Xi. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_6

129

6 Path Planning of Nano-Robot Using …

130

6.1

Path Planning for Landmark Observation Using Probability Distribution Region

As shown in Fig. 6.1, if the distribution region of the tip is large at the position xk near the nanoparticle, it may need multiple scans (L0 ! L1 ! L2) to find the nanoparticle when observing it. In order to ensure that the nanoparticle can detect all the region of uncertain distribution of tip position, this algorithm plans the path according to the principle of the shortest distance of tip motion. Figure 6.1 shows the process of path planning for landmark observation based on stochastic approach, and Fig. 6.2 is the flow chart of this process. Before the analysis, it is assumed that the motion uncertainty of the tip in horizontal and vertical directions satisfy the Gauss distribution. The initial position distribution of the tip is represented by the elliptical blue dot distribution area centered on xk in Fig. 6.1a. According to the 3r principle of Gauss distribution, the horizontal radius xk (a) Nano particle

xk 3δ x_k 3δ y_k

(b)

L2

xk+1

uk lk_1

xmj

l'1 l0

xtop_mj

L0

L0

l1 D1

RP

xmj mj

Area3

(d)

xk+3

xk+2 xtop_mj

xtop_k+3

xk+1 A

RP

3Δσy_2 l1 l2

xbottom_mj xk+2 uy_k+1

L1

(e)

(f)

xk+4

xk+4

xk+5

L2

RP

uy_k+3

3Δσy_1

Area2

L1

(c)

xk+3

Area1

lk_2

RP

xBottom_mj

Fig. 6.1 Path planning for landmark observation using stochastic approach

6.1 Path Planning for Landmark Observation …

131

Fig. 6.2 Flow chart of local scanning landmark

and the vertical radius of the ellipse are respectively: 3rx_k and 3ry_k. rx_k and ry_k are the variances of the tip position in horizontal and vertical directions respectively. The relationship between them is as follows. When the tip moves in the x-direction, the horizontal variance rx_k increases linearly with the increase of the motion distance, and the slope is Kh1. At the same time, the vertical variance ry_k increases slightly linearly, and the slope is Kh2. When the tip moves in the y-direction, the vertical variance ry_k increases linearly with the increase of the motion distance, and the slope is Kv1. At the same time, the horizontal variance rx_k increases slightly linearly, and the slope is Kv2. To make the calculation concise and clear, the operator representation is as follows: The controlled variable ux is a vector, which contains the horizontal component ux_k and the vertical component uy_k. xk is a position random vector, which contains the horizontal component xx_k and the vertical component xy_k. xmj is the central position vector of nanoparticle mj, which includes xx_mj and xy_mj in horizontal and vertical components respectively.

6 Path Planning of Nano-Robot Using …

132

After the tip scans the nanoparticle at the first time, it is divided into several probability distribution regions, and then the tip position in the horizontal direction is estimated by scanning the nanoparticle between left and right motion with moving up or down [2, 3]. In this book, the tip motion in the whole course of landmark observation is divided into five steps which are as followed: Step 1: Plan the control input uk when the tip moves along L0 from xk to xk+1 to determine whether the tip scans the nanoparticle. As shown in Fig. 6.1b, after the tip scans the nanoparticle, the position distribution area is divided into three probability distribution areas: Area1, Area 2, and Area 3. ① If the Tip Is in the Area 2, The Nano-Particle is Scanned In this case, the length of L0 should be shortened as much as possible according to the principle of the shortest moving distance of the tip. Meanwhile, in order to avoid the tip “contacting” and moving the nanoparticle when scanning to the position xk+1 in tapping mode, the distance between the distribution center of the tip at position xk+1 and the edge of the nanoparticle is set to be 3rx_k+1 + RT, rx_k+1 is the Gauss distribution covariance root of the tip at position xk+1, and RT is the radius of the tip. According to the above requirements, the calculation of control input uk is as follows: The vertical component uy_k of uk is 0, which means that the tip displacement in the vertical direction is 0. The horizontal component consists of two parts: lk_1 and lk_2, in which lk_1 is the displacement of the tip from xk to xmj in the center of the nanoparticle, and lk_2 is the displacement of the tip from xmj in the center of the nanoparticle to xk+1. lk_1 can be obtained by the following equation: lk

1

¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðxx k  xx mj Þ2

ð6:1Þ

The equation for calculating lk_2 is: 3

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kh1 ðlk 1 þ lk 2 Þ2 þ r2 þ RT þ RP ¼ lk

2

ð6:2Þ

After simplifying the above equation, the univariate quadratic equation about lk_2 is obtained. The root of the equation needs to satisfy lk_2  3rx + RT + RP. If there are two roots, the smaller value is taken as the solution. At this time, the upper boundary of the tip distribution area (Area 2) is determined as follows. It is assumed that the tip passes through the upper boundary xtop_mj of the nanoparticle, when it reaches position xk+1, the upper boundary of the uncertainty distribution area of the tip position expands from l0 to l′1(dotted line marking), and its height position (vertical distance from the upper boundary of the nanoparticle) is pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi xtop mj þ 3 Kh2 lk 2 . The vertical distance D′1 between l′1 and the distribution center of xk+1 is as follows:

6.1 Path Planning for Landmark Observation …

D01 ¼ ðxy

mj

 xy

133

k þ 1 Þ þ RP

þ3

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kh2 lk 2

ð6:3Þ

② In this scan, if no nano-particle was scanned, the tip is localized in Area 1 or Area 3. At this point, the tip is moved vertically down to the second step of planning. Step 2: Plan the control input uk+1 when the tip moves downward from xk+1 to xk+2 along the vertical direction to prepare for the second horizontal scan. Before the tip moves down, as shown in Fig. 6.1b, the height of the lower boundary l1 (dotted line marking) at the bottom of the Area1 is pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi xtop mj  3 Kh2 lk 2 . The vertical distance D1 between l1 and the distribution center of xk+1 is as follows: D1 ¼ ðxy

mj

 xy

k þ 1 Þ þ RP

3

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kh2 lk 2

ð6:4Þ

The calculation of the upper boundary of Area 3 is similar to the abovementioned process, which is not described here. In the control input when the tip moves from xk+1 to xk+2, ux_k+1 is 0, and uy_k+1 is the downward displacement of the tip. Figure 6.1c shows that the bottom boundary l1 of Area 1 will expand to boundary l2 after the tip moved down to xk+2, and the vertical distance between l2 and xbottom_mj (the bottom of nanoparticle) will become as follows: D1 þ RP  uy

kþ1

3

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kv1 uy k þ 1

ð6:5Þ

In order to ensure that the tip located above the boundary l2 can scan the nanoparticles along L1, the intersection point of l2 and the tip at the right boundary of the elliptical distribution area centered on xk+2 is assumed to be A and the point A scans to the right side of the nanoparticle, the uncertainty region of the tip distribution is above the bottom xbottom_mj, which satisfies the following formula: pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi D1 þ RP  uy k þ 1  3 Kv1 uy k þ 1 [

rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kh2 ð6

r2x

kþ1

þ Kv2 uy

kþ1

þ RP þ RT Þ ð6:6Þ

According to the upper formula (6.6), the upper bound of uy_k+1 can be calculated. In order to simplify the calculation, the Kv2uy_k+1 in the formula can be estimated first (when the tip moves down, the variance in the horizontal direction increases). The estimation method is that the tip downward displacement uy_k+1 is assumed to exceed its upper bound, so that l2 is aligned with the bottom xbottom_mj of the nanoparticles on the same line.

6 Path Planning of Nano-Robot Using …

134

xy

kþ1

 xbottom

mj

þ D1  3

pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Kv1 uy k þ 1  uy

kþ1

¼0

ð6:7Þ

After the simplification of the above formula, a quadratic equation of one variable is obtained. The root of the equation u0y k þ 1 should satisfy: 0  u0y

kþ1

 D 1 þ xy

kþ1

 xbottom

mj

ð6:8Þ

After u0y k þ 1 is obtained, the obtained Kv2 u0y k þ 1 is taken as the Kv2 uy k þ 1 in the upper bound substitution (6.6). The right side of the Eq. (6.6) is a constant. Based on this, the upper bound of uy_k+1 is further estimated, and the upper bound is used as the control of the tip downward movement. Step 3: Plan the tip control input uk+2 from xk+2 to xk+3 along L1 to determine whether the tip is in Area1. The planning method in this process is like the first step. ① In this scan, if the nanoparticle is scanned, the tip position could be determined in Area 1. As shown in Fig. 6.1d, after the tip is moved to the position xk+3, the upper bound xtop_k+3 of the tip in the position distribution region of xk+3 (blue dotted line) and the upper bound xtop_mj of nanoparticles are compared. If xtop_k+3 < xtop_mj, nanoparticles can detect whether the tip is in the distribution region Area 1 of xk+2. Conversely, the tip continues to move down and process as the second step, continuing to detect the remaining areas in Area 1, which are not described here. ② If the tip moves to xk+3 without scanning the nanoparticle, and the nanoparticle has detected all the areas of Area 1, then it is determined that the tip is in Area 3 and needs to be moved upward for planning the next step. Step 4: Planning the tip to move upward from xk+3 to xk+4 along the vertical direction to control the input amount of uk+3. As shown in Fig. 6.1e, the planning method is the same as the second step. This process prepares for the third scan. Step 5: Planning the input amount of the tip along L2 from xk+4 to xk+5 is uk+4 to determine whether the tip scans to nanoparticle. As shown in Fig. 6.1f, the planning method is like the third step. In this process, if the nanoparticle can detect the whole area of Area 3 where the tip is located at xk+5, the path planning will be completed. Conversely, the tip will continue to move up and repeat the above process. After the tip observes nanoparticle horizontally, it is necessary to observe nanoparticle vertically. The specific process is to move the tip up/down RT + 2RP, then up/down to the center of nanoparticle, and move down/up to scan the nanoparticle, so as to improve the localization accuracy in the vertical direction. During the whole planning process, the tip moves from the initial point in task space to the vicinity of nanoparticle, and then observes the nanoparticle by local scan to improve the position accuracy of the tip. However, the larger is the uncertainty area of the tip near the nanoparticle, the more is the probabilistic times is to scan the nanoparticle. The “total cost” of finding nanoparticle is calculated by using several times of tip scanning.

6.1 Path Planning for Landmark Observation …

135

When the tip locally scans the nanoparticle mj, it may need three times of scans along L0, L1 and L2 to find the nanoparticle. The probabilistic value of the tip in Area 1, Area 2 and Area 3 can be calculated by integrating the probabilistic density function (Gauss function) to obtain Pj_0, Pj_1 and Pj_2 respectively. Since these regions are overlapping with each other to form the entire elliptical region of the tip position distribution, the sum of the probability values will be greater than 1, so normalization is required: Pj

i

¼ Pj i =

n X

Pj

ð6:9Þ

k

k¼0

This formula indicates that the probability of scanning nanoparticle along L0 is Pj_0 surrounded by Area 2 when the tip locally scans mj; similarly, the probability of scanning nanoparticle along L1 is Pj_1; and the probability of scanning nanoparticles along L2 is Pj_2. When the tip scans the nanoparticle several times, TH is set as the probability threshold of the k-th scan. This book takes TH = 0.01. If the tip scans along Lk to the probability value Pj_k (k = 0, … n) < TH, it is considered that the scan is a small probability event, which cannot occur and does not include the total number of scans. The relationship between the total cost of local scan and the number of scans and the desired moving distance of the tip is obtained as follows: Lp

j

¼ sc  ðPj

0

 Lj

0

þ

n1 X

Pj

k

 ðLj

k

þ uy

k þ 1 ÞÞ

ð6:10Þ

k¼1

where, sc is the scan times, k is the number of local scan, Lj_k is the shortest distance from the tip to the nanoparticle mj along Lk, and uy_k+1 is the distance between the two adjacent scans of Lj_k and Lj_k+1. Abovementioned work analyzes the larger distribution region at position xk near the nanoparticle. When the uncertainty region near the nanoparticle is small, the tip can find the nanoparticle by scanning once. As shown in Fig. 6.3, the vertical range of the uncertainty region of the tip position is 6ry_k/Dnp L0 (Dnp is the diameter of nanoparticle) as the “cost” of local scan, which is the observation distance. Fig. 6.3 Estimation of observation distance with small region of the tip distribution

0.1

6σ y _ k + 1

Tip distribution

y (μm)

0.05 0

xk Nanoparticle

-0.05 -0.1

3σ y _ k Nanoparticle diameter: Dp -0.2

-0.1

0 x (μm)

0.1

0.2

6 Path Planning of Nano-Robot Using …

136

6.2

Tip Path Planning in Task Space

Based on the path planning of the abovementioned landmark observation, the planning method of the tip motion path in task space is studied. There are simple environment of single landmark and complex environment of multiple landmarks in task space where the tip is localized [4–6]. In order to meet the task requirement of high accuracy of task position, short moving path and fast scanning, it is necessary to plan the moving path of the tip.

6.2.1

Basic Path Planning in Single Landmark Environment

When the tip scans the landmark, it first moves from the initial position to the nearest nanoparticle (landmark), then scans the landmark horizontally and vertically, as shown in Fig. 6.4a–d, and finally localize itself at the four optional positions near the nanoparticle, as shown in Fig. 6.4e. In the single landmark environment, in order to achieve the precise localization of the tip around the target position, it is necessary to plan the path. The tip is usually required to move along an optimal or suboptimal route in task space according to certain criteria, such as the shortest total length of the moving route or the highest position accuracy. Before path planning, the accuracy of tip localization is analyzed under different initial positions and different scanning modes of landmarks. Local scan Local scan

Nanoparticle LX_P

Nanoparticle (a) The first local scan path (b) The second local scan path

Local scan

Nanoparticle

Nanoparticle

Local scan

(c) The third local scan path (d) The fourth lcoal scan path Fig. 6.4 Localize the tip near the nanoparticle

(e) Localization around the nanoparticle

6.2 Tip Path Planning in Task Space

137

(1) Analysis of Tip Localization Accuracy at the Different Tip Initial Positions Aiming at the different initial tip positions relative to the same nanoparticle, the position accuracy of the tip after scanning the landmark observation is analyzed by simulation results. In Fig. 6.5, the red solid line represents the local scan path of the 0.4

Initial position 0.3

y (μm)

0.2

P4

0.1

P1

P3 P2

0

P5

-0.1 -0.4

-0.2 x (μm)

0

0.2

(a) Tip localization with a smaller initial position distribution 0.4

Initial position

y (μm)

0.3 0.2

P4

P3

0.1

P1

P2

0

P5

-0.1

-0.4

-0.2

x (μm)

0

0.2

(b) Tip localization with a large initial position distribution 0.6

Initial position

y (μm)

0.5

0.4 0.3 0.2

P4

0.1

P1

P2

0

-0.1

P3

P5 -0.6

-0.4

-0.2 x (μm)

0

0.2

(c) Tip localization with a long distance from the initial position Fig. 6.5 Tip localization result under different initial conditions

138

6 Path Planning of Nano-Robot Using …

Table 6.1 Variance of tip distribution at each position Standard deviation of tip distribution at each position

Figure 6.5a

Figure 6.5b

Figure 6.5c

Initial standard deviation (nm) P1 standard deviation (nm) P2 standard deviation (nm) P3 standard deviation (nm) P4 standard deviation (nm) P5 standard deviation (nm)

(5.0, 5.0) (14.0, 11.0) (3.5, 11.0) (3.5, 12.7) (4.4, 12.7) (4.4, 4.8)

(10.0, 10.0) (14.0, 11.0) (4.4, 11.0) (4.4, 12.7) (4.5, 12.7) (4.5, 4.8)

(5.0, 5.0) (17.8, 13.7) (3.5, 13.7) (3.5, 15.1) (4.5, 15.1) (4.5, 4.8)

tip, the red dotted circle represents the distribution range of the tip at the current position, and the blue dotted circle represents the position distribution range of the tip at last position. By comparison, it can be seen that there is the variation of the uncertainty distribution range of the tip position. Figure 6.5a shows the simulation result of the position estimation of the tip moving from the initial position to the left side of the nanoparticle, then partially scan the nanoparticles, and finally performing the position estimation around the nanoparticle. Figure 6.5b shows the simulation results when the initial position distribution range of the tip increases. Figure 6.5c shows the results of position estimation by using local scan when the initial position of the tip is further away from the nanoparticle. Table 6.1 records the standard deviation e of the tip position distribution in the x and y directions when the tip is at six positions of initial position P1, P2, P3, P4 and P5 under the abovementioned three conditions. It can be seen from the table that the value of the standard deviation e of the tip position distribution at the target position P5 is close to 5 nm in all three cases. It can be concluded that the accuracy of tip localization is consistent at different initial positions, i.e. they are kept within a certain range. (2) Analysis of Tip Localization Accuracy in Different Scanning Modes Based on the analysis of the localization accuracy of different initial tip positions, the tip localization accuracy in four optional positions around the landmarks is further analyzed. In the simulation experiment, it is assumed that the initial position distribution condition of the tip is the same, then the position distribution variance of the tip under four different scanning modes is calculated, as shown in Fig. 6.6. In Fig. 6.6, the red dotted coil represents the distribution range of the tip at the current position, the blue dotted coil represents the distribution range of the tip at last position, and the red solid line represents the local scan path of the tip. Figure 6.6a–d are the simulation results of the position distribution of the tip under four different scanning modes. Under these four scanning modes, the tip is finally localized at the left, right, upper and lower positions of the nanoparticle. Table 6.2 shows the values of the standard deviation r of the tip in x and y directions at six positions of initial position, P1, P2, P3, P4 and P5.

6.2 Tip Path Planning in Task Space

0.4

139

0.4

Initial position

0.3

P4 P3 P1

P2

0

y (μm)

y (μm)

0.2

0.1

P5 -0.4 -0.3 -0.2 -0.1 0 x (μm)

0.1 0.2

P1

P2 P5 -0.4 -0.3 -0.2 -0.1 x (μm)

0

0.1

(b) Localizing the tip using local scan for the second position 0.4

Initial position

Initial position

0.3

0.3

P1

0.2 0.1

y (μm)

y (μm)

P3 P4

0.1

- 0.1

(a) Localizing the tip using local scan for the first position

P5

P4

0

-0.1

0.2

0

-0.1

0.4

Initial position

0.3

P3 -0.4 -0.3 -0.2 -0.1 0 x (μm)

0.2

P3

0.1

P1

-0.1 -0.4 -0.3 -0.2 -0.1 0 x (μm)

0.1 0.2

(c) Localizing the tip using local scan for the third position

P5

P4

0

P2

P2

0.1

(d) Localizing the tip using local scan for the fourth position

Fig. 6.6 Tip localization process at four optional locations around the landmark

From the data in Table 6.2, it can be seen that the change of tip local scan path mode has little effect on the tip localization accuracy at the target position P5, that is, the distribution range of the tip position can maintain consistency. From the above analysis, it can be seen that the localization accuracy of the tip is consistent in the initial position and different scanning modes of landmarks. Table 6.2 Error distribution of four optional positions of the tip around the nanoparticle Standard deviation of tip distribution at each position

Figure 6.6a

Figure 6.6b

Figure 6.6c

Figure 6.6d

Initial standard deviation (nm) P1 standard deviation (nm) P2 standard deviation (nm) P3 standard deviation (nm) P4 standard deviation (nm) P5 standard deviation (nm)

(5.0, 5.0) (14.0,11.0) (13.5,11.0) (13.5,12.7) (4.4, 12.7) (4.4, 4.8)

(5.0, (7.6, (4.3, (4.3, (5.0, (5.0,

(5.0, 5.0) (14.0, 14.2) (14.0, 4.8) (6.5, 4.8) (6.5, 4.2) (4.9, 4.2)

(5.0, (7.6, (7.6, (8.1, (8.1, (4.4,

5.0) 5.9) 5.9) 6.2) 6.2) 4.0)

5.0) 6.5) 4.2) 8.1) 4.5) 4.5)

140 Fig. 6.7 Tip path planning in single landmark environment

6 Path Planning of Nano-Robot Using … Initial position Selected path Other path

Manipulation point position RP

PL1

PL3

Localization position

PL4

PNP PL2 PNP: Particle position

Therefore, as for the single landmark environment, in order to ensure that the localization accuracy of the tip at the manipulation position is within the allowable range, and at the same time to enable the tip to move to the manipulation point quickly, the following path planning guideline are proposed: Guideline 6.1 The localization of the tip around the observation landmark is nearest to the manipulation point PTask. Guideline 6.2 The tip moving path is the shortest. According to these two criteria, Fig. 6.7 shows a diagram of tip path planning in single landmark environment. Before path planning, initial position, particle position in task space and task position of tip are known. In order to ensure that the accuracy of the tip at the task position meets the requirements (Guideline 6.1), the nearest localization PL1 is selected as the first planning point. It is assumed that the tip can find the nanoparticle through a single scanning landmark, so PL2 is set as the second planning point according to the local scan direction. As for the remaining two points PL3 and PL4, the PL3 closest to the initial position of the tip is selected as the third planning point, and then according to the local scan direction, the fourth planning point PL4, so the path of tip motion (black thick line marking) is planned to be initial position ! PL3 ! PL4 ! PL2 ! PL1 ! task position, This path is the shortest moving distance of the tip compared to other paths (such as the path marked by the red dotted line), which ensures that not only can the tip reaches the target position with the fastest time, but also the smallest error distribution at the target position. If the nanoparticle need to be scanned many times after the first point PL1 is planned, then the path planning method for landmark observation is described in Sect. 6.1. It is not mentioned here, but the PL3 point nearest to the initial position of the tip is still selected as the last planning point for the tip path planning.

6.2 Tip Path Planning in Task Space

6.2.2

141

Path Planning in Multi-landmark Environment

On the basis of tip path planning in single landmark environment, path planning in multi-landmark environment is carried out. Firstly, according to Guideline 6.1, the nearest landmark is selected to observe and localize the tip, and then the path of the tip is planned according to Guideline 6.2 to ensure the shortest path. However, in the complex landmark environment, when the initial position of the tip is far from the target point in task space and there are multiple landmarks in the space, if the tip moves directly to the nearest landmark for observation and localization, the uncertainty distribution area of the tip position will become larger due to the long moving distance, the nanoparticle can be found only after several scanning observations, which results in a longer scanning time and lower manipulation efficiency. In this case, if the tip scans the landmark close to the initial position, then scans one or more landmarks nearby, and finally finds the landmark nearest to the target point for localizing the tip accurately. In this moving process, because of the short observation distance of the tip and the small uncertainty distribution of the tip position in each step, the nanoparticle can be found only by scanning once time when local scanning of the nanoparticle is carried out, which greatly reduces the scanning time. On this foundation, the optimal tip moving path can be planned by minimizing the moving path. From the abovementioned analysis, it can be concluded that when the tip is localized in a complex landmark environment, it is not the best path to move directly to the nearest nanoparticle from the task point, and other nanoparticles may need to be found before the optimal path can be obtained. Therefore, according to the criterion “the moving distance in the process of the tip observing landmarks is shortest”, the observation distance between two landmarks in task space is defined in this book, and the adjacent matrix of multiple landmarks with the observation distance is established as the weight, and the shortest path algorithm is used to select landmarks and plan the moving path. As shown in Fig. 6.8, on the basis of formula (6.11), the minimum observation distance Lp_i, of the tip from nanoparticle mi to mj is defined as:

Fig. 6.8 Minimum observation distance between landmarks

Lx_P

PL1

Lx_P: interval distance between the tip and the nanoparticle center after localizaiton.

Nanoparticle PL4

PL2

mi PL3

d

mj

6 Path Planning of Nano-Robot Using …

142

Lp

i;j

¼ scantimes  ðPj

0

 ðLj

0

þ Dist=Dr Þ þ

n1 X

Pj

k

 ðLj

k

þ uy

k þ 1 ÞÞ

k¼1

ð6:11Þ where, Dist is the distance from the localization point PL3 to the left of the nanoparticle mj, Dr is the spatial range of the nanoparticle in the map (the edge length of the map). Formula (6.11) represents the cost of moving the tip to mj for observation after the localization near mi. After the observation distance between the two landmarks is calculated, the adjacent matrix is defined by the following method: Definition 6.1 A set of landmarks M¼½m1 ; m2 ; . . .; mn , where m1 ¼ ðx1 ; y1 Þ,i ¼ 1; 2; . . .; n, xi, yi denote the position of the landmark mi in task space. Definition 6.2 Weight adjacent matrix W between landmarks: 2

0 6 w21 6 6 w31 W ¼6 6 w41 6 4


wn1

w12 0 w32 w42


wn2

w13 w23 0 w43


wn3

w14 w24 w34 0


wn4





















3 w1n w2n 7 7 w3n 7 7 w4n 7 7


5 0

ð6:12Þ

The wij in the matrix represents the weight of the landmark mi to mj, when i = j, set wij = 0. In this book, the weight wij is set as the observation distance from the landmark mi to mj, that is, the minimum observation distance (cost) when the tip is localized near a landmark and performs local scan around other landmark. According to the definition and formula of observation distance (6.11), it can be concluded that the smaller is the observation distance, the higher is the accuracy of tip localization. After the adjacent matrix of the landmark is established, the problem of tip path planning in complex multi-landmark environment is discussed in two situations in this book: one is the manipulation of a specific nanoparticle in task space using a tip, the other is manipulation of multiple nanoparticles in task space. In order to ensure the shortest path and the fastest scanning nanoparticle in the whole manipulation process, in the first case, it need to calculate the observation distance between nanoparticles in task space, so as to establish the adjacent matrix. Then, according to the adjacent matrix, the Dijkstra method is used to select a path with the lowest cost, and plan as trajectory for manipulation. In the second case, when the tip needs to manipulate multiple nanoparticles, the different operation sequence will make the tip planning path different. The inappropriate manipulation sequence will lead to the longer distance of the tip in the whole operation process, the larger uncertainty distribution of the tip

6.2 Tip Path Planning in Task Space

143

position, and ultimately result in long manipulation time and low accuracy. In this book, according to the established adjacent matrix, ant colony algorithm is used to determine the optimal manipulation sequence of multiple nanoparticles, and tip trajectory is planned according to this sequence.

6.3 6.3.1

Simulation and Experimental Verification Path Planning Based on Dijkstra Method

As for the need to manipulate a specific nanoparticle in task space, an adjacent matrix is firstly established in this book, and then Dijkstra method is used to plan an optimal path with the shortest and fastest scanning path. As shown in Fig. 6.9, it is assumed that after the tip is localized around the nanoparticle P1, it needs to perform local scan around a landmark among P2, P3, P4 and P5, and then the tip is localized near the nanoparticle P6 for manipulation. In order to ensure the shortest observation distance when the tip moves to P6, it is necessary to calculate the observation distance between each two nanoparticles, then the adjacent matrix is established, and finally the Dijkstra method is used to find the shortest path. In this example, the adjacent matrix is shown in Table 6.3. According to the adjacent matrix, the Dijkstra method is used to get the minimum cost when the tip moves from P1 to P6 after local scan observation of P3. As shown in Fig. 6.9, P1 ! P3 ! P6 is the optimal path for manipulation. According to the adjacent matrix established in Table 6.3, and Table 6.4 lists the observation distances of the tip from P1 to P6 by P2, P3, P4 and P5 respectively. The Fig. 6.9 Path planning using Dijkstra algorithm

40

P1

35

P2 30

P4 P3

y (μm)

25 20

P5

15

P6

10 5 0

0

5

10

15

20

x (μm)

25

30

35

40

6 Path Planning of Nano-Robot Using …

144 Table 6.3 Landmark adjacent matrix (Unit: lm)

P1 P2 P3 P4 P5 P6

P1

P2

P3

P4

P5

P6

0.0000 0.7697 1.0054 0.9678 3.1106 3.1856

0.7697 0.0000 0.9222 0.7295 1.0197 3.1108

1.0054 0.9222 0.0000 0.6956 0.9858 1.0095

0.9678 0.7295 0.6956 0.0000 1.0065 3.0627

3.1106 1.0197 0.9858 1.0065 0.0000 0.7792

3.1856 3.1108 1.0095 3.0627 0.7792 0.0000

Table 6.4 Observation distances under different observation paths (unit: lm) Moving path

P1 ! P2 ! P6

P1 ! P3 ! P6

P1 ! P4 ! P6

P1 ! P5 ! P6

Sum of observation distance

3.8805

2.0149

4.0305

3.8898

comparison shows that the observation distances of the tip moving through path P1 ! P3 ! P6 are the smallest, which verifies the effectiveness of the path planning algorithm based on Dijkstra method.

6.3.2

Path Planning Based on Ant Colony Algorithm

On the basis of the abovementioned path planning, the path planning method for constructing nanostructures by manipulating multiple nanoparticles is further studied. According to the established adjacent matrix, ant colony algorithm is used to determine the optimal manipulation sequence of multiple nanoparticles in this book, the specific maneuvering steps are as follows: (1) Set the structure of nanoparticles and the target position of each nanoparticle; (2) The position of nanoparticles and its target position are regarded as a node, and the corresponding node information of all nanoparticles is recorded; (3) The nanoparticles in the two nodes were selected for observation and the observation distance is calculated; (4) Establish the adjacent matrix between the nodes; (5) The ant colony algorithm is used to plan the path of the tip moving among these nodes, that is, to optimize the path. Figure 6.10 shows an example of nano-structure construction. Sixteen nanoparticles (shown in yellow dots) are distributed in task space. The tip is needed to push 15 of them into a regular pentagon, and then push the last nanoparticle to the designated position. The pushing direction of each particle in these 16 nanoparticles is shown by the white arrow in Fig. 6.10a, which ultimately requires the result shown by the blue dots in Fig. 6.10a.

6.3 Simulation and Experimental Verification

145

4.0 Nanoparticles

Target position

2.0 P5 y(μm)

1.0

0 -1.0 -2.0

P3

P4

3.0

P9

P2 P6

P8 P10

P12 P15

P7

P13

P11 P14

-3.0 2μm

(a) Structure design for pushing nanoparticles

P1 P16 -5.0 -5.0 -4.0-3.0-2.0-1.0 0 1.0 2.0 3.0 4.0 5.0 x (μm)

-4.0

(b) Target position planning for pushing nanoparticles

Fig. 6.10 Position distribution and target position of nanoparticles in task space

In the process of nano-manipulation, the initial position of each nanoparticle and their corresponding target position are determined, which forms a node. The number is shown in Fig. 6.10b. The tip needs to push a nanoparticle to the target position in a certain sequence, then move to the next nanoparticle for local scan to complete the tip localization, then continue to perform the push operation, and so repeat until the final operation is completed. In this book, the observation distance is defined between the current target position and the next node (nanoparticles that may be pushed), and the adjacent matrix is established as shown in Table 6.5. On the premise of ensuring the accuracy of tip localization, the ant colony algorithm is proposed to plan the sequence of tip manipulation and observation path. In Fig. 6.11, the blue circle represents the initial position of each nanoparticle and the black circle represents the target position. Figure 6.11a shows the optimal planning path found by ant colony optimization algorithm based on the established adjacent matrix, i.e. the order of tip manipulation for nanoparticles is 15, 12, 13, 14, 16, 1, 11, 10, 2, 7, 8, 3, 4, 6, 5 and 9. The tip push these nanoparticles to the target position and construct nanostructure in turn. In order to illustrate the effectiveness of the algorithm, two manipulation paths are selected randomly and compared with the path planned based on the algorithm. Figure 6.11b is a randomly selected path A whose manipulation sequence is 1, 2, 3, 5, 6, 4, 7, 9, 8, 10, 11, 12, 14, 13, 15, 16. Figure 6.11c is a randomly selected path B. Its manipulation sequence is 13, 16, 14, 12, 15, 6, 5, 9, 8, 4, 3, 7, 2, 10, 11, 1. The comparison results are shown in Fig. 6.11 and Table 6.6. Table 6.6 lists the observation distances under these three paths. From Table 6.6, it can be seen that the “cost” for manipulation of the path planned by ant colony algorithm is 2.499 lm, while the “cost” for manipulation of the path planned by random path A and B is 3.6094 um and 2.7704 lm, respectively. So the path planning method proposed by this algorithm minimizes the “cost” of tip manipulation. In addition, the final experimental results based on ant colony algorithm are shown in Fig. 6.12. It can be seen from the Fig. 6.12 that the precise nanostructure construction can be achieved by this algorithm.

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16

0.0000 0.4024 0.6482 0.6801 0.5847 0.6563 0.5082 0.5965 0.5006 0.3049 0.2360 0.3139 0.2458 0.2124 0.4013 0.1947

P1

0.4024 0.0000 0.2571 0.2911 0.3012 0.2703 0.1388 0.2079 0.4015 0.1900 0.2714 0.4933 0.4779 0.4331 0.4661 0.3704

P2

0.6482 0.2571 0.0000 0.1659 0.3221 0.2316 0.2184 0.1485 0.4194 0.4000 0.4743 0.5517 0.5679 0.5545 0.4987 0.5444

P3

0.6801 0.2911 0.1659 0.0000 0.2498 0.1645 0.2431 0.1695 0.3442 0.4201 0.4861 0.4847 0.5283 0.5452 0.4263 0.5447

P4

0.5847 0.3012 0.3221 0.2498 0.0000 0.1214 0.2418 0.2564 0.1939 0.3958 0.4532 0.3660 0.4368 0.4720 0.2814 0.4916

P5 0.6563 0.2703 0.2316 0.1645 0.1214 0.0000 0.1799 0.1840 0.1987 0.3114 0.3638 0.3190 0.3742 0.3963 0.2671 0.4056

P6 0.5082 0.1388 0.2184 0.2431 0.2418 0.1799 0.0000 0.1187 0.2950 0.1981 0.2573 0.3837 0.3826 0.3546 0.3523 0.3181

P7

Table 6.5 Adjacent matrix of nanoparticles in task space (lm) P8 0.5965 0.2079 0.1485 0.1695 0.2564 0.1840 0.1187 0.0000 0.2163 0.2600 0.3109 0.3191 0.3384 0.3533 0.2763 0.3557

P9 0.5006 0.4015 0.4194 0.3442 0.1939 0.1987 0.2950 0.2163 0.0000 0.2858 0.3451 0.2592 0.3250 0.3597 0.1835 0.3801

P10 0.3049 0.1900 0.4000 0.4201 0.3958 0.3114 0.1981 0.2600 0.2858 0.0000 0.1819 0.2509 0.2485 0.2262 0.2274 0.2210

P11 0.2360 0.2714 0.4743 0.4861 0.4532 0.3638 0.2573 0.3109 0.3451 0.1819 0.0000 0.2549 0.2415 0.2075 0.2332 0.1649

P12 0.3139 0.4933 0.5517 0.4847 0.3660 0.3190 0.3837 0.3191 0.2592 0.2509 0.2549 0.0000 0.1953 0.2218 0.1182 0.2394

P13 0.2458 0.4779 0.5679 0.5283 0.4368 0.3742 0.3826 0.3384 0.3250 0.2485 0.2415 0.1953 0.0000 0.1729 0.1366 0.1859

P14 0.2124 0.4331 0.5545 0.5452 0.4720 0.3963 0.3546 0.3533 0.3597 0.2262 0.2075 0.2218 0.1729 0.0000 0.1589 0.1359

P15 0.4013 0.4661 0.4987 0.4263 0.2814 0.2671 0.3523 0.2763 0.1835 0.2274 0.2332 0.1182 0.1366 0.1589 0.0000 0.3007

P16 0.1947 0.3704 0.5444 0.5447 0.4916 0.4056 0.3181 0.3557 0.3801 0.2210 0.1649 0.2394 0.1859 0.1359 0.3007 0.0000

146 6 Path Planning of Nano-Robot Using …

6.3 Simulation and Experimental Verification 4

147

(3)

(4)

Nanoparticle

3 2 (5) (6)

1 y/μm

Target position

(2)

Nano-manipulation path

(7)

(8) (9)

(10)

0

Tip motion path based on ant colony algorithm

(12) -1

(13)

(15)

(11) (14)

-2 -3

(1)

-4

(16) -5 -5

-4

-3

-2

-1

0 x/μm

1

2

3

4

5

(a) Tip optimization manipulation path 4

(4)

(3) Nanoparticle

3 2

(5)

Target position

(8)

(6)

1

(7)

(2)

y/μm

(9) (12) -1

Nano-manipulation path

(10)

0

(13)

(15)

Tip motion path based on path A

(11)

(14)

-2

-3 -4

(1)

(16)

-5 -5

-4

-3

-2

-1

0 x/μm

1

2

3

4

5

(b) Random tip manipulation path A 4

(3)

(4)

Nanoparticle

3 (6)

2 (5)

(8) (7)

y/ μm

1 0

Nano-manipulation path

(10)

(9) (12)

-1

Target position

(2)

(15)

(13)

Tip motion path based on path B

(11) (14)

-2 -3

-4 -5 -5

(1)

(16) -4

-3

-2

-1

0 x/μm

1

2

3

4

5

(c) Random tip manipulation path B

Fig. 6.11 Comparisons of two random paths with the optimized path

6 Path Planning of Nano-Robot Using …

148

Table 6.6 Observation distances between multi-planned paths (lm)

1 2 3 4 5 6 7 8

Optimization path

Path A

Path B

0.1182 0.1953 0.1953 0.1359 0.1947 0.236 0.1819 0.19

0.4024 0.2571 0.3221 0.1214 0.1645 0.2431 0.295 0.2163

0.1953 0.1359 0.2218 0.1182 0.2671 0.1214 0.1939 0.2163

9 10 11 12 13 14 15 Sum

Optimization path

Path A

Path B

0.1388 0.1187 0.1485 0.1659 0.1645 0.1214 0.1939 2.499

0.26 0.1819 0.2549 0.2218 0.1729 0.1953 0.3007 3.6094

0.1695 0.1659 0.2184 0.1388 0.19 0.1819 0.236 2.7704

Fig. 6.12 Experimental result of nano-manipulation

2μm

6.4

Landmark Dynamic Configuration

When there is no landmark or the landmark far away from the task position, as shown in Fig. 6.13a, the localization accuracy of the tip is not high enough, which affects the manipulation result. Therefore, it is necessary to dynamically configure the landmark to push it near the target position to improve the localization accuracy of the tip, as shown in Fig. 6.13b. When dynamically configuring the landmark, the virtual nano-hand method (described in detail below) is used to push the nanoparticle stably and reliably to near the task point, and then the image is scanned again to determine whether the landmark after dynamically configuring meet the requirement. If the requirement is not met, the landmark is need to be reconfigured. Figure 6.14 is the flow chart of dynamic configuration of the landmark. In the process of dynamic landmark configuration, the longer the manipulation distance is, the more maneuvering times are needed because of the unstable

6.4 Landmark Dynamic Configuration

149

AFM Tip

AFM Tip

AFM Tip

PTask

PTask P'2

P1

P1

PTask P'2_L

P1_L

P1_L P2

P'2 P1 P1_L

P'2_L

P2 P2_L

(a) The position of the task point is far from the landmark

(b) Configure the active landmark to near the position of the task point

(c) Localize the tip at the task point after actively configuring the landmark

Fig. 6.13 Dynamic landmark configuration for AFM manipulation environment

Map creation

Calculation of landmark positions in the map

Selection of landmark

Estimate the localization error of the tip at the task position

Whether the localization N accuracy meets the requirement

Dynamically configuring the position of landmark

Landmark configuration using virtual Nano-hand

Y

Map reconstruction using landmark Move to the task point

Nano-manipulation Fig. 6.14 Flow chart of nano-manipulation and dynamic landmark configuration

6 Path Planning of Nano-Robot Using …

150

manipulation of nanoparticles. In order to ensure that the manipulation path of the tip is as short as possible and the localization accuracy of the tip is within the allowable range, the method to define a region around the landmark (called the landmark domain) is proposed in this book. First, the boundary of the landmark domain is calculated by the tip motion model, and then the landmark is pushed to the target position so that the task position is included in the landmark domain, which ensures the tip localization accuracy [6].

6.4.1

Definition of Landmark Domain

Landmark domain estimation method is represented as follows. It is assumed that the localization accuracy (Var(Pi)) and the target position accuracy (Var(Pt)) of the tip around the landmark are known, the distance (dl,t) of the tip from the localization position Pl to the target position Pt should meet the following constraints: Varðdl;t Þ þ VarðPl Þ ¼ VarðPt Þ

ð6:13Þ

The error distribution Var(dl, t) of tip motion increases linearly with the increase of moving distance (dl, t). Var(dl, t) can be calculated by the tip motion model. As shown in Fig. 6.15a, b, the tip moves in a single step along the horizontal or vertical direction during its movement. After observing the nanoparticle, the tip will be localized near the nanoparticle. In order to simplify the calculation of landmark domain, it is assumed that the tip is localized around the center of nanoparticle and the distance between the tip and the center of nanoparticle is greater than or equal to the sum of the radius of nanoparticle and the radius of the tip, as shown in Fig. 6.15c, so as to ensure a high accuracy of the tip localization and avoid collision between the nanoparticle and the tip. According to analysis of tip localization accuracy in Sect. 6.2, the localization accuracy around the nanoparticle is approximately equal. These positions can be seen as a circle with a radius of Rn_t.

Cl Nanoparticle

Tip

Rn,t (a) The tip moves vertically

(b) The tip moves in the arbitrary direction

Fig. 6.15 Tip motion and localization

(c) Tip localization around the landmark

6.4 Landmark Dynamic Configuration

151

Figure 6.16 sets the center point (0, 0) as the landmark position, and calculates the domain boundary of the landmark according to the relationship between the distance and the position error of the tip in the tip motion model. Formulas (6.14), (6.15) take the horizontal variance and the vertical variance as constraints, respectively. Var_x (*) and Var_y (*) are used to calculate the variance of a specific position or a certain motion. The variance of translation dl,t is calculated by the motion model, and formula (6.13) can be derived from formulas (6.14) and (6.15). The region that satisfies three restrictions is the required domain of the landmark (the black area in the center of Fig. 6.16). Var xðdl;t Þ þ Var xðPl Þ ¼ Var xðPt Þ

ð6:14Þ

Var yðdl;t Þ þ Var yðPl Þ ¼ Var yðPt Þ

ð6:15Þ

In the process of dynamic landmark configuration, nanoparticles need to be pushed near the target position to ensure that the target position is within the domain of the landmark, that is, to complete the landmark configuration. In the nano-manipulation process, the errors of the position, size, direction and dynamic model of the pushing point lead to the instability of manipulation result. In order to achieve the goal, many manipulations are needed, and the maneuvering result are closely related to the operator’s experience. Therefore, the dynamic landmark configuration method can use virtual nano-hand and other methods to improve the nano-manipulation efficiency [7].

Fig. 6.16 Landmark domain that satisfies the constraints

3

2

y (μm)

1

0

-1

-2

-3-3

-2

-1

0

x (μm)

1

2

3

6 Path Planning of Nano-Robot Using …

152

6.4.2

Virtual Nano-hand Method

In the nano-manipulation process, because of the uncertainty of the position, pushing force size, direction of the interaction point between the tip and the object to be manipulated and the uncertainty of manipulation dynamics model, the output position of the manipulation model cannot actually reflect the position after manipulation. Although some researchers have proposed a real-time feedback method based on local scan to feed back the actual maneuvered position to the manipulation interface in real time for assisting the operator in nano-manipulation, this method also needs repeated operations, and the manipulation effect is closely related to the operator’s experience. Therefore, in order to further improve the efficiency of nano-manipulation, the single-push manipulation of the tip is planned as multiple operations, forming a virtual nano-hand to realize the stable operation of the controlled object. In order to realize the stable manipulation of nanoparticles, on the foundation of pushing model of the nanoparticle (as shown in Fig. 6.17), the uncertainty between the tip position and the nanoparticle position is considered in this book, which makes a preliminary study on the design of virtual nano-hand. Figure 6.17 is a kinetic model analysis for pushing the nanoparticle. According to the experiment result, there are forward displacement and longitudinal deflection of nanoparticle when the nanoparticle is pushed by AFM tip. That is to say, the nanoparticle under the action of the tip can be regarded as a uniform circular motion with angular velocity x around the transient rotating center IRC. The equation of motion satisfies the following requirement:

y Vp

RP R

ω

AFM Tip

y0

O VO

VP RP

R O

LP

x

Nanoparticle IRC

(a) Kinematic analysis of the tip manipulation of the nanoparticle

x0

(b) Torque analysis of tip manipulation of the nanoparticle

Fig. 6.17 Model analysis of pushing nanoparticle by using AFM tip

6.4 Landmark Dynamic Configuration

153

VP VO ¼ LP S

ð6:16Þ

where Vp is the velocity of tip motion, VO is the instantaneous velocity of the nanoparticle, LP and S are distances from the transient rotation center to the tip center and the center of the nanoparticle respectively. Here Vp is known, and VO identifies the motion state of the nanoparticle, which is the resolve target. LP is related to S, as long as one of them is known, the other value can be derived. The process of calculating S is as followed, during movement of the nanoparticle, the Coulomb friction force associated with the positive pressure and the viscosity-related viscous friction force are combined to form the main friction, so S can be calculated according to the balance between the pushing force and the friction force and the balance of the torque. Furtherly the rotation center position of the nanoparticle is calculated to describe the motion state. p 2þ

R

ZarcsinS

ðfh
sin h
LP
cos h0 Mh Þ
dh ¼ 0

ð6:17Þ

p R 2 arcsin S

In the formula, fh is the “total friction” calculated after integration of the contact area between the nanoparticle and the substrate. And Mh is the torque when the nanoparticle is subjected to the substrate. Reference [7] is provided for the detailed description of the relevant parameters. On the basis of the tip-pushing model, it is assumed that the relative position of the nanoparticle and the tip is uncertain, so is the contact point of the actual force between the tip and the object to be manipulated, which will lead to the uncertainty of the tip-pushing result. However, under the virtual nano-hand based manipulation, the nanoparticle will be stabilized in a distribution area during the pushing process.

6.4.3

Nano-manipulation Simulation Based on Virtual Nano-hand

The simulation result is shown in Fig. 6.18. The probability distribution of the particle center position is a normal distribution of r = 10 nm. The uncertainty distribution is described by Monte Carlo method. In Fig. 6.18a, the red dot represents every possible position of the nanoparticle center, and the number of sampling points is 500. If the AFM tip pushes the particles according to the plan, the pushing force acts on all possible positions and obtains a corresponding number of pushing results. In this way, the planning pushing strategy can be evaluated. We call it Monte Carlo-based stochastic pushing model. The simulation result shows that the planning of the tip position and trajectory is very important to achieve stable nano-manipulation, and different manipulation strategies may produce

6 Path Planning of Nano-Robot Using …

Rp

0

0

y (nm)

200 100

y (nm)

200 100

y (nm)

200 100

Rp

-100

-100

0

0.10 Probability

154

Rp

-100

0.02

200 y (nm)

100

Tip motion path

200 100

(d)

0

-100

0.30

Probability

y (nm)

y (nm)

y (nm)

200 100

0

0.20

0

-100

First Second -200 push -200 push -200 -200 -100 0 100200300 -200 -100 0 100200300 -200 -100 0 100200300 x (nm) x (nm) x (nm)

(f)

180

The uncertainty of the result is reduced Deflection angle distribution of nanoparticles relative to the horizontal pushing direction

(e)

-100

0 90 Angle(°)

Deflection angle distribution of nanoparticles relative to the horizontal pushing direction The uncertainty of the result is large

-200 -200-100 0 100200300 x (nm) 200 100

0 -180 -90

(c)

(b)

Center 0 position of nanoparticle -100

0.06

0.04

-200 -200 -200 -200-100 0 100200300 -200 -100 0 100200 300 -200 -100 0 100200300 x (nm) x (nm) x (nm)

(a)

0.08

(g)

0.10 0 -180 -90

(h)

0 90 Angle(°)

180

(i)

Fig. 6.18 Simulation analysis of pushing the nanoparticle by using AFM tip

different results. This is a comparison of the simulation result of the traditional and proposed strategies considering the uncertainties of the central position of the nanoparticle. One is the traditional method of pushing the nanoparticle center. The simulation result shows that the probability distribution of the particle center is no longer Gauss distribution after the first push operation. The reason is that the position of some particle changes during manipulation, and form a circular arc with radius RP near the termination position of the tip, while some particles do not contact the tip because of the small push step and remain unchanged. After continuous push operation, the probability distribution of particle center is transformed into one-dimensional arc distribution, and the distribution area of particle points decreases. It shows that although the manipulation strategy based on particle center can control the tip with high precision, the manipulation result is still uncertain, which requires a stable push strategy. Figure 6.18e shows the uncertainty distribution of the nanoparticle position relative to the tip; Fig. 6.18d shows the angle distribution of the nanoparticle center moving direction relative to the horizontal direction after the tip pushing repeatedly along the horizontal direction of the nanoparticle center from Fig. 6.18a–c; after the nanoparticle is pushed by virtual nano-hand (from Fig. 6.18f–h), the angular distribution of the center moving direction relative to the horizontal pushing direction is shown in Fig. 6.18i. The manipulation strategy of pushing the particle center divides the uncertainty distribution of the nanoparticle center into two parts and increases with time, which

6.4 Landmark Dynamic Configuration

155

results in the instability of operation. If the strategy of symmetrical manipulation on both sides of nanoparticles is adopted, that is, the pushing points of continuous manipulation are located on both sides of the particle center, and the pushing path is along the edge of the distribution area of the particle center, thus ensuring that the nanoparticle center is always distributed on one side of the tip. The simulation results show that the possible position of the center is still between two parallel manipulation paths after six push operations, and the particle distribution range is convergent in a certain region, which achieves stable nano-manipulation.

References 1. Chen, X., Zhao, Y.L., Han, J.D.: An improved ant colony algorithm for robot path planning. Control Theor. Appl. 27(6), 821–825 (2010) 2. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: A probabilistic approach for on-line positioning in nano manipulations. In: Proceedings of IEEE Conference WCICA 8th, JiNan, China, pp. 450–455 (July 2010) 3. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: A probability approach for on-line tip localization with local scan based landmark sensing in nanomanipulations. In: Proceedings of Conference 3M-NANO 1-th, China, pp. 1–6 (2011) 4. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: AFM tip on-line positioning by using the landmark. In: IEEE NMDC 10th, CA, USA, pp. 75–80 (2010) 5. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: Stochastic trajectory planning and navigation for AFM based nano robotic manipulation. In: IEEE WCICA 11th, Shenyang, China, pp. 958–963 (2014) 6. Yuan, S., Liu, L.Q., Wang, Z.D., et al.: Active landmark configuration for accurate nano-positioning. In: IFAC Symposium on Mechatronic Systems 6th, Hangzhou, China, pp. 594–599 (2013) 7. Hou, J., Liu, L.Q., Wang, Z.D., et al.: AFM-based robotic nano-hand for stable manipulation at nanoscale. IEEE Trans. Autom. Sci. Eng. 10(2), 285–295 (2013) 8. Hou, J., Liu, L.Q., Wang, Z.D., et al.: Modeling and analyzing nano-particle pushing with an AFM by using nano-hand strategy. In: IEEE International Conference on Nano/Micro Engineered and Molecular Systems 5th, Xiamen, China, pp. 518–523 (2010)

Chapter 7

AFM-Based Nano-manipulation Platform

Abstract In the previous chapters, the problems of tip localization and path planning in AFM nano-manipulation are studied, and the landmark localization and path planning based on stochastic method is proposed. In order to study these theories and methods, it is necessary to improve the existing experimental platform and build a new augmented reality nano-manipulation system for real-time feedback of task space. On this foundation, the calibration of the algorithm model parameters and experimental verification of theoretical methods is performed by using a large number of related experiments. On this experimental platform, the landmark observation model and the tip motion model based on the stochastic method are used to localize the tip position in task space. Meanwhile, a statistical experimental scheme is designed. For these parameters in the models, such as PI model parameters, creep model parameters, thermal drift model parameters, landmark position error and local observation scan errors are calibrated, and the effectiveness and rationality of the proposed method are illustrated by tip localization experiments. Based on the tip localization, the stable nanoparticle pushing manipulation is realized by virtual nano-hand. The effectiveness of the method is verified and the efficiency of nano-manipulation is improved. It provides technical support and guidance for AFM automated nano-assembly manufacturing. Keywords Experimental platform tem Parameter calibration




7.1


Real-time feedback
Nano-calibration sys-

Hardware and Software Implementation of System

The system is the integration platform of the abovementioned research contents. The platform is based on Dimension 3100 of Veeco Company, USA. It integrates the tip model method based on blind modeling algorithm, the AFM tip localization strategy based on stochastic approach and the nano-manipulation technology based

Authors: Shuai Yuan, Lianqing Liu, Zhidong Wang, Ning Xi. © Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9_7

157

7 AFM-Based Nano-manipulation …

158

on virtual nano-hand, which lays the foundation for efficient nano-manipulation. The hardware structure and software framework of the system are given below.

7.1.1

Hardware Platform

The hardware of AFM based nano-manipulation system consists of three parts: AFM actuator, the AR system and the real-time control module, which are controlled by three computers respectively. The core of the AFM actuator is composed of Dimension 3100 equipped with three-axis closed-loop scanning. The maximum scanning range of the system is 110 lm  110 lm, and the Z-direction stretching range is 7 lm. The peripheral equipment of the AFM system also includes optical microscope, CCD camera and signal interface module, in which optical microscope and CCD camera help the operator to find the interested sample area and avoid the tip damage caused by the collision between the tip and the sample in the process of the tip approaching the sample. The signal interface module connects the scanning head, the controller and other devices of the AFM, through which the operator can obtain all real-time signals in nano-manipulation. The main control computer of the AFM is connected with the controller, and the corresponding software is run to provide the operation interface of the AFM imaging. Augment reality system provides an augmented reality environment for operators, including a tactile-feedback handle and a computer capable of providing visual feedback and operation. In nano-manipulation, the operator can not only control the three-dimensional tip motion by using the handle, but also sense the interaction between the tip and the object in real time. The visual display interface provides the operator with a real-time updated image of the manipulation results, which is generated by the environment model and local scan, so it has high reliability. The real-time control module runs on a real-time control computer equipped with Data Acquisition (DAQ) card (produced by Yanhua Company). The computer controls the DAQ card to carry out real-time control module for data input and output, local scan module and tip real-time movement. It can be seen from the system structure diagram that the DAQ card directly outputs the voltage signal to the modified AFM controller to drive the tip motion, so the real-time and high-speed control of the tip can be realized by the computer. In the local scan process, the voltage signal applied to the Z direction of the PZT can be obtained. The signal represents the surface morphology information along the scanning path. After collecting the morphology information along the scanning path through the signal interface module, the real nano-manipulation results can be obtained and the localization based on the landmark can be achieved.

7.1 Hardware and Software Implementation of System

7.1.2

159

Software Implementation

The three computers in the hardware system are interconnected by LAN, which form a nano-manipulation system with augmented reality feedback together. The human-computer interface provided on Manipulation PC is designed and implemented with OpenGL and VC programming language, which ensures the efficiency and performance of the program. The flow chart of the program is shown in Fig. 7.1a. It mainly implements lithography and manipulation. It needs to work with the other two computers during manipulation. Figure 7.1b, c show the program flow of the computer (Main PC) responsible for the control of the AFM actuator. During the program run, it needs to interact with the Manipulation PC as shown in Fig. 7.1d. The motion command of the tip is obtained, local scan is performed, and the calculation result is sent back to the augmented reality system. Planning PC is responsible for collecting the scanning information of the AFM actuator and controlling the tip for fast local scan according to the requirement of the program running on Manipulation PC. The software of nano-observation and manipulation system is based on the tip manipulation system, which provides the functions of sample scanning imaging, molecular force detection, nano-scale localization operation and so on. The operation

0

Start

Start

Start

Start

Initialization

Initialization

Initialization

Initialization

Judge 1 the operation mode

Characterize

Push

0

Judge the operation mode

Handle mode

1

Predefined mode

Send data

Is there data?

N

Wait for Y Mutex W

Y

N

Wait Y for Semaphore W N

Fetch data

Receive data

Release Murex W

Release Semaphore R

Put data into buffer

Send data

Manipulation

Release Murex W

Release Semaphore W

Local scan

Finished

Send data

Receive data

Wait for Y Mutex W

N

Receive experimental images

N

Y Wait Y for Semaphore R N

Whether N completed Y

End

End

(a) Flow chart of manipulation program

(b) Flow chart of receiving data

(c) Flow chart of transferring data

Fig. 7.1 Program flow chart of AFM nano-manipulation system

N

Finished Y End

(d) Flow chart of command execution

7 AFM-Based Nano-manipulation …

160

Fig. 7.2 Human-computer interface for AFM nano-manipulation

software includes real-time image graphics generation module, human-computer interaction interface (as shown in Fig. 7.2), tip localization and force control module, multi-dimensional haptic operation interactive device (Phantom) and network communication interface. Figure 7.3 shows the process of arranging nanoparticles into structure using a manipulation model and an observation model during nano-manipulation. Figure 7.4 shows the enhanced virtual reality interface during tip lithographing. Figure 7.4 includes 2D and 3D interfaces. The depth and width of the groove are determined according to the tip shape and the depth of the depression.

Nanomanipulation

Model Position

Observe position

Nanomanipulation

Fig. 7.3 Real-time feedback nano-manipulation process based on local scan

7.2 AFM Tip Localization

161

(a) Nano-lithography manipulation interface

(b) 3D visual feddback in manipulation

Fig. 7.4 Augmented virtual reality interface in tip lithographing

7.2 7.2.1

AFM Tip Localization Framework of Tip Localization System

The framework of the tip localization system (Fig. 7.5) adds a stochastic approach for feature based tip localization and planning (SAFLP) module contrasting with other AFM manipulation diagrams in Chap. 1. The module provides real-time feedback control of tip localization in task space, which reduces the position error

Environmental model

Real-time force information

Partial scan Kalman diagnostic

Active probe controller

AFM system Real-time visual feedback interface

Augmented reality environment

Force feedback device

Operation command

Fig. 7.5 Framework of stochastic approach based nano-manipulation system for AFM tip localization and planning

7 AFM-Based Nano-manipulation …

162

distribution caused by PZT non-linearity, system thermal drift and other uncertainties in the system, and achieves the precision of the tip relative to the object being manipulated, which improves the efficiency of nano-observation and manipulation.

7.2.2

Model Parameters Calibration and Experimental Verification

(1) Experimental Calibration of Model Parameters The parameters of tip motion model and landmark observation model are calibrated by statistical experiment, and then the algorithm is verified by simulation and experiment of tip localization. 1) Calibration of Motion Model Parameters The motion model includes PI model, creep model and thermal drift model. The calibration of these three models is described as follows: • Parameter Calibration of PI Model The PI model in the motion model is calibrated in the horizontal and vertical directions after fitting the sampling points of PZT hysteresis by using least square method. The calibration process in the vertical and horizontal directions is detailed in this book. The hysteresis sampling points of PZT in the vertical direction will be obtained through locating the pits punched by the tip on the CD surface in Fig. 7.6a. The process of tip indentation is carried out according to the hysteresis sampling plan as follows: firstly, the AFM scans in the PZT calibration range of 5 lm  5 lm to find a flat area. secondly, the tip stops scanning and moves to the center of the scanning area, while the input control voltage of PZT in the lateral direction (including x and y) remains 0. Thirdly, the y-direction input control voltage gradually increases to the upper bound of the calibration range with the sampling step size, then gradually decreases to the lower bound of the calibration range, and finally gradually increases to the upper bound of the calibration range again. During the whole movement between the upper and lower bounds of the last two calibration ranges, the tip indentations a pit as a sampling point with four steps interval for identifying the PZT hysteresis. After the hysteresis sampling points are obtained by using the abovementioned steps, the PI parameters are calibrated by using least square method, and then the control voltage of the inverse model compensation PZT is obtained. When the PI model is used to compensate the PZT driver, the model error of the tip motion will increase with the increase of the tip motion distance. The PI model error analysis method is to calibrate the motion errors with six groups of expected

7.2 AFM Tip Localization

163 100 Number of samples

80

μ = 0.3733 σ = 0.0083

60 40 20

0 0.30

0.35 0.40 0.45 0.50 Displacement distance (μm) (c) Experimental data statistics for model compensation 0.8 FV (x) = a * x + b a = 0.981; 0.6 b = 0.00047;

Output displacement distance(μm)

97 pits

0.4 0.2

Output displacement covariance σ2

1μm

0 0.8 0.6 0.4 0.2 0 Expected displacement distance (μm) (d) Model data compensation for experimental compensation -4 4 ×10 SV (x) = a * x + b

1μm

Input control voltage interval between tip pits (four sampling steps) The input voltage decreases, and the pit is indented with four steps interval . The input voltage increases, and the pit is indented with four steps interval .

Top

3

2

a = 5.51e-005; b = 5.8e-005; Upper error limit

1 0

0.8 0.6 0.4 0.2 0 Expected displacement distance (μm) (a) PI model calibration (b) Experimental data (e) Upper bound calibration of model after compensation error data data

bottom

Fig. 7.6 PZT establishes PI compensation model in the direction of vertical motion and carries out error analysis

distances (8, 12, 16, 24, 32, 64 integer times of a basic step) in the range of tip motion calibration, and then to count the model motion errors of PZT in the whole range of calibration. In Fig. 7.6a, in order to obtain the hysteresis sampling points of PZT in the vertical direction of motion, the input voltage gradually changes between the minimum and maximum in accordance with the sampling step size to generate a cycle. The tip presses a pit with four sampling steps interval on the CD surface. Figure 7.6b is the partial experimental result of the tip pressing a pit with every 32 basic steps interval after the PZT is compensated by the PI model, and the error analysis of the tip motion distance (the expected distance is 0.381 lm) is

164

7 AFM-Based Nano-manipulation …

performed. Figure 7.6c is a fitting of the experimental data in Fig. 7.6b using the Gaussian method. Figure 7.6d is the relationship between 6 groups of expected distances of the tip motion and the experimental distance, whose relationship satisfies the linear relationship. Figure 7.6e is an error characteristic diagram of the tip motion model, which increases as the desired distance increases. The error statistics method for each group of expected distances (statistic of motion model in vertical direction) is that the tip moves repeatedly between the top (upper bound) and bottom (lower bound) along the y-direction within the calibration range. During the tip motion, the pit is punched with an expected distance interval. The experimental results of the expected distance are obtained by measuring the distance between adjacent pits. The experimental results of the tip motion model on this group of expected distance are counted for obtaining the error statistical result. After obtaining the error statistical results of the six groups of discrete expected distances within the calibration range of the tip, the model error of the tip during continuous motion is further statistically obtained. • Calibration of Creep Model The creep model of PZT is calibrated in horizontal and vertical directions by fitting creep sampling points with non-linear least squares method. This book will detail the calibration process in the horizontal direction, and the calibration process in the vertical direction is the same. The creep sampling point of PZT in the horizontal direction will be obtained by locating the pits punched by the tip on the CD surface (shown in Fig. 7.7a). The process of tip indentation is carried out according to the creep sampling plan as follows. In the first step, similar to the PI model calibration method, the AFM tip first scans in the range of 5 lm  5 lm to find a flat area, then stops scanning and moves to the center of the scanning area, while keeping the input control voltage of PZT in the lateral direction (including x and y) to zero. In the second step, the input control voltage in the x-direction decreases gradually to the lower bound of the calibration range (the left boundary of the scanning area) according to the sampling step size, and remains unchanged; in the third step, the control voltage in the y-direction decreases gradually to the lower bound of the calibration range and remains unchanged; in the fourth step, the control voltage in the x-direction gradually increases to the center of the calibration range, i.e., 0 value. Finally, the y-direction control voltage gradually increases to the upper bound of the calibration range, while the tip punches a pit with four basic steps interval during movement to identify a sampling point in the vertical direction of PZT creep. After 20 groups of sampling points of creep curve are obtained, the creep parameters are calibrated by using non-linear least square method to compensate the tip motion model, as shown in Fig. 7.7b. When the creep model is used to compensate the PZT driver, the creep model error increases as PI error increases with the increase of the tip motion distance. The error analysis method of creep model is to calibrate the error of PZT during the creep period (25 s) at each sampling time (a sampling interval is 60 basic steps of tip motion), and then calculate the model motion error of PZT during the creep process. At each sampling time, 20 groups of experimental data were fitted by

7.2 AFM Tip Localization

165

Creep curve (μm)

Sample number

20 15

10 5 0 0.02

Time(sec)

(b) Fitting of experimental data of creep model One sampling step

Expected value of creep model error (μm)

Creep model

0.03

0.05

0.04

Creep distance after 3.83s (μm) (c) Statistics of creep experimental data -3

10

×10

5

DH(x) = a * x + b a = -1.49e-005; b = 0.0010;

0

Step 4: The input voltage increases in the vertical direction. The tip indents a pit with eight steps interval. Step 3: The input voltage gradually increases in the horizontal direction, and the tip is moved right to the center of the scanning area.

(a) Calibration of creep model experimental data

Covariance of creep model σ2

1μm

Step 1: The input voltage gradually -5 decreases in the horizontal direction, and the tip moves left to the left boundary. -10 0 5 10 15 20 Step 2: The input voltage gradually Time(sec) decreases in the vertical direction (d) Fitting of creep experimental data and the tip moves down to the lower boundary -5 C H (x) = a * x + b 5

×10

4 3

a = 2.8e-007; b = 1.7e-005; Upper limit of error

2

1 0

0

5

10

15

20

Time(sec) (e) Calibration of the upper limit of the experimental data error

Fig. 7.7 The creep model of the tip in the horizontal direction is established and the error analysis is performed

statistics, as shown in Fig. 7.7c. The statistical results satisfy the Gauss distribution, and the difference between the mean of the Gauss distribution and the expected value of the model was calculated. Statistical analysis of the difference at different sampling times shows that the difference tends to zero, as shown in Fig. 7.7d. At the same time, the error distribution of different sampling time is counted. The error increases linearly with time, as shown in Fig. 7.7e. • Parameter Calibration of Thermal Drift Model The system thermal drift model uses the data of continuous scanning imaging in different time to calculate the thermal drift speed. Some continuous scanning images are shown in Fig. 7.8. After several hours of work, the thermal drift velocity of the system fluctuates around a value and gradually tends to be stable. According to the statistical analysis of the experimental results, the thermal drift velocity satisfies the Gauss distribution, the mean value of the thermal drift velocity in the x-direction is close to 0, and the thermal drift velocity in the y-direction is close to a constant, as shown in Fig. 7.9.

7 AFM-Based Nano-manipulation …

166

P1

P2

P3

P4 P5 P7

P6

1μm

Nanoparticle image 1 with frame up

Nanoparticle image 1 with frame down

Nanoparticle image 2 with frame down

Nanoparticle image 3 with frame up

Nanoparticle image 2 with frame up

Nanoparticle image 3 with frame down

Fig. 7.8 Thermal drift characteristics of AFM system using a continuous imaging region of nanoparticles

150

μ = -0.004 nm/s σ = 0.027 nm/s

Sample number

Sample number

150

100

100 50 0 -0.2

μ = 0.099 nm/s σ = 0.209 nm/s

50 0

-0.1

0 Vx_d (nm/s)

0.1

(a) Statistics of thermal drift speed in x-direction

0.2

-0.4

0 0.2 -0.2 Vy_d (nm/s)

0.4

(b) Statistics of thermal drift speed in y-direction

Fig. 7.9 Calculating the thermal drift velocities in the x and y directions of the horizontal plane based on the abovementioned thermal drift estimation strategy

(2) Calibration of Observation Model The tip position is estimated by observing the center of the nanoparticle xkp. The observation error is divided into three parts:

7.2 AFM Tip Localization

vmap  N 0; Qmap

167




vz

kl

 N ð0; Qz

kl Þ

vh  N ðdh ; Qh Þ

ð7:1Þ

• Calibration of Landmark Position Task space coordinate system is established on the scanned image of the target manipulation area on sample surface. Because the scan process of the tip is affected by PZT driver non-linearity, system thermal drift and other uncertainties, there is uncertainty in calculating the center position of the nanoparticle in the image. The error (uncertainty) analysis method is to continuously scan multiple images of area with dispersed nanoparticles. Figure 7.10a shows that after registration of these images representing the morphology of the sample, the position of specific nanoparticle is calculated in each image, and then the distribution characteristics of position uncertainty are obtained by statistical analysis of these data. The experimental data are 40 continuous scanned images of the nanoparticle P2 corrected in x, y and z directions. Moreover, six different height thresholds are used for each map to calculate the center of the nanoparticle. Map registration is to correct the overall offset of the scanned image in the x, y and z directions, which are obtained at different times in the system. The specific steps are as follows: Firstly, the top area center of the reference nanoparticle P1 (shown in the left-hand image of Fig. 7.10a) in each map is set as the correction control point in the height direction, and one of the images is selected as the reference image, the control point of P1 in the reference image is used as criteria to correct z-direction coordinates in other maps. The second step is to set the image height threshold, binaries the image, and calculate the position of P1 and P2 by calculating the center of gravity. The third step is to calculate the position of nanoparticle P2 relative to P1 in each map, and compensate for the deviation in x and y directions. In the fourth step, the position of P2 in different maps is counted. The calculated results satisfy the Gauss distribution, and the calibration values of vmap errors are obtained, as shown in Fig. 7.10b, c. • Error Calibration in Local Scan of Different Landmark Areas In landmark observation, local scanning of different region of the nanoparticle will affect the calculating results of the nanoparticle center. Figure 7.11 shows that tip may locally scan the nanoparticle different region along different paths in real time, such as l0 or l1. In order to calibrate this uncertainty, the position of the nanoparticle center is calculated by intercepting the value of the pixels along the scan line in multiple images as the real-time scan line. Then, the distribution of these positions of the nanoparticle center is counted. As shown in Fig. 7.11, it satisfies the Gauss distribution and the error calibration of different regions of local scanning the landmark is obtained. The data used in the statistics are 40 images of nanoparticle P2 with center position errors. There are 11 local scan lines in each image. The calculated result of nanoparticle P2 center are fitted with Gaussian distribution curve. • Error Calibration of Local Scan Line Deflection The mechanical structure and creep effect of the PZT scanner deflect the local scan line, which affects the observation result of the nanoparticle center. In this

7 AFM-Based Nano-manipulation …

168

P1

P2

Nanoparticle

1μm 11:39

11:43

11:47

11:52

11:56

(a) Calculate the center of the nanoparticle using successive scan images and different height thresholds 200

μ = 0 nm σ = 5.0 nm

150

Sample number

Sample number

200

100

50

μ = 0 nm σ = 4.5 nm

150 100 50 0

0 -15 -10

-5

0 5 x (nm)

10

15

(b) Position error of the nanoparticle in x-direction

-15 -10

-5

0 5 y (nm)

10

15

(c) Position error of the nanoparticle in y-direction

Fig. 7.10 Statistical error distribution for calculating the center of nanoparticle in maps

l0 l1

400 350 Sample number

Fig. 7.11 Observation error distribution of the nanoparticle center due to scan the nanoparticle different regions

300 Scan different regions of the nanoparticle

250 μ = 0 nm σ = 4.7 nm

200 150 100

50 0 -15

-10

-5

0 x (nm)

5

10

15

experiment, the tip motion is controlled by program, and the deflection is obvious. In order to obtain and count the deflection angle h of the scanning lines, the tip punches the pit at the two ends of the scan line. As shown in Fig. 7.12a, the deflection angle is calculated according to the position of the pits. According to the statistics of the deflection angles of these scan lines, the mean of the deflection angle is less than 1°, and the variance is less than 0.392°, which satisfies the Gauss distribution. In Fig. 7.12b, dkp is the vertical distance from the center of the

7.2 AFM Tip Localization Fig. 7.12 Error analysis caused by local scan line deflection in landmark observation

169

Scan line

θ

Sample number

1 μm (a) Calibration of experimental data for local scan line deflection

Center

dkp

θ

30

μ = 0.746 ˚ σ = 0.393 ˚

20 10 0

0

3 1 2 Angular offset (˚)

(b) Calculation of the deflection (c) Statistics of local scan line deflection of the local scan line

nanoparticle to the scan line of the nanoparticle. It is assumed that the maximum dkp is the radius of the nanoparticles, the deflection extremum of the center calculation of the nanoparticle caused by the deflection of the scan line is close to 2 nm. Compared with the abovementioned error variance 5 nm, the influence on the observation model can be neglected. Furthermore, if the tip motion is directly controlled by hardware, the deflection degree and the scan line deflection in the scanned image are the same order of magnitude, which has less impact on the observation results and need not be considered at all. The deflection angle of the scan line is obtained by punching pits at both ends of the local scan line. The mean of deflection angle of the scan lines is less than 1° by counting 40 scanning lines. (2) Landmark Based Localization Experiment According to the previous simulation verification scheme, this chapter will use the corresponding tip localization experiment to illustrate the effectiveness of the tip localization method based on landmark observation. The device used in the experiment is Veeco’s Demision 3100. The object of the experiment is the polystyrene spheres distributed on the CD surface. The tip used in the observation, imaging and manipulation is Macromesh’s NS15/ALBS/15 tip. This algorithm uses the above parameter calibration method to calibrate the parameters of motion model and observation model. The tip motion control is shown in Fig. 7.13. In the experiment, the tip is moved to the initial position of x0, it is assumed that the tip has the same initial distribution at the starting position of x0. The tip moves to x8 by two paths, one is to move directly through xd_1 (the path is marked by dotted lines) without observing the landmark, the other is to use nanoparticles near the target position as a landmark to

7 AFM-Based Nano-manipulation …

170 1.5

x0 Starting point

1.5

Path planning based on landmark observation Path planning based on direct motion

1

x5 x1

x4

x2

x3

Nanoparticle

-1

-1

x6

x7 -0.5

0 x (μm)

0.5

1

Image center

0

x5

-0.5

x3

-1

xd_1

x8 1.5

-1.5 -1.5

x4

x2 Nanoparticle

Target position

xd_1

y (μm)

y (μm)

0.5

0

-1.5 -1.5

In the initial localization, path planning for moving back to x0 after observing the landmark

1

0.5

-0.5

x0 Starting point

-1

x6

x7 -0.5

0 x (μm)

0.5

1

Target position

x8 1.5

Fig. 7.13 AFM tip motion path plan based on landmark observation

observe the tip position at x8. The position error distributions of the two paths are compared to illustrate the effectiveness of the landmark-based observation method. Figure 7.14 shows the flow chart of the experiment. Tip localization experiments based on direct movement and landmark observation are to record the position of starting point x0, target point x8, route point xd_1, or position of x2, x5, x6 on CD surface by indentation. In the localization experiment based on landmark observation, the tip moves to x6 to estimate its position after observing the nanoparticle. After improving its position accuracy, the tip moves to the target point of x8 according to its motion model. In this experiment, the position distribution of the tip needs to be counted, which has three distinct characteristics: • In order to obtain the statistical distribution of the tip position, a certain number of repetitive test results are needed. • In view of the small area of the tip position distribution, if the same nanoparticle is used to observe the landmark and press pit to record the tip position, it may lead to the overlap of the pits in different groups of experiments, and the experimental records cannot be obtained. • In order to avoid the overlap of the pits in different groups of experiments, different nanoparticles should be used for observation in each group. In the experiment, nanoparticles with radius of about 100 nm are distributed on the CD surface, and flat areas with multiple nanoparticles are found by imaging. Then the algorithm stops scanning and stops the tip in the center of the scanning area. Next, the nanoparticles are used as landmarks to carry out statistical experiments of tip localization. The comparative experiments based on the two tip control models are conducted in 50 separately. The coordinate system of each group of experiments is registered by setting the center of nanoparticle as the control point. The first step in the experimental scheme is to move the tip to the starting position x0 and satisfy the same distribution. The mobile strategy is shown in Fig. 7.13: Firstly, the tip moves from the center of the scanning area to the position x2.

7.2 AFM Tip Localization

171

Initialization

Open-loop scan to find an area where exists nanoparticles

Stop scanning, move the tip to the center of the scan area,then move the tip to position x2

Move the tip to position x0 according to the initialtip positioning flowchart

Perform a local scan with the path:x2 x3 x4 x5 x6

Plan the path, move the tipfrom x2 to x3, perform a horizontal landmark observation, and estimate the tip position

Optimal estimation atx6, plan the path, and move the tip tox0, the path is:x6 x7 x8 x9 x0

Plan the path, move the tip from x3 to x5, then indent and record the tip position Move the tip from x5 to x6 and observe the nanoparticle in the vertical direction

Indent pit at position x0 , recording the tip position, and the position distribution of the tip at x0 in each groups of experiments is consistent by using the above mentioned path planning.

Plan the path,move the tip from x6 to x8, then indent and record the tip position Rescan this region to obtain position x0, x2, x5, x6 and x8 which are relative to the center of the nanoparticle.

Fig. 7.14 Flow chart for controlling tip motion in validation experiment

Secondly, perform local scan (moving path: x2 ! x3 ! ∙∙∙ ! x6), observe the center of nanoparticles, and use the observation model to estimate the optimal location x6. Thirdly, calculate the distance difference between x6 and x8 after improving the position accuracy of the tip at x6, and move to x8 according to the motion model (moving path: x6 ! x7 ! x8). Finally, the tip returns to x0 along the path x8 ! xd_1 ! x0, marked by the dotted line in Fig. 7.13b. Because the tip has a long moving distance from x8 to x0, the error distribution area of position x0 increases greatly. The simulation result based on the two tip control models are shown in Fig. 5.15, and the corresponding experimental results are shown in Figs. 7.15, 7.16 and Tables 7.1, 7.2. Fifty groups of experiments based on direct movement are performed, and a group of experimental result are shown in Fig. 7.15. The center of nanoparticle acts

7 AFM-Based Nano-manipulation …

172

x0

Punched dent

Direct motion based experimental result 1μm

1.5

x0

Direct motion based experiment

1.0

y (μm)

0.5 0

-0.5 -1.0 xd_1

x8

xd_1

-1.5 -1.5

(a) Experimental data for direct motion of the tip

-1.0

x8

-0.5

0 x(μm)

1.0

0.5

1.5

(b) Statistics analysis of tip localization experiments based on direct motion

Fig. 7.15 Localization experimental result based on direct movement

x0

Punched dent

SAFLP based experimental result

1.5

x0

SAFLP based experiment

1.0

1μm y (μm)

0.5

x5

0

x5 x2

-0.5

x2

x6

-1.0

x6 -1.5

x8

-1.5

x8 -1.0

-0.5

0 0.5 x (μm)

1.0

1.5

(a) Tip localization experimental data based (b) Statistical analysis of tip localization experiments based on landmark observation on landmark observation Fig. 7.16 Localization experimental result based on landmark observation

as the control point in the coordinate system, and its coordinate is set as follows: (0.801 lm, −0.684 lm). The pressed pit of the tip is marked by the white coil in the figure. Figure 7.15a shows the direct movement of the tip from x0 to x8. The tip records its position at these points such as x0, x8 and xd_1 by indentation pit. In the image, the position of the marked pit in the white circle can be seen. A total of 50 experiments have been done in this experiment. 50 groups of experimental data are counted to obtain the statistical distribution of errors at the three positions during the direct movement of the tip.

7.2 AFM Tip Localization

173

Table 7.1 Tip position distribution in the tip localization experiment without landmark observation (Unit: lm)

x0 xd,1 x8

Simulation data ly lx

rx

ry

Experimental data lx ly

rx

ry

−1.245 −1.245 1.031

0.013 0.013 0.018

0.016 0.021 0.021

−1.333 −1.346 1.087

0.014 0.014 0.021

0.018 0.024 0.025

1.316 −1.381 −1.434

1.435 −1.324 −1.374

Table 7.2 Tip position distribution in the tip localization experiment with landmark observation (Unit: lm)

x0 x1 x2 x3,tp x3 x4 x5 x6,tp x6 x7 x8

Simulation data ly lx

rx

ry

−1.245 −1.245 0.239 0.802 1.220 1.255 0.804 0.771 0.764 0.761 1.101

0.013 0.013 0.016 0.006 0.006 0.006 0.008 0.008 0.008 0.008 0.009

0.016 0.020 0.020 0.020 0.020 0.020 0.020 0.006 0.006 0.008 0.008

1.316 −0.645 −0.700 −0.701 −0.701 −0.407 −0.371 −0.682 −0.985 −1.431 −1.471

Experimental data lx ly

rx

ry

−1.321

1.472

0.015

0.017

−0.016

−0.603

0.017

0.019

0.790

0.005

0.781

0.009

1.174

−1.499

0.009

0.008

In Fig. 7.16, white circle shows the experimental result of location based on landmark observation. Figure 7.16a shows that the tip moves from x0 to x8 by using the landmark. The tip records its position at x0, x8, and the location of the pit in the white circle can be seen in the image. 50 groups of experimental data are counted and the error statistical distribution of the tip at these positions is obtained. Table 7.1 shows that the tip moves directly from x0 to x8 over a long distance of xd_1, and the variance of its position error increases to about 20 nm. Table 7.2 shows that the tip moves from x0 to x8 by observing the nanoparticle, and the variance of its position error is reduced to about 10 nm. The simulation result is very close to the experimental results, which illustrates the rationality of model parameter calibration and the effectiveness of the algorithm. By comparing the experimental results of the two methods, it is further verified that the uncertainty of tip localization can be reduced through landmark observation. The distribution of the tip at x8 is enlarged and shown in Figs. 7.17 and 7.18, and the fitting results of the Gaussian curve in the x and y directions are also shown in the Figs. 7.17 and 7.18 to illustrate the effectiveness and feasibility of the method. In Figs. 7.17 and 7.18a, b are enlarged to show the statistical results of the tip at position x0 and x8; (c) and (d) are the fitting curves of the tip at x0 in the x and y directions; (e) and (f) are the fitting curves of the tip at x8 in the x and y directions.

7 AFM-Based Nano-manipulation …

174

Sample number 12

Sample number

Sample number

10 12 8

4 0

12

8

8

6

4 4

0

2

(b) Experimental data statistics of the target position using direct motion Probability

Probability

(a) Experimental data statistics of the initial position using direct motion 0.4 0.2 0 -1.46 -1.42 -1.38 -1.34 -1.30 -1.26 xx0 (μm)

0.2 0 -0.98

-1.02

-1.06 -1.10 -1.14 -1.18 xx8 (μm)

0.2 -1.36 -1.40 -1.44 -1.48 -1.52 yx0 (μm)

(d) y-direction statistics of the initial position Probability

Probability

0.4

0.4

0 -1.32

(c) x-direction statistics of the initial position

0

0.4 0.2

0

-1.48

-1.44

-1.40 -1.36 -1.32 yx8 (μm)

(e) x-direction statistics of the target position (f) y-direction statistics of the target position

Fig. 7.17 Gauss fitting results at x0 and x8 based on the direct motion of tip from x0 to x8

7.2.3

Accuracy Improvement of Tip Localization

Stochastic approach based landmark observation can localize the tip in real time in task space. The localization accuracy is related to the accuracy of observation model and motion model. Therefore, improving the accuracy of observation model and motion model can further improve the localization accuracy of the tip. The accuracy of observation model can be improved by improving the accuracy of map and local scan of landmarks. Figure 7.19 shows that after scanning the map with high resolution, the map accuracy of AFM is less than 1 nm. Under the open-loop condition, the PI model parameter calibration method is used to calibrate the linear motion of the tip in a small range of 1 lm. The results are shown in Fig. 7.20. Based on the tip motion model and observation model, carbon nanotubes are used as the landmark to localize the tip in the vertical direction, as shown in Fig. 7.21. The specific experimental process is as follows: right along the direction of the nanotube, observe the nanotube center from top to bottom at 1–6 positions, and punch a pit at x3 position below the nanotube. After the tip observes the center of the nanotube in the vertical direction at Pos_1 and punches pit below it, it is shown that the indentation process at other locations (Pos_2 ! Pos_6) is similar to that at Pos_1 localization. Figure 7.22a shows the simulation and experimental results of the algorithm using carbon nanotubes as a landmark for localization

7.2 AFM Tip Localization

175 Sample number 12 Sample number

Sample number

10 12

8 4 0

12

8

8

6

4

4

0

2 0

(b) Experimental data statistics of the target position using landmark observation

0.2 0 -1.40 -1.36 -1.32 -1.28 -1.24 -1.20 xx1 (μm)

Probability

Probability

(a) Experimental data statistics of the initial position using landmark observation

Probability

Probability

0.2 0 1.10

1.14

1.18 1.22 xx8 (μm)

1.26

0.2 0 1.38

1.42

1.46 1.50 yx1 (μm)

1.58

1.54

(d) y-direction statistics of the initial position

(c) x-direction statistics of the initial position 0.4

0.4

1.30

(e) x-direction statistics of the target position

0.4 0.2 0 -1.60 -1.56 -1.52 -1.48 -1.44 -1.40 yx8 (μm)

(f) y-direction statistics of the target position

Fig. 7.18 Gauss fitting results at x0 and x8 based on landmark observation of the tip movement from x0 to x8

500nm

500nm

500nm

500nm

From top to bottom

From bottom to top

From top to bottom

From bottom to top

t = 20:20

t = 20:29

t = 20:38

t = 20:47

120

100

100

80

μ = 0 nm σ =0.3 nm

60

40 20 0

-3.0

-1.0 -2.0 x (nm)

0

(b) The position distribution of the nanoparticle center in the x-direction

80

60

400

Sample number

120 Sample number

Sample number

(a) High resolution scanning of the map with 512 × 512 pixels within 2 μm × 2 μm to obtain a highprecision position of the nanoparticle center

μ = 0 nm σ = 0.5 nm

40 20 0 -2.0

-1.0

0 y (nm)

1.0

2.0

l0 l1

300 200

μ = 0 nm σ = 1.1 nm

100 0

-3.0 -2.0 -1.0

0

1.0

2.0

x (nm)

(c) The position distribution of (d) Calculating the position distribution of the the nanoparticle center in the nanoparticle center caused by local scan of y-direction different regions of the nanoparticles

Fig. 7.19 Obtain an AFM scanning map with its accuracy of less than 1 nm

7 AFM-Based Nano-manipulation …

176 100

10.0

Y = ax + b

Y = ax + b

80

8.0

Output displacement covariance (σ2 )

a=1.4480 b=0.6423 y (nm)

60

40

20

0

a=0.0701 b=0.4345

6.0

4.0

2.0

0

20

0

60

40

x (nm) (a) Relationship between the expected distance of and the experimental distance with ingredient of 3 groups data

0

20

40

60

x (nm) (b) Tip position error increases as the motion distance increases

Fig. 7.20 Parameter calibration method of PI model

The tip stops at the initial position x0 after scanning an image

Punch pits

Pos_1

Pos_2

Ly

Pos_3 Pos_4

Pos_6 Pos_5

Scan 500nm up to position x1 and estimate the distribution of the tip at position x1 by observing the center of the nanotube

200nm

0.6

Scan down to the x2 position at 50 nm below the nanotube, then estimate the distribution of the tip at x2 by observing the nanotube center

Probe position distribution x1

0.4

x2,kp x2

0.2

y (μm)

Positioning : 0

After the tip is estimated at position x2 , it is moved to the position of 130 nm below the nanotube to punch the pit.

x'0

x3

x0

-0.2

Move horizontally to position x'0 , repeat the above procedure, and punch the pit at position Pos_2 →Pos_6 .

-0.4

-0.6

-0.6

-0.4

-0.2

0

x (μm)

0.2

0.4

0.6

(a) Simulation and experimental results of positioning and punching pits under the nanotube as landmark

(a) Experimental flow chart of positioning and punching pits under the nanotube as landmark

Fig. 7.21 Localization experiment using the carbon nanotube as the landmark

7.2 AFM Tip Localization

177

Fig. 7.22 Fitting curve of tip localization experimental result

1.00

μ =129.6 σ = 1.5

Probability

0.75 0.50 0.25 0.00

127.0

129.0 131.0 x (nm)

below (130 nm from the center of the carbon nanotubes in the vertical direction). Table 7.3 lists the distance distribution of the tip from the center of carbon nanotubes after localization. Figure 7.22 is a curve fitting the distance between the tip and the center of carbon nanotubes. The localization accuracy is 1.5 nm. Table 7.4 shows the simulation result of the tip localization experiment, and Table 7.5 shows the position of the observed carbon nanotube centers in the y-direction.

Table 7.3 The distance between the pit position and the center of the carbon nanotube in the vertical direction (Unit: nm) Distance

d1

d2

d3

d4

d5

d6

131.1

129.5

129.3

129.7

128.1

132.2

Table 7.4 Tip position distribution (Unit: nm)

0 x1 x2,kp x2 x3

Pos_1 uy

ry

Pos_2 uy

ry

Pos_3 uy

ry

Pos_4 uy

ry

Pos_5 uy

ry

Pos_6 uy

ry

0.0 507.4 251.5 208.2 122.0

10.0 3.7 1.2 1.9 2.8

122.0 612.8 231.2 169.7 101.4

2.8 4.4 1.2 2.0 2.8

101.4 588.1 185.2 118.2 54.6

2.8 4.5 1.2 2.1 2.8

54.6 543.3 135.2 70.6 4.8

2.8 4.6 1.2 2.1 2.8

4.8 507.4 114.1 65.1 −15.7

2.8 4.5 1.2 1.9 2.8

−15.7 496.1 143.0 94.1 13.5

2.8 4.2 1.2 1.7 2.8

Table 7.5 The position of the center of the observed nanotube in the y-direction (Unit: nm) Ly

Pos_1

Pos_2

Pos_3

Pos_4

Pos_5

Pos_6

252.0

231.4

184.6

134.8

114.3

143.6

7 AFM-Based Nano-manipulation …

178

7.3

AFM Nano-manipulation

7.3.1

Virtual Nano-hand Nano-manipulation

According to the simulation result of virtual nano-hand manipulation in Chap. 6, this chapter will validate that manipulation strategy can perform stable nano-manipulation through the corresponding experimental results. The experimental verification scheme is to stop the tip under the nanoparticle and push the nanoparticle along the Zigzag path in the vertical direction for stable manipulation. After the push manipulation is completed, the image is rescanned and the position of nanoparticle before and after manipulation are compared to verify the operation effect. In order to accurately compare the scanned images of nanoparticles before and after manipulation, two image control points need to be set under the particles by tip indentation. The experimental scheme can be carried out in the following steps: (1) The tip first stops at the center of the scanned image. Because of the large error of the initial position, the tip can scan the nanoparticles locally along the path of the image. After localizing the tip position below, the pitting operation is performed to set the control points of the scanned image. (2) The initial map of the nanoparticles (including image control points) is obtained by rescanning the image. Then the tip is first stopped in the image center. Then the tip moves to the initial position S of pushing the tip under the nanoparticle by using landmark observation, as shown in Fig. 7.23. The tip is initially docked in the scanning area center, and moves to the starting point below the nanoparticle through landmark observation, which is ready to push the nanoparticle. 0.5

0.5

Image center

Image center 0

y (μm)

y (μm)

0

Nanoparticle

-0.5

-1.0

Manipulation -0.5

Nanoparticle

-1.0

S Manipulation for starting point

Image control point -1.5 -0.5

0

0.5

x (μm)

1.0

1.5

-1.5 -0.5

0

0.5

x (μm)

1.0

1.5

Fig. 7.23 The tip moving to the initial position S of manipulation under the nanoparticle by using landmark observation

7.3 AFM Nano-manipulation

179 0.2 0

Nanoparticles

Pushing result Contact radius

y (μm)

-0.2 Tip localization based on local scan

-0.6 Image control points

1μm

(a) Designing a scheme of pushing nanoparticle Pushing on the left

Lleft_push

Linitial_h

Lleft_back

Scanning path

-0.4 Pushing path

Nanoparticles

-0.8

Lstep S -1.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 x (μm)

(b) Planning for tip pushing trajectory Pushing on the right

Preparation for next motion

Lright_push

Lh

Move

(e) One push on the right side

(c) One push on the left (d) Tip retraction side trajectory Push 13 steps

1μm

Image control points

(f) Experimental result

530 nm

20 nm

1μm

Image control points

(g) Comparison of experimental results before and after pushing

Fig. 7.24 Nano-manipulation experiment based on virtual nano-hand

(3) The tip begins to push progressively according to the strategy in the simulation. The first and second steps of the push manipulation are shown in Fig. 7.24c–e. After the tip moves to the starting point below the nanoparticle, the first step is completed. The tip will retreat to the right side and prepare the second step to make the push operation. The tip completes the third step. After 1, 3 steps, the image is rescanned to get the push result, as shown in Fig. 7.24f. Table 7.6 shows the parameter settings for pushing process. (4) By referring to the image control point, the scanned images before and after manipulation are overlapped (as shown in Fig. 7.24g). The pushing distance of the nanoparticle is obtained, which is 530 nm in vertical direction and 20 nm in horizontal direction. Comparing with the setting table in the program simulation, the pushing distance of 507 nm is obtained after considering the radius of

7 AFM-Based Nano-manipulation …

180 Table 7.6 Parameters of tip motion (unit: nm) 156

push

Lright 156

Fig. 7.25 Pushing 19 steps in the vertical direction according to the abovementioned nanoparticle pushing strategy

push

Lleft

back

120

Lright

Linitial

back

120

h

78

0.2

Tip localization based on local scan

-0.2 -0.4

Lh

Lstep

120

36

Contact radius

0 y (μm)

Lleft Value

Pushing result Uncertainty of tip position

Scan path

Pushing path

Nanoparticle

-0.6

1μm

Control points

-0.8 Step: 36 nm 0.2

(a) Scheme for pushing nanoparticle

0.4

S 0.6

0.8 1.0 x (μm)

1.2

1.4

(b) Path planning for tip manipulation 40 nm

804 nm

Push 19 steps

1μm

(c) Experimental results

Control points

1μm

Control points

(d) Comparison of experimental results before and after pushing

the tip 50 nm and the radius of nanoparticles about 100 nm, which is close to the simulation result. The experimental result of another group of experiment pushing 19 steps are shown in Fig. 7.25. These results verify that the proposed strategy can achieve stable operation. The error between simulation and experimental result mainly comes from the estimation of effective contact radius of tip, tip pushing model and displacement of tip motion, which need further study in the future.

7.3.2

Demonstration of AFM Nano-manipulation

On the basis of the abovementioned experimental result to verify the stable manipulation of the virtual nano-hand, the operation example of the virtual nano-hand (including rod-like and arc nano-hand) in Chap. 5 is simulated and demonstrated in the form of program simulation. The operation process of pushing

7.3 AFM Nano-manipulation

181

(b) Pushing nanotube with (c) Bridging nanotube between (a) Rodlike virtual nano-hand fixed pose using rodlike virtual electrodes using rodlike virtual for pushing nanotube nano-hand nano-hand

(d) Arc virtual nano-hand for pushing nanoparticle

(e) Pushing nanoparticle with fixed pose using a rc virtual nano-hand

(f) Arc virtual nano-hand for constructing nano-structure

Fig. 7.26 Program simulation demonstration of virtual nano-hand

the nanotube onto the electrode is shown in Fig. 7.26a–c. Figure 7.26d–f demonstrates that the arc clamp pushes the nanoparticle into a hexagonal shape. In AFM nano-manipulation, it is difficult to perform stable manipulation because of the uncertainty of relative position between the tip and the object being manipulated, the error of manipulation model and other uncertainties. Usually the nano-manipulation process is carried out according to the scanning-planningmanipulation-scanning mode, which is time-consuming and inefficient. When the AFM tip performs manipulation at multiple points concurrently, a virtual nano-hand can be used to improve the stability of manipulation. However, design of these action points of the tip and planning of the motion path have an important influence on the virtual nano-hand. Further research and discussion are needed to achieve stable and efficient nano-manipulation.

Index

A Adjacent matrix, 129 Ant colony algorithm path planning, 129, 144 Antioxidant mask, 5 Atomic Force Microscope (AFM) introduction, 33–36 milestones, 3 observation method, 9 B Baki sphere, 3 Bi-cubic and B-spline interpolation disadvantages, 59 formula, 57–59 Bilinear interpolation, introduction formula, 55, 56 Biosynthesis, 4 Blind modeling, 95 C Carbon Nanotubes (CNTs), 19, 22, 104 CCD camera, 158 Cellular organelles, 4 Controlled Auto-Regressive and Moving-Average (CARIMA) controlled auto-regressive and moving-average, 14 Creep model, 25, 107, 121, 122, 157 introduction, 122 Crystallography, 4 D DAQ card, 158 De Broglie wavelength, 11

Denoising threshold definition, 99 estimation, 101 Descartes coordinate system, 42 Dielectrophoresis nano-manipulation, 17 Dijkstra method path planning, 144 G Gauss distribution, 113, 130, 154, 165, 168 Generalized Predictive Control (GPC), 14 Global image reconstruction experimental result, 79 I Image geometric transformation, 53 Image interpolation method application, 51–53, 55–59 Image repairing, 52 Image stitching, 53 Images twisting, 52 Image zooming, 52 Integrated Circuit (IC), 5 K Kalman filter optimal estimation of tip position, 117–120 L LAN, 159 Landmark localization real-time nano-manipulation, 44, 45 strategy based on stochastic approach, 107–111

© Science Press and Springer Nature Singapore Pte Ltd. 2020 S. Yuan et al., AFM-Based Observation and Robotic Nano-manipulation, https://doi.org/10.1007/978-981-15-0508-9

183

184 Local scan analysis of landmark observation, 114 flow chart, 131 real-time nano-manipulation, 39–42 virtual nano-hand, 152 M Mapping, 84 Monocrystalline silicon, 3 Moore’s law, 5 Multiple-Walled carbon Nanotube (MWT), 3 N Nano-manipulation based on AFM, 21, 22 based on DEP, 19 based on self-assembly, 17 based on SEM, 20 based on tweezers, 17 Nano-observation main method, 9–16 Nano-rod, 108 Nanotechnology introduction, 1–8 Nearest neighbor interpolation definition, 55 Newton iteration introduction, 50 O Offset vector calculation, 66–70 definition, 63–65 OpenGL, 159 Optical tweezers nano-manipulation, 17 P Path planning ant colony algorithm, 144, 145, 147, 148 Dijkstra method, 143 for landmark observation, 130–135 multi-landmark environment, 141–143

Index probability distribution interval, 129 simple landmark environment, 136–140 Photosynthesis, 4 Piezoelectric ceramics, 35 PI model motion model, 121 Plasma waveguide, 5 Protein microarray, 5 PZT, 23 Q Quantum cellular automata control, 5 R Real-time feedback, 152 Root Mean Square (RMS), 36 S Scanning Probe Microscopy (SPM), 8 Scanning Tunneling Microscope (STM), 2 Self-assembly nano-manipulation, 17 Single-Walled carbon Nanotube (SWNT), 3 Stochastic approach AFM robotic nano-manipulation, 42–45 landmark localization strategy, 107–111 System thermal drift, 42, 49 T Target Oriented Pushing (TOP), 24 Taylor series, 50 Tip broadening effect, 23 Transmission Electron Microscopy (TEM), 8 V Van Der Waals force, 12, 34 Virtual nano-hand AFM nano-manipulation, 45 VR, 36 Z Zigzag path, 178