Threshold Logic [Reprint 2019 ed.] 9780520329560

Citation preview

threshold logic

SZE-TSEN HU

threshold logic

UNIVERSITY OF CALIFORNIA PRESS BERKELEY AND LOS ANGELES 1 9 6 5

University of California Press Berkeley and Los Angeles, California Cambridge University Press London, England © 1965 by The Regents of the University of California Library of Congress Catalog Card Number: 65-21982 Designed by Geoffrey Ashton Manufactured in the United States of America

PREFACE

Theoretical aspects of logical elements operating on the threshold principle were discussed by McCulloch and Pitts in 1943, and later by von Neumann with emphasis on reliability. These authors regarded such logical elements as mathematical models of neurons in living organisms. Since then, logical devices based on the threshold principle, such as paramétrons, magnetic core switches, and Esaki diode circuits, have been used for electronic digital computers. These devices are called threshold devices. As electronic computers are becoming more and more widely used, requirements on their speed increase. Therefore, logical devices capable of performing complex logical operations by themselves are desirable. Threshold devices may be used to satisfy this desire. In fact, since the switching function produced by a single threshold device is relatively complex in comparison to the usual "AND" and " O R " switches, a given switching function can in general be mechanized with fewer threshold devices. Thus the possibility exists for increases in speed and savings in equipment by proper use of threshold devices in computers. These two factors seem to have stimulated much interest in a class of switching functions which are variously referred to as threshold functions, majority decision functions, or linearly separable functions; and an appreciable number of papers and technical reports on this subject have appeared during the last six years. Most of these works deal with one or more of the following three areas:

vi

Preface

(1) Conditions which switching functions must s a t i s f y to be threshold functions. (2) Algorithms for determining whether or not a given switching function is a threshold function and for finding a (most economic) physical realization in case it is. (3) Heuristic methods for synthesizing networks built from threshold devices, mostly from 3-input majority decision gates. This book is written with a dual purpose: first, to serve as .a reference work on the subject, constituting the first comprehensive exposition of most of the contributions to date; second, to serve as a research presentation, containing the author's original work previously published only in LMSC (Lockheed Missiles and Space Company) technical reports (available to the public only through the Office of Technical Services, Department of Commerce, Washington 25, D. C.). In fact, most of the contributions in t h e final chapter h a v e not y e t appeared e v e n in t e c h n i c a l

notes. The first four chapters cover roughly the area (1) mentioned above; and the next three chapters are devoted mainly to the area (2). In the final chapter, I present an effective and mathematically rigorous finite process for constructing a minimal threshold network realizing an arbitrarily given switching function. This chapter falls within the area (3), where existing methods are all heuristic. Since most readers of this book will not be mathematicians, I have attempted to present mathematical theories in a somewhat leisurely manner, hoping that the non-mathematician might be able to follow without too much effort. In giving proofs, I include details that might ordinarily be omitted, preferring intelligibility over brevity. Mathematical proofs are mostly indicated by " P r o o / " at their beginning and by the symbol " I " at their end. Engineers who are interested only in the formulations and statements of the results may certainly omit all proof without experiencing much loss of continuity.

Preface

vii

The bibliography at the end of the volume includes almost all works on the subject available to me. In particular, it contains most of the reports and notes on threshold logic distributed by Lockheed Missiles and Space Company. References to this bibliography are cited briefly by names and number enclosed in brackets. Besides direct references cited in the text, I also list related works at the end of the sections, approximately in the order of publication dates. Cross references are given in the form (III; 8.1), where III stands for Chapter III and 8.1 for the numbering of the statement in the chapter. (The chapter number is omitted, of course, in intrachapter references.) A list of special symbols and abbreviations is given on pages xiii-xiv. In particular, I have used the abbreviation " i f f " for the phrase " i f and only i f . " The original works of mine included in this book were supported by the Lockheed Independent Research Fund while I was acting as consultant to the Electronic Science Laboratory, Lockheed Missiles and Space Company Sunnyvale, California. I am grateful to LMSC for granting permission to include these and other results in this volume. It is a further pleasure to acknowledge the financial support from the Air Force Office of Scientific Research during the summer of 1964 while I was preparing this manuscript. I am also indebted to M. A. Fischler, K. W. Henderson, Alfred Horn, R. L . Mattson,R. D. Merrill, R. I. Tanaka, M. Tannenbaum, and E. A. Whitmore for their suggestions, comments, and assistance, and especially to D. G. Willis for several extensive discussions that initiated this research work in 1959 and improved the results in my series of technical reports. Finally, it is a pleasure to thank the publisher and the compositor for their courtesy and cooperation. Sze-Tsen Hu University of California Los Angeles, California

CONTENTS

Special Symbols and Abbreviations

xiii

Chapter I: SWITCHING FUNCTIONS 1. 2. 3. 4. 5. 6. 7.

Introduction The n-Cube Qn Switching Functions Boolean Operations Set-Theoretic Operations Boolean Expressions Cubical Complexes

.

.

.

.

1 1 4 8 13 17 22

Chapter II: THRESHOLD FUNCTIONS 1. Introduction 2. 3. 4. 5. 6. 7. 8. 9. 10.

28

Linear Separability Tolerance Integral Separating Systems . Convex Hulls Boundary Points Connectedness Similarity Characterizing Parameters . Number of Threshold Functions

.

.

.

.

.

. .

.

.

28 32 37 39 42 48 50 52 55

Chapter III: MONOTONICITY AND ASUMMABILITY 1. Introduction 2. On Cubes 3. Restricted

58 n

in Q Switching

Functions

.

.

.

58 60

Contents

X

4. 5. 6. 7. 8. 9. Chapter IV: 1. 2. 3. 4. 5. 6. 7. 8. 9.

Complete Monotonicity V-Mono tonicity Unateness Property of 2-Monotonic Functions Numerical Tests Asummability

.

.

64 65 68 70 72 77

CLASSIFICATION Introduction Classes of Switching Functions Complementation of Variables Permutation of Variables . Invariance Theorem Canonical Switching Functions Canonical Partial Order mQn Regular Switching Functions Canonical Separating Systems

. . .

. .

. . .

. .

. .

. . . . .

. .

. .

83 83 86 91 94 97 99 103 106

Chapter V: ARITHMETIC INVARIANTS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

Introduction Connectivity Numbers Numbers of Connecting Edges Dimension Higher Dimensional Invariants Completeness and Existence Enumeration and Labeling . Computation of Parameters . Dual Switching Functions . Duality Theorems Extreme Points Computation of Maximal Points Computation of Minimal Points

Chapter VI: SYNTHESIS OF LINEAR 1. Introduction 2. General Method 3. The Willis Reduction

.

.

.

. .

. .

. .

. . .

. . .

. .

. . .

. .

. .

Ill Ill 113 114 117 122 126 130 133 137 143 144 149

SEPARABILITY 153 153 159

Contents

xi 4. 5. 6. 7. 8. 9. 10.

Chapter VII:

1. 2. 3. 4. 5. 6. 7. 8. 9. Chapter VIII:

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

Further Reduction Illustrative Examples Successive Elimination An Illustrative Example Successive Approximation Analytical Algorithm Other Synthesis Methods

. .

. .

. .

. .

.

.

.

.

167 172 176 185 189 194 199

Introduction Preliminaries Minimality Criteria Uniquely Minimal Separating Systems . The Minimization Problem . . . . Dual Simplex Method Dual Simplex Tableaux Solution of the Minimization Problem . Minimal Integral Separating Systems .

202 203 208 213 218 224 234 247 254

MINIMAL SEPARATING

MINIMAL THRESHOLD

SYSTEMS

.

. .

NETWORKS

Introduction Decompositions of Switching Functions Minimal Linear Decompositions . Simple Decompositions Reduction Theorem Total Degree of Repetition . . . Ashenhurst's Fundamental Theorem Cases with "Don't-Care" Points . Assignment of Values to *'s . . Singular Simple Decompositions . Discussion of Component Functions Minimization Algorithm Illustrative Example . . .

. .

.

. . . . . .

. . . . .

.

.

256 257 264 267 270 276 280 285 289 295 299 302 308

Bibliography

315

Index

335

SPECIAL SYMBOLS AND ABBREVIATIONS (The

number

which

indicates

the symbol

the page first

on

appears)

implies K»)» (if xl+x2 + -- + xn will be turned into a Boolean

by introducing

two binary operations, the conjunction together with a unary operation, the

algebra

A and the disjunction complementation.

v,

If,. Boolean

Operations

9

Let f and g be any two switching functions of n v a r i a b l e s . the conjunction of f and g, we mean the switching function

By

fAg:Qn*Q

h= of n variables defined by

h(x) = A [/(a;), ^(ar)] for every point x of the ra-cube Qn. Here, A on the right-hand side of the equation stands for the conjunction function A

:

Q2 - Q

with its truth table given in T a b l e 1-3-3. Since, for an arbitrary point x of Qn, if A g)(x) -1 iff fix) - 1 and g(x) - 1, the symbol f /\g ordinarily reads "f and g'\ On the other hand, one can e a s i l y verify that if A g)ix) = f{x)gix) for every xcQn. Here, fix)g(x) s t a n d s for the ordinary product of the integers fix) and g(x). B e c a u s e of this fact, the conjunction f A g will be called the product of the switching functions f and g and will be denoted simply by fg. Let f and g be any two switching functions of n v a r i a b l e s . By the disjunction of f and g, we mean the switching function h =

fvg:Qn*Q

of n variables defined by MaO = v [/(x), ^(a:)] for every point x of the n-cube Qn. Here, v on the right of the equation stands for the disjunction function v : Q2 - Q with its truth table given in T a b l e 1-3-3. Since, for an arbitrary point x of Qn, ifvg)(x) = 1 iff fix) = 1 or g(x) = 1, the symbol fvg ordinarily reads " / or g".

10

I: Switching

Traditionally, fvg g.

is called the sum or the logical

Functions sum of f and

However, one must observe that, in general, ifvg)(x)

= f{x) + g(x),

ix£

Qn),

fails to be true even in the s e n s e of modulo 2. the disjunction fvg

B e c a u s e of this,

will simply be c a l l e d the join of the switch-

ing functions f and g throughout this book, s o as to avoid possible ambiguity. By the definitions of the products and the joins of switching functions, one can e a s i l y verify the following statements ( 4 . 1 ) (4.5). (4.1)

T H E COMMUTATIVE LAWS: For any two switching

tions f and g of n variables,

(4.2)

we

func-

have

fg = g f ,

(4.1a)

/ v g = g v f.

(4.1b)

T H E ASSOCIATIVE LAWS: For any three

tions f, g, and h of n variables,

we

switching

func-

have

(,fg)h = f{gh),

(4.2a)

(fvg)vh = fv{gvh).

(4.2b)

B e c a u s e of ( 4 . 2 ) , the product fgh and the join fvgvh

are well

defined. (4.3) functions

THE

DISTRIBUTIVE

LAWS:

f, g, and h of n variables,

For

any

three

fig v A ) = / y v fh, f v gh = ( / v g)(f v h). (4.4)

T H E IDEMPOTENT LAWS:

of n variables,

switching

we have

For any switching

(4.3a) (4.3b) function

f

we have ff = / ,

(4.4a)

/ v / = f.

(4.4b)

J/,. Boolean (4.5)

THE

Operations LAWS

11 OF

FUNCTIONS 0 AND 1: ables,

OPERATION

WITH

For any switching

THE

CONSTANT

function

f of n vari-

we have 0v/=/,

(4.5a)

1/=/,

(4.5b)

Of = 0,

(4.5c)

1 v f = 1.

(4.5d)

Next, we will introduce the unary operation in $ as follows: Let f be an arbitrarily given switching function of n variables. By the complement

of f, we mean the switching function f' '• Qn

Q

of n variables defined by /'(«) = c[f{x)] for every point x of the n-cube Qn. Here, c stands for the complementation function c : Q Q defined in the previous section.

Thus, we have

f'(x) = 1 - f(x) n

for every point xcQ One can e a s i l y verify the following statements (4.6)-(4.8). (4.6)

T H E LAWS O F C O M P L E M E N T A R I T Y :

ing function f of n variables,

(4.7)

For

switch-

we have ff'= 0,

(4.6a)

/ v / ' = 1.

(4.6b)

DE MORGAN'S LAWS: For any two switching

and g of n variables,

any

functions

f

we have (/«7>'=/'v

c)xl + "' +cnxn

bn and B such have

if fix) = 1, B,

if fix) = 0.

< C, if fix) = 1,

clxi + ~- + cnxn^C,

if fix) = 0.

(v) There exist real numbers d , •••,. dn and D such that, for an arbitrary point x = xx ••• xn of Qn, we have d1x1 + -- + dnxn>

D, if fix) = 1,

¿jiij + ••• +dnxn

^ D, if fix) = 0.

Proof. (i)=»(ii). Assume that f is linearly separable. by definition, f admits a strict separating system iwr

wn;

Then,

T).

Take A = T and ai = wi for every i = 1, n. Then the condition (ii) holds. (ii) =$> (iii). Assume that / satisfies the condition (ii). Take B = - A and bi = -ai for every i = 1, n. Then we obtain (iii) by multiplying the inequalities in (ii) by - 1.

2. Linear Separability

31

(iii) =*>(iv). Assume that f s a t i s f i e s t h e condition (iii). Consider the off-set F'-= / - 1 ( 0 ) of f . Then, for each point x = xx '"Xn in F', we have p(x) = 6 i +- + bnxn-B > 0. Define a positive real number r as follows: If F' is empty, we s e t r = 1; otherwise, we s e t r to be the s m a l l e s t of the real numbers p(x) for all x€F'. Then we obtain (iv) by taking C = B + r and ci = bi for every i = 1, n. (iv) (v). Assume that f s a t i s f i e s the condition (iv). T a k e D = -C and di = -ci for every i = 1, n. Then we obtain (v) by multiplying the inequalities in (iv) by - 1. (v) (i). Assume that f s a t i s f i e s the condition (v). Consider the on-set F = / _ 1 ( 1 ) of F. Then, for each point x = in F, we have q{x) = d j + + dnxn-D > 0. Define a positive real number s as follows: If F is empty, we s e t s = 1; otherwise, we s e t s to be the s m a l l e s t of the real numbers q(x) for all xzF. L e t T = D + %s and wi = di for every i = 1, n. Then (tc,, T) is clearly a strict separating system for f . Hence (i) holds. I For the s a k e of d e f i n i t e n e s s , the system ( a ^ an\ A) of n + 1 real numbers in the condition (ii) will be called a separating sys~ tern for the linearly separable switching function f : Qn •* Q, with °i» * " ' an a s weiffhts and A as threshold. Hence, every s t r i c t separating system for / is always a separating system for f but the converse is not always true. COROLLARY 2.3. If a switching function f : Qn •* Q is ly separable, so is its complement f': Qn -» Q. Proof. Since / is linearly system (to lf wn; T). Then its complement / ' with C = T Hence / ' i s linearly separable.

linear-

s e p a r a b l e , it admits a separating the condition (iv) in (2.2) holds for and ci = wi for every i = 1, n. I

II:

32

Threshold

Functions

Hereafter, linearly separable switching functions will be called threshold,

functions.

RELATED

Gilbert 1, Karnaugh 1, von

REFERENCES:

Neumann 1, McNaughton 1, Wigington 1, Muroga 1, Minnick 1, Cheney 1, Willis 1, Henderson 1, Paull-McCluskey 1, Winder 1, Hu 1, E l g o t 1, Gabelman 1, C o a t e s - L e w i s 1-2, Winder 4.

3.

Tolerance

In the realization of a threshold function by means of an electronic d e v i c e , the components used to fix the weights and the threshold computed for the threshold function cannot be completely accurate. Hence, in determining the required accuracy of these components, it is necessary to estimate the maximal percentage errors in the weights and the threshold which may be allowed without disturbing the function to be realized.

For this purpose,

it is quite obvious that we have to use the strict separating systems. Throughout the present section, l e t f

Qn

Q

be an arbitrarily given threshold function of n variables and let S = (»,,

wn\ T)

be a given strict separating system for /.

For each point x =

x,1••• a71 in Q n , let w(x) = U>lXl + ••• + WrXn. Then it follows from the definition of a strict separating system that tafje) > T,

(if ftx) = 1),

vix)

(if fix) = 0).

,,

and T' instead of the

n

wn and T as weights and threshold.

Our main objective in the present section i s to find how small the real numbers |A,|, •••, |AJ and |A| should be so that the real numbers

•••, w'n and T' form a strict separating system for f.

Precisely, we are going to find a positive real number 8 such that if IA| < 8, then (to',

w T ' )

€ / - 1 ( l ) and

w'(x)


7".

w'(x)

Then we have w{x) g A and

therefore T - w ( x )

£

T - A

z

m.

As before, we have T ' - w ' ( x ) =

T ' -

2 i d ; * , 1 1 i= l

= (1 + A ) T - 2 i=i

(1 + \ ) w i x i

= [r-M)(aj)] + [ A r Since

T - w { x )

m

n 2

f-1

4

\,W,X.). 1

1

and

|Ar-£ x ^ l t=l

l|A||f|+ £ I X J I ^ I ^ . 1=1

3.

Tolerance

35 < ?? [|FI +

M it follows that T' -w'(x)

%

I^N

i=i

n > 0 and hence w'(x)

,| + — + |toJ. Hence we obtain m £ B-T

£ |f|+|fi|

1 |r| + |w)1| + -- + |u)n| = M.

This implies r(S) £ 1.1 For a given threshold function / of n variables and a given system (tflj, wn) of weights, the threshold T may be any real

36

II:

Threshold,

Functions

number between A and B; that i s , M a x M z ) | f(x) = Oj = A < T < B = Min M a ; ) | f(x) = l i . It turns out that the t o l e r a n c e r(ui ,

T) r e a c h e s i t s maxi-

wn;

mum when T is the arithmetic mean C = (A + B)/ 2 of the real numbers A and B.

P r e c i s e l y , we have the following

theorem. THEOREM 3 . 3 . T) for a threshold,

For every strict function

r(u>j, Wher°. C stands Proof.

f :

Qn

separating

-» Q of n variables,

wn\ T) £ rCMj,

for (A +

u>n; C)

B)/2.

Let s = |u),| +— + \wn\, m = Minfr-4, M= jn* =

system

B-TI,

ID+s, iB-A)/2,

M* = |C| + s. Then it follows immediately that m* = m + \C\$\T\ +

\C-T\, \C-T\.

Therefore, we obtain ; D - a < 2L±J£zI1 n' M = M+\ C-T\ m'

in+ic-n+a

(u> , we have

io n ;

Jp. Integral Separating

Systems

37

RELATED REFERENCES: Henderson 1, Winder 1, Hu 1, Muroga 4, Mays 1, Lewis-Coates 1, Coates-Lewis 3.

4. Integral Separating Systems Throughout the present section, let f '• Qn -* Q be any given threshold function of n variables. there is a strict separating system S =

By definition,

wn, T)

for/. Let 5 = r(S)denote the tolerance of the separating system S. If Aj, •••, An and A are real numbers such that |A|

+

+

+ «)*.

Since «>* +1 is an integer, this implies that w*+1 £ 2q. | RELATED REFERENCES: 1, Winder 4.

Hu 1, Elgot 1, Myhill-Kautz

5. Convex Hulls Let f : -> Q be an arbitrarily given switching function of n variables, and consider its on-set F = / - 1 ( 1 ) and its off-set F' = /"HO). By the power n(S) of a finite set S, we mean the number of points in S. Throughout the present section, we assume

40

II: Threshold p=n(F),

Functions

q=7r{F')

and therefore we have p+q = 2 " . L e t yx, of

yp denote the points of F and zv

zq the points

F'.

By the convex

hull of a s e t F in the Euclidean ra-space Rn,

mean the s m a l l e s t convex s e t H(F) of Rn

which contains

Hence, H(F) is the intersection of all convex s e t s in

Rn

we F.

contain-

ing F and c o n s i s t s of all points of Rn of the form x

where a x ,

aiyi + -- + apyp,

=

a p are non-negative real numbers s a t i s f y i n g flj + ••• + c

Similarly, the convex hull H(F') Rn of the form + Z = where

of F'

c o n s i s t s of all points of

+

& ? a r e non-negative real numbers s a t i s f y i n g + ••• + 6

T H E O R E M 5.1. The switching separable

iff the convex

is, H(F) and H(F') Proof:

= 1.

Necessity.

hulls

= 1.

function

f : Qn -» Q is

H[F) and H(F')

contain no common

linearly

are disjoint,

that

point.

Assume f to be linearly separable.

Then,

by definition, there e x i s t s a hyperplane n in the Euclidean ns p a c e Rn which strictly separates F from F'. half-spaces

of

Rn

are convex, the hyperplane n a l s o strictly

separates H(F) from HiF'Y, of 7r, H(F')

Since the open

that i s to s a y , H(F) l i e s on one side

l i e s on the other side of n, and n contains no point

of the union H{F) v

H{F'\

In particular, this implies that H(F) and H(F') Sufficiency.

Assume the convex hulls H(F)

are disjoint. and H(F')

to be

41

5. Convex Hulls

disjoint. As closed bounded s e t s in Rn, H{F) and H{F') are compact. Since H(F) and H{F') are two d i s j o i n t compact convex s e t s in the Euclidean n - s p a c e R n , it follows from a c l a s s i c a l separation theorem in the theory of convex s e t s [Klee 1, p. 457] that there exists a hyperplane n in Rn which strictly s e p a r a t e s H(F) from H(F'). Since FCH(F), F'CH(F'\ rr also strictly s e p a r a t e s F from F'. ly separable. I

T h i s proves that f is linear-

As a direct consequence of (5.1), we deduce the following nece s s a r y and s u f f i c i e n t condition of linear s e p a r a b i l i t y , discovered by [Chow 1] as a restatement of Ky F a n ' s condition for the cons i s t e n c y of a system of linear inequalities [Fan 1]. THEOREM 5.2. The switching function f : Qn -> Q of n variables is linearly separable i f f , for any 2 n non-negative real numbers cj, c )'• d l'. ••••' ag where p = n(F) and q = n(F'), the

relations

P ? 2 C< = 2 d.J t=i j=\

(i)

and P 2 ciV it=i

= 2 d1.a1. j=i

(ii)

imply ci = 0 and dj = 0 for all i = 1, p and j = 1, yt, yp denote the points of the on-set F = /-1(1) z denote the points of the off-set F' = / - 1 ( 0 ) .

q, where and

Proof: Necessity. Assume that the condition i s not s a t i s f i e d . Then there are 2 n non-negative real numbers c.*1< —i cp , dl ,

dq

not all zero, such that the relations (i) and (ii) hold. Then both s i d e s of the equality (i) are equal to a positive real number k. Dividing (ii) by k and setting

42

II: a

for every i = 1,

b

i =

j> and j = 1,

k

j /

3 a

iVi

i=l

of the convex hulls H(F)

Functions

q, we g e t a common point

P * =

d

j =

Threshold

=

3

J=1

and H{F').

J

Hence f is not linearly

separable according to (5.1). Sufficiency.

Assume that f is not linearly separable.

the convex hulls H(F)

and H(F')

there are non-negative real that

and

p

q

2 c. = 1,

1 d, = 1

j= 1

J

X =

1

p x =

are satisfied.

cp,

numbers c l t

i=1

2

t=i

Hence

dx,

such

(iii)

q c

i V i

,

;=i

d;2

7 1

•

(iv)

Clearly, ( i i i ) implies ( i ) and ( i v ) implies ( i i ) .

cause of ( i i i ) , the real numbers c l f •••, cp, all zero.

By (5.1),

have a common point x.

dx,

dg

Be-

cannot be

Therefore, the condition is not s a t i s f i e d . I

This approach to Chow's condition was described by [Gabelman 2 and Highleyman 1]. About the same time as [Chow i ] , a condition essentially similar to (5.2) was established in [ E l g o t 1].

T h i s is called asum-

mability and will be studied in the next chapter.

RELATED

REFERENCES:

Schoenberg

1, K l e e

Kirchberger

1, E l g o t

1, Chow

1, R a d e m a c h e r -

1, Gabelman 1,

Muroga 3, Highleyman 1, Gabelman 2, Gaston 1.

6. Boundary Points Throughout the present section, we are concerned with an arbitrarily given threshold function f : Qn

Q of n variables.

sider its on-set F = f~K 1) and its o f f - s e t F'

=

/-1(0).

Con-

6. Boundary

Points

43

A point b of the n-cube Qn i s s a i d to be a boundary point of f iff the switching function g : Qn -» Q d e f i n e d by

g X

_ lA®).

(if * 4 b),

"ll-flM.

(if®-ft),

is linearly s e p a r a b l e . The s e t of all boundary points of f will be c a l l e d t h e boundary set of f and d e n o t e d by d(f). Clearly,