CoNTEMPORARY

MATHEMATICS
63

Integral Geometry
AMS-IMS-SIAM
Summer Research Conference
on Integral Geometry
August 12-18, 1984
Bowdoin College, Brunswick, Maine

Robert L. Bryant
Sigurdur Helgason
R. 0. Wells, Jr.
Editors

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]

Recent Titles in This Series

184 L. A. Bokut', A. I. Kostrikin, and S. S. Kutateladze, Editors, Second International
Conference on Algebra, 1995
183 William C. Couuett, Marc-Oiivier Gebubrer, and Alan L. Schwartz, Editors,
Applications of hypergroups and related measure algebras, 199 5
182 Selman Akbulut, Editor, Real algebraic geometry and topology, 1995
181 Mila Cenkl and Haynes Miller, Editors, The Cech Centennial, 1995
180 David E. Keyes and Jincbao Xu, Editors, Domain decomposition methods in scientific
and engineering computing, 1994
179 Yoshiaki Maeda, Hideki Omoro, and Alan Weinstein, Editors, Symplectic geometry and
quantization, 1994
178 Ht!Ume Barcelo and Gil Kalai, Editors, Jerusalem Combinatorics '93, 1994
177 Simon Gindikin, Roe Goodman, Frederick P. Greenleaf, and Paul J. Sally, Jr., Editors,
Representation theory and analysis on homogeneous spaces, 1994
176 David Ballard, Foundational aspects of "non"standard mathematics, 1994
175 Paul J. Sally, Jr., Moshe Flato, James Lepowsky, Nicolai Reshetikhin, and Gregg J.
Zuckerman, Editors, Mathematical aspects of conformal and topological field theories
and quantum groups, 1994
174 Nancy Childress and John W. Jones, Editors, Arithmetic geometry, 1994
173 Robert Brooks, Carolyn Gordon, and Peter Perry, Editors, Geometry of the spectrum,
1994
172 Peter E. Kloeden and Kenneth J. Palmer, Editors, Chaotic numerics, 1994
171 Riidiger GObel, Paul Hill, and Wolfgang Liebert, Editors, Abelian group theory and
related topics, 1994
170 John K. Beem and Krishan L. Duggal, Editors, Differential geometry and mathematical
physics, 1994
169 William Abikofr, Joan S. Birman, and Kathryn Kuiken, Editors, The mathematical legacy
of Wilhelm Magnus, 1994
168 Gary L. Mullen and Peter Jau-Shyong Shiue, Editors, Finite fields: Theory, applications,
and algorithms, 1994
167 RobertS. Doran, Editor, C*-algebras: 1943-1993, 1994
I 66 George E. Andrews, David M. Bressoud, and L. Alayne Parson, Editors, The Rademacher
legacy to mathematics, 1994
165 Barry Mazur and Gleuu Stevens, Editors, p-adic monodromy and the Birch and
Swinnerton-Dyer conjecture, I 994
164 Cameron Gordon, Yoav Moriah, and Bronislaw Wajnryb, Editors, Geometric topology,
1994
163 Zhong-Ci Shi and Chung-Chun Yang, Editors, Computational mathematics in China,
1994
162 Ciro Ciliberto, E. Laurs Livorni, and Andrew J. Sommese, Editors, Classification of
algebraic varieties, 1994
161 Paul A. Schweitzer, S. J., Steven Hurder, Nathan Moreira dos Santos, and Jost! Luis
Arrant, Editors, Differential topology, foliations, and group actions, I994
160 Niky Kamran and Peter J. Olver, Editors, Lie algebras, cohomology, and new applications
to quantum mechanics, 1994
159 William J. Heinzer, Craig L. Huneke, and Judith D. Sally, Editors, Commutative algebra:
Syzygies, multiplicities, and birational algebra, 1994
158 Eric M. Friedlander and Mark E. Mahowald, Editors, Topology and representation
theory, 1994
(Continued in the back of this publication)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CoNTEMPORARY
MATHEMATICS
63

Integral Geometry
AMS-IMS-SIAM
Summer Research Conference
on Integral Geometry
August 12-18, 1984
Bowdoin College, Brunswick, Maine

Robert L. Bryant
Victor Guillemin
Sigurdur Helgason
R. 0. Wells, Jr.
Editors

American Mathematical Society

Licensed to AMS.
Providence. Rhode Island
License or copyright restrictions may apply to redistribution; see [Link]
 Editorial Board
Irwin Kra, W. H. Jaco
managing editor Gerald J. Janusz
Thomas F. Banchoff Jan Mycielski
M. Salah Baouendi Johannes C. C. Nitsche
Alan D. Weinstein

The AMS-MAA-SIAM Joint Summer Research Conference in the Mathematical

Sciences on Integral Geometry was held at Bowdoin College, Brunswick, Maine on
August 12-18, 1984 with support from the National Science Foundation, Grant
DMS-8218075.
1991 Mathematics Subject Classification. Primary 53C65;
Secondary 53C35, 52A22, 60D05, 43A85.

Library of Congress Cataloging-in-Publication Data

AMS-IMS-SIAM Joint Summer Research Conference in the Mathematical Sciences on Integral
Geometry {1984: Bowdoin College) Integral Geometry.
(Contemporary mathematics, ISSN 0271-4132; v. 63)
Bibliography: p.
1. Geometry, Integral-Congresses. I. Bryant, Robert L. II. American Mathematical Soci-
ety. III. Institute of Mathematical Statistics. IV. Society for Industrial and Applied Mathe-
matics. V. Title. VI. Series: Contemporary mathematics (American Mathematical Society);
v. 63.
QA649.A49 1984 516.3'62 86-28902
ISBN 0-8218-5071-7 (alk. paper)

Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting
for them, are permitted to make fair use of the material, such as to copy an article for use
in teaching or research. Permission is granted to quote brief passages from this publication in
reviews, provided the customary acknowledgment of the source is given.
Republication, systematic copying, or multiple reproduction of any material in this publication
(including abstracts) is permitted only under license from the American Mathematical Society.
Requests for such permission should be addressed to the Manager of Editorial Services, American
Mathematical Society, P.O. Box 6248, Providence, Rhode Island 02940-6248. Requests can also
be made by e-mail to reprint-permission«<math. ams. org.
The appearance of the code on the first page of an article in this publication (including ab-
stracts) indicates the copyright owner's consent for copying beyond that permitted by Sections
107 or 108 of the U.S. Copyright Law, provided that the fee of $1.00 plus $.25 per page for each
copy be paid directly to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers,
Massachusetts 01923. This consent does not extend to other kinds of copying, such as copying for
general distribution, for advertising or promotional purposes, for creating new collective works, or
for resale.

© Copyright 1987 by the American Mathematical Society. All rights reserved.

All rights reserved except those granted to the United States Government.
Printed in the United States of America.
This volume was printed from author prepared copy.
§ The paper used in this book is acid-free and falls within the guidelines
established to ensure permanence and durability.
0 Printed on recycled paper.
10 9 8 7 6 54 3 2 00 99 98 97 96 95
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 TABLE OF CONTENTS

Preface vii

Spectral synthesis on symmetric spaces

By Carlos A. Berenstein 1

The finite Radon transform

By Ethan D. Balker 27

Hyperfunctions in representation theory and mathematical

phys~cs

By Edward G. Dunne 51

The exponential Radon transform

By David V. Finch and Alexander Hertle 67

Integral geometry as geometry and as analysis

By S. G. Gindikin 75

Euclidean Radon transforms: ranges and restrictions

By Eric L. Grinberg 109

Perspectives in integral geometry

By Victor Guillemin 135

Some results on Radon transforms, Huygen's principle and

x-ray transforms
By Sigurdur Helgason 151
Classical integral geometry in Riemannian homogeneous spaces
By Ralph Howard 179
Differential operators and Cartan motion groups
By Kenneth D. Johnson 205

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 vi TABLE OF CONTENTS

An L2 -cohomology analogue of the Penrose transform for the

oscillator representation
By Lisa A. Mantini 221
Injectivity of rotation invariant Radon transforms on complex
hyperplanes in en
By Eric Todd Quinto 245
On overdetermined systems associated with integral geometry
transforms in the real projective space
By Radu RO§U 261
The Pompeiu problem in exterior domains in symmetric spaces
By Mehrdad Shahshahani and Alladi Sitaram 267
Curvature integrals and Chern classes of singular varieties
By Theodore Shifrin 279
Hypothesis testing in integral geometry: guessing the shape
of a plane domain
By Peter Waksman 299
Non-linear integral transforms
By Richard S. Ward 307

Integral geometry and twistor theory

By R. 0. Wells, Jr. 317
Some inverse problems of potential theory
By Lawrence Zalcman 337

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PREFACE

In the summer of 1984 a one-week conference on Integral Geom-

etry was held at Bowdoin College in Brunswick, Maine. It was one
of a series of conferences organized under the auspices of the
AMS-SIAM-IMS Summer Research Conference Series. The organizing
committee consisted of Robert Bryant (chairman), Victor Guillemin,
Sigurdur Helgason, and R. 0. Wells, Jr. The papers in this volume
are contributions by participants whose papers, after refereeing,
were considered suitable for inclusion. They range from purely
expository to quite technical papers, and as a whole represent a
good survey of contemporary work in this area.
The topic of Integral Geometry as such is not nearly so uni-
versally known as its counterpart, Differential Geometry, and most
of the major results in this area are part of the twentieth cen-
tury, while Differential Geometry has its very strong nineteenth
century tradition going back to Gauss and Riemann.
The conference brought together researchers from three differ-
ent areas of contemporary research in topics involving integral
geometry. The first area deals with new developments concerning·
the classical problems of computing geometric invariants by statis-
tical averaging procedures (to oversimplify greatly). The second
area relates to a circle of ideas concerning what is now called
the Radon transform, going back to the seminal work of Funk and
Radon around 1916-1917. The third area deals with integral-

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 viii PREFACE

geometric transforms which are now being used in the study of field
equations in mathematical physics, work which derives from the
twistor geometry of Penrose. Some of these overlapping areas
involve group-representation theoretic problems also. The papers
in this volume represent contributions from all of these areas.
It is the hope of the organizers of this conference (who are
simultaneously the editors of this volume) that bringing together
these three branches of integral geometry will be a stimulant for
further developments in these diverse areas. The questions asked
in each case are often quite different, but the basic integral
·geometric idea lies at the basis of all of them. Fundamentally,
integral geometry involves a transformation of information from
one space to a derived space, using an integration procedure (or
statistical averaging process) . The most famous example is the
well-known CAT-scan now used routinely in medical diagnosis, and
this is, of course, related to the original Radon transform involv-
ing integration of functions over lines in 3-space. Understanding
the range, and the inversion of such integral-geometric transforms,
is a major technical problem in the area, as a number of the papers
below give evidence for.
In the modern era, differential geometry is intertwined with
the global theory of manifolds, although its initial impetus was to
local problems. Hopefully, the developments in integral geometry
in its many guises can give equally deep insight into the structure
of both the global and local nature of manifolds (and their gener-
alizations), and the functions (and their generalizations) which
reside on these spaces.
We would like to express our gratitude to Russell Poley for
typing these papers. He passed away before the volume was com-

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PREFACE ix

pleted, and we want to express our warmest sympathy and thanks to

his widow, Anita Poley, who, under very trying circumstances,
helped us complete the volume.

Robert L. Bryant
Houston, Texas
Victor Guillemin
Cambridge, Hassachusetts
Sigurdur Helgason
Cambridge, Massachusetts
R. 0. Wells, Jr.
Houston, Texas
1 October 1986

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

SPECTRAL SYNTHESIS ON SYMMETRIC SPACES

CARLOS A. BERENSTEINl

1. INTRODUCTION. There is a problem in complex analysis that

leads naturally to the questions of spectral synthesis and spec-
tral analysis on symmetric spaces that we will consider below.
Let D be a region in en, consider the n differential forms
of degree (n, n-1) given by

A • • • A • • • 1\

If f 6 C(D) and n is a region n cc D with reasonably

smooth (but not necessarily connected) boundary r, it makes
sense to consider the integrals Ir fwk. If the function
f 6 H(D), i.e., f is holomorphic in D, then all these inte-
grals vanish; that is precisely Cauchy's theorem. Conversely, if
f 6 C(D), and all those integrals vanish for sufficiently many
regions n, say for all hypercubes of sides parallel to the
axes contained in D, then f 6 H(D) and we should call this
Morera's theorem. The easiest proof, say in the case of cubes,
consists in applying the following well-known argument: Regular-
ize f so we can assume f 6 Cm(D) and still preserve the
vanishing of the integrals. Then by Green's theorem,

0 = fr fwk = 0 n,k a~
fn Clzk d).

lThis work was supported by a grant frorr. the National Science

Foundation.
© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

1
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 2 CARLOS A. BERENSTEIN

where dA is the Lebesgue measure in R2 n and cn,k are some

non-zero constants. Hence f satisfies the Cauchy-Riemann
n
equations -Clf =
~;
L
a_f -
dzk 0 and therefore f 6 H(D).
1 Clzk
It has been the object of some recent research (see [26] for
extensive references) to try to find out what happens if we use
only a very small family of regions. The natural place to start
is to assume the domain D has a "large" group G of auto-
morphisms and restrict ourselves to a single topological ball (or
a finite family of such subregions of D) and its translations
under the group G. After considering the isotopy group K of a
point we can represent D as G/K and introduce a G-invariant
measure d~, mutually absolutely continuous to dX, and hence
we are reduced to deciding whether the system of equations

Jo(O)g(z)d~(z) = 0 Vo 6 G (1)

implies that g = 0. If that is the case we say 0 has the

Pompeiu property. (In the Morera theorem we have g = Clf h,
azk
where dA = hd~). We note that in this context we do not want to
impose any a priori growth conditions on f or g, like f 6
L~(D) or some other Sobolev space, although there are other
situations where this kind of condition would be perfectly natu-
ral. Before proceeding further let us point out a few natural
examples.

Example 1. D =c.
Here the group G = M(2) = group of rigid motions,
K = S0(2) = group of rotations, r is a piecewise - c1 Jordan
curve and if r is not real analytic then r has the Morera

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 3

property (that is, 0 = int r has the Pompeiu property as de-

fined above). If r is a circle it fails to have the Morera
property, and whether there are other real analytic curves which
fail to have this property is unknown (cf. [4]). We can also
easily replace e by en.

Example 2. D = 8 1 = {z 6 e:lzl < 1}.

Here the group G = group of Moebius transformations of
K = U(1) = S0(2), but the situation is the same as in Example 1,
[4]. Hence this property of non-real analytic Jordan curves is
valid for every simply connected D domain in c.
There are two natural generalizations of Example 2 to en.
n
Example 3. D = 8n = { z e en : Iz I < 1} , Iz I
~ I z j 12 =
j=1
We can also take for G the group of Moebius transforma-
tions of the unit ball [16] and then any connected piecewise -c 1
hypersurface r which fails to be a rea~analytic manifold will
have the Morera property [4]. From [10] or [6] it follows that a
sphere r will not have this property.

Example 4. The Siegel upper-half plane s n·

Since 81 is conformally equivalent to the upper half-plane
{z 6 e : Im z > 0}, another natural generalization of Example 2
is the set of n x n matrices z of the form:
Z =X+ iY, X, Y real symmetric matrices,
Y >> 0 (positive definite),
where n ~ 2. The group G of "Moebius transformations" is the
group Sp(n,R) of 2n x 2n matrices satisfying

~ ~) I . 0
t ( (
0 -I

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 4 CARLOS A. BERENSTEIN

(all the matrices A, B, C, D are square n x n matrices, I

is the n x n identity matrix) which acts on Sn by the trans-
formation law

Z + (AZ + B)(CZ + D)- 1

then Sp(n,R)/(S0(2n) n Sp(n,R)) is a complex manifold of

n(n+1)
dimension 2
(Cf. [16]). This is biholomorphic to the

bounded domain of type R11 in [19] of complex symmetric matrices

satisfying I - ZZ >> 0. In this case we do not know yet any
large class of hypersurfaces r which have the Morera property.
The difference between Examples 3 and 4 lies in the fact that the
unit ball Bn is a Riemannian symmetric space of rank 1, repre-
sentable as SU(n,1)/S(Un x u1 ) while Sn is a symmetric space
of rank n.
In fact, the Pompeiu problem in spaces of the form G/K, G
a unimodular Lie group, K a compact subgroup, was investigated
in [10] and shown to be related to the problem of spectral
analysis (for definitions see Section 2 below). The difference,
to our knowledge, between Examples 2 and 3 and Example 4 in the
final analysis is due to the fact that spectral analysis is valid
in c"'(R) [21 J and not in c"'(Rn), n ~ 2 [15].
In the present paper we will review the problem of spectral
analysis in a Riemannian globally symmetric space of the non-
compact type as well as the problem of synthesis. It has been
proved recently that for a rank one space the synthesis is valid
[24] and for spaces of rank ~ 2, spectral analysis fails [5].
We will give in Sections 2 and 3 the main ideas involved in these
two results. In Section 3 we will also point out the positive
results that are still possible in the case of arbitrary rank.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 5

In Section 4 we will review briefly the situation in the group

itself and mention some open problems.
I would like to thank the referee and Prof. s. Helgason for
their useful comments.

2. SPECTRAL ANALYSIS AND SYNTHESIS IN THE CASK OF RANK 1. As we

said in the introduction we will restrict ourselves to the study
of symmetric spaces, though the Pompeiu problem makes sense in a
more general context [10]. Following the notation and background
material from [16] we will assume now that G is a connected
non-compact semisimple Lie group with finite center and K is a
maximal compact subgroup of G. Then X = GIK is a (globally)
symmetric space of the non-compact type. As such it is a manifold
and we consider the usual spaces C~(X) of Cw-functions, its
dual E'(X) of distributions with compact support, and V(X) of
C~-functions with compact support, as well as their counterparts
in G. All of them carry their standard topologies. Recall that
any function f (or distribution) in X can be regarded as
being a right K-invariant function (or distribution) in G,
i.e. f(gk) = f(g) for all k e K. We will also need to consid-
er the spaces of K-invariant functions (or distributions), de-
noted c;, E (G),0 V0 (G), i.e. f e c;(G) if and only i f

Both V0 (G) and 0

E (G) are commutative algebras under the
convolution product

f*cj>(g) I G
f(h)cj>(h- 1 g)dh,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 6 CARLOS A. BERENSTEIN

the identity in 0
E (G) is the Dirac measure oe concentrated at
the identity element e of G.
There is a particular kind of function in C~(G) that plays
the role of the exponentials, the spherical functions ~A. They
allow us to introduce the spherical Fourier transform in v0 (G)
(and E•(G)) by the formula

(Ff)(A) I f(g)~A(g
-1
)dg, (2)
G

the parameter A varies in an n-dimensional real vector space

A*, n being the rank of G/K (also called the real rank of
G). This function has a holomorphic extension to the complexifi-
cation of A*, "c* (identified to Cn), and satisfies the fol-
lowing symmetry condition:
There is a finite group W (Weyl group) of orthogonal
transformations in A* generated by reflections such that if
o e w, f e V0 (G) (or f e E0(G)) then

Ff(oA) = Ff(A) VA 6 A*. (3)

The remarkable Paley-Wiener theorem of Helgason and Gangolli

states that these are the only compatibility conditions imposed
by the group G; otherwise Ff behaves like the Fourier trans-
form of a function in V(Rn) (or distribution in E'(Rn)). Let
us state it in the form that we will use it.

THEOREM [12]. The spherical Fourier transform is a topological

isomorphism between the algebra E0(G) and the algebra (E'(Rn))W
of those elements in E'(Rn) which are invariant under the Weyl
group.
Recall that the classical Paley-Wiener theorem states that

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 7

{$ entire function in en jA > 0 such that

(4)
IH A.) I ~ I
AeA ImA. I (1 +I A. I ) A, VA. 6 en}.

Hence, the above theorem says that

In the case of rank X = 1, i.e. n = 1, the Weyl group

contains only two elements, multiplication by ±1, hence the
functions in (E'(R))W are just the even functions in the Paley-
Wiener class E'(R).
Figuring out which are the solutions to the system of equa-
tions (1) is a particular case of understanding the structure of
the closed subspaces V of c=(X) which are invariant under G,
i.e. if f 6 V then for any g 6 G, the function

( 5)

also belongs to V. When considering c=(x) s c=(G) then, this

condition is the same as V(G)*V s V or E'(G)*V s V.
If we introduce the average operator MK,

MKf(g) = I
K .
f(kg)dk,

it is clear that MKc=(X) = C~(G), and similar identities hold

for the other spaces considered. Then, given a G-invariant
subspace v of c=(X) as above we can consider

( 6)

Note that v0 s V.
We have that

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 8 CARLOS A. BERENSTEIN

and, furthermore, it is well known that (see e.g. [10])

VI {0} implies v0 I {0}. (7)

In effect, let 0 = eK and f I 0, f 6 V then for some g 6 G,

fg(O) I 0, and fg 6 V, But MKfg(O) = JK fg(kO)dk = fg(O) ! 0

shows v0 I { 0}. Now, if we denote, as usual,

VJ..
0 : = {T 6 E' (X) : T(f) = 0, Vf 6 v0 }

then

u := MK(Vt) is a closed ideal in 0

E (G) (8)

and

(9)

where

UJ.. : = { f 6 c
CD ( G) : T ( f) = 0' VT 6 U}.

Note that MK(UJ..) is just the annihilator of U when C~(G) is

considered as the dual of E0(G).
From the above it follows that the Pompeiu problem reduces
to deciding whether the associated invariant subspace V of solu-
tions is trivial or not. Hence, using (7) and (9) this is equiv-
alent to deciding whether U = E0(G) or not. By the Paley-
Wiener theorem, this is equivalent to finding out whether the
closed ideal FU coincides with (E'(Rn))W. The last one being
a space of entire functions it is natural to consider the set

spec v 0 := Z(FU) : = 0. 6 en FT(~) = 0 for all T 6 U},

called the "spectrum of v0 " as well as the "zero set of the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 9

ideal FU". Clearly, a necessary condition for V = {0} is

then that spec v 0 = 0. The interest lies in its converse,
since it would provide an "easy" way of checking whether
v = {0}.

SPECTRAL ANALYSIS PROBLEM. Does V ~ {0} imply spec V0 ~ 0?

In other words, if V ~ {0} does there exist a spherical func-

tion 'h. 6 V?
The last paraphrasing of the question is just a consequence
of the fact that spherical functions satisfy

and the way we have defined spec v 0 (if Ao 6 spec v 0 then

cp_f.. s V). Note that cpA 6 v is the same as cpA 6 vo, since cpA
0
is K-biinvariant. We will also write spec V for spec v 0 .
The spectral analysis question is a kind of Nullstellensatz
question. That is, given a closed ideal I, such that Z(I) = 0.
Does it follow that I = (E'(Rn))W (or whatever other algebra of
holomorphic functions we may be considering)? More generally we
could ask whether the variety Z(I) determines the ideal. We
know that even for polynomials it only determines the radical of
the ideal, so one needs to take into account multiplicities.
This is done by introducing a new closed ideal I 10 c

I 10 c: = {~ 6 (E'(Rn))w: for every A 6 en there is a neighbor-

hood w of A and a finite collection of functions
~, . . . . , ~i e I, w1 , ... , ljii 6 H(w) such that ~

(The reader can provide a shorter description of Iloc in terms

of sheaves.) In the case n = the variety Z(I) is a
discrete collection of points {Ak}. To each of them one can

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 10 CARLOS A. BERENSTEIN

associate a multiplicity mk, 0 < mk < ~. namely the smallest

multiplicity of >..k as zero of all ~ 6 I, and Iloc is just
the collection of even functions ~ in the Paley-Wiener class
such that

k=1,2, •••• ( 1 0)

What does condition (10) mean in terms of the spherical func-

tions? Recall that for T 6 U we have

so that for 0 ~ j < ~·

Using the closure of modules theorem as in [11] one can also

see for any n that the condition ~ 6 Lloc can be written in
terms of Taylor coefficients at each point of Z(I), hence the
corresponding combination of derivatives of sperical functions
will be in U~ (or what is the same MK<U~) since they are
K-biinvariant). This leads to the following two versions of the
spectral synthesis question.

SPHERICAL SPECTRAL SYNTHESIS PROBLEM. Let I = FU, does

I = Iloc? For instance for n = 1
this can be translated as:
dj
Is vo the closed space of the functions
- . ft..I>..=-A. •
d>..J k
0 ::ii j < mk' >..k e spec v0 ?
We will phrase the other version only for n = 1:

SPECTRAL SYNTHESIS PROBLEM. Let X have rank 1. Is V the

smallest closed G-invariant subspace generated by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 11

We let the reader phrase the spectral synthesis problem for

the case of arbitrary rank.
It is clear that the formulations of these three problems
given above correspond exactly to the classical formulation for
C~(Rn). In this case we have the following two results.

THEOREM [21]. The spectral synthesis holds for every closed

translation invariant subspace of C~(R).

THEOREM [15]. For every n ~ 2 there are six distributions

• • • J such that the spectral analysis fails for
the space V = {f 6 C~(Rn): JJ/f = 0, j = 1, ... , 6}.
As an immediate consequence of the first theorem one
obtains:

THEOREM [10]. Spectral analysis holds in C~(X) if rank

X .. 1.

THEOREM [2]. Spectral synthesis holds in C~(G) if real rank

G • 1. Equivalently, the spherical spectral synthesis holds in
C~(X) if rank X= 1.

The usefulness of the spectral analysis theorem in the case

of the Pompeiu probl~m is tied to the fact that it is easy to
identify the generators of the ideal U in (8) associated to a
family of sets n 1 , •.. ,nm via equation (1). Namely,
let X 1 , ••• ,Xm be the characteristic functions of these
sets considered as right K-invariant functions in G, recall
f(g) • f(g- 1 ), then the generators of U are the functions of
the form [10]

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 12 CARLOS A. BERENSTEIN

T € E' (X).

From this remark one can prove the following

THEOREM [6]. Let 0 be a relatively compact region in a symmet-

ric space X of rank 1, its boundary r piecewise-c 1 and
connected. Then n fails the Pompeiu property if and only if
ja > 0 and a non-trivial solution u of the overdetermined
boundary value problem for the Laplace-Beltrami operator

+ au 0 in n,
t:.uu (N )
{
=1 EJ!~o on r'
' an

Using well-known results on regularity of free boundary problems

this theorem explains Examples 1, 2, and 3 of the Introduction.
Leaving to the next section the discussion of the case rank
X ~ 2, we turn now to the spectral synthesis problem. We follow
the path suggested by [24].
Let us consider for A € C, the space

( 11 )

where fg has been defined in (5). Clearly AA is a G-in-

variant subspace of C~(X). To verify it is closed we need to
find out explicitly the multiplicative constant that appears in
the definition of AA, i.e. we have for every f € AA, g € G

fKf g (kx)dk = c(f,g)¢_A(x).

We write, as usual, X = 0, for the point eK in X. The

definition of cp_A in [16] shows cp_A(O) = 1 • hence

c(f,g) = f fg(k•O)dk = f f(gk)dk = f(g)

K K

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 13

since f is right K-invariant. It clearly follows that if

fn 6 AA, and fn converge to f in the C~(X) topology (in
particular uniformly over compacts), then one obtains tor every
g 6 G, X 6 X

=lim fn(g)$_A(x) f(g)$_A(x),

n

showing that AA is closed. We clearly have, recalling the

definition (6), spec AA = spec (AA)O = {A}. It is possible to
relate AA to VA, the G-inva·riant closed subspace of C~ (X)

generated by $_A(= $A, since rank X= 1), as follows: Let

cA be the eigenval~e ~f 6 associated to $A, 6$A = cA.A' then
AA is the eigenspace of 6 for the eigenvalue cA ([16] p.
439) which also shows that AA 2 VA and AA is closed.
G always acts irreducibly on VA, so [Theorem 2.1; 17] one
has

VA = AA if and only if e(A)e(->.) I 0, ( 1 2)

where the function rx(A) = e(A)-1e(-A)-1' called the Gamma

function of X, is defined in [17]. The main point is that the
exceptional set E where e(A)e(-A) = 0 is a discrete subset of
the imaginary axis. To see a simple example of a point in the
exceptional set let us recall the definition of the plane waves
eA,b [17]. These are the functions introduced to define the
Fourier transform in V(X) and they depend on two parameters
A6 .i\~ and b 6 B the plane wave eA, b is constant in every
geodesic ending at b. It is in fact defined as follows.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 14 CARLOS A. BERENSTEIN

Let G = KAN be the Iwasawa decomposition of G, given

g 6 G it can be written as

g = k • exp(H(g))•n, H(g) 6 A.

Let B = KIM, M the centralizer of A in K, hence H is

constant in the classes mod M, and if b = kM

e A, b (g • 0 ) =
e A(H(g-lkM))

For instance in the case of the unit disk B1 we have B unit

circle and

that is, a power of the Poisson kernel.

The Fourier transform of a function h 6 L1 (X) can be
defined as

h(A,b)

The relation between plane waves and spherical functions is what

interests us here.

~A(g) = f exp(iA-p)(H(gk))dk.
K

One can show [17, p. 94] that eiA-p,b 6 AA for every b 6 B.

For the value A = -ip we see then that ~A e 1 and
VA = C ~ AA.
One concludes then that the spectral synthesis, as stated
above, is not possible in general. The following converse, which
is a consequence of (12) and the spherical spectral synthesis
theorem of [2], can be found in [24]:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 15

THEOREM. Let V be a closed G-inva~iant subspace of X, ~ank

X= 1. If spec V n E = 0 then the spect~al synthesis p~ope~ty

holds fo~ V.

To deal with the gene~al case, we int~oduce a gene~alization

of the space Ax:

A
X,m
{f6C ... (X): fo~ all g 6 G, J f(gkx)dk
K
dm-1 ( 1 3)
6 span {cp_X (x), ••• , -m:1 Lx(x)}}.
dX
THEOREM [24]. Let v be as above, then V ~ closu~e of
iAx , spec V = {Xk}' mk thei~ multiplicities.
k'IIk
To finish this section let us point out one could have
defined spect~al analysis and synthesis in te~ms of plane waves.
It is not yet known whethe~, in the ~ank one case, the~e is
spect~al analysis in te~ms of plane waves.

3. RANK X ~ 2. As we pointed out in the p~evious section the

spect~al analysis fails fo~ ce~tain t~anslation inva~iant sub-
spaces of C... (Rn) when n ~ 2. In fact one can extend this type
of example to a~bit~a~y g~oups G, ~eal ~ank G ~ 2.

THEOREM [5]. If ~eal ~ank G ~ 2 (i.e. ~ank X~ 2), the~e

exist N functions ~ 1 • . . . • ~N in V 0 (G) such that if we

cons ide~

vo = {f 6 C~(G) : f*~·
J
~ 0, j = 1 ••••• N},

then spec vo = (2l while

vo "' { 0} • N can be taken equal to
2(/I(W)+1), whe~e II(W) is the o~de~ of the Weyl g~oup w of
G.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 16 CARLOS A. BERENSTEIN

This theorem is a consequence of the following:

THEOREM [5]. Let a 1 ,a 2 , .•• ,an be positive numbers (n ~ 2),

p(z) = lz,la1 + ••• + lznlan and

Hp(en) = {~ entire in e FA> 0 such that

l~(z)l SAeAp(z) Vz 6 en}.

Let W be a finite subgroup of U(n). Then there exists N

functions ~, • . . . • ~N 6 Hp(en) such that

{z 6 en : ~ 1 (z) ~N(z) = 0} = 0,

~j(oz) = ~j(z) for all a 6 W, j = 1, ••• ,N

and the closed ideal I generated by ~, •.••• ~N in HP(en)

is a proper ideal.

Though the first theorem shows the failure of spectral

analysis even in the class of K-biinvariant functions, one should
still be able to provide positive results for solutions of systems
of convolution equations, both in c0 (G) and C~(X), using the
corresponding spectral synthesis theorems in Rn from [7], [23],
the point being that if a system v1 , ,pN 6 E'(Rn) is "slow-
ly decreasing" in the sense described in [7] one has spectral
synthesis for the space of solutions of the system of convolution
equations. In fact, one has even a Fourier representation of all
the solutions. A typical example of an application of [7] is the
following result of Kuchment. To state it we need to recall the
concept of a K-finite function in c=(X) and of a generaliza-
tion of the plane-waves.

DEFINITION. f 6 c=(X) is K-finite if dim span {fk k 6 K} <

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 17

DEFINITION. Let 6 denote an irreducible representation of K

and 6. . ( k)
l,J
its matrix elements. The generalized plane-wave

fi,j, 6 (x,A) is defined for x 6 X, A 6 A~ by, x = g•O,

Kuchment [20] states the following theorem:

THEOREM. Let T 6 E0(G) whose Fourier transform FT is slowly

decreasing, then there is a locally finite covering {zt}t 6 L by
closed subsets of the variety Z = {A 6 en T(A)=O}, a
partition {La} of L into ftnite subsets and differential
operators at acting on the A-variable such that any K-finite
f S C (X}, 00
solution of the equation T*f = 0 has a representa-
tion of the form

• •6
f(x) - ~
· · ~ a ..•eL a
~ ~ Jr
zR;
atf 1. J. (x,A)d~i J.
• •u •
~ t(A)
l,J,u

for some Radon measures d~.

1
. ~ •
J 0 u, ..
supported in zt. The sum on
0

i,j,6 is finite, the series in a converges in the Cm(X)

topology and the integrands are solutions of the same equation as
f.
This theorem generalizes an older result of [2] which was
based on the work of Malgrange.

THEOREM [2]. Let T 6 E0, T f. 0, f e C~(G), T*f = 0, then f

is a limit of linear combinations of functions fA,a such that
T*fA,a - 0, fA,a are derivatives of the spherical functions fA
with respect to A.

It is in this spirit that one would like to approach the

question raised in Example 4. As pointed out in Section 2 the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 18 CARLOS A. BERENSTEIN

way to get a very general criterion for the validity of the

Morera property,for a hypersurface r, like that of r not
being real analytic depends on the equivalence between the failure
of the Pompeiu property for n = int r and the existence of
eigenvalues for the overdetermined Neumann problem (N). The proof
in [6] uses that rank X= 1. On the other hand it is shown
there that it is always true that the existence of an eigenvalue
for (N) implies that 0 fails the Pompeiu property.

PROBLEM 1. Show that the failure of the Pompeiu property for

an open, relatively compact set n with connected piecewise
c1 -boundary r in a symmetric space X implies the existence of
a positive eigenvalue for (N).

The same equation (N) shows that if n is a geodesic ball

in X, rank X= 1, then there are infinitely many eigenvalues
a, corresponding to those solutions u that depend only on the
distance to the center of n, since then (N) becomes a Sturm-
Liouville problem for an ordinary differential operator. On the
other hand it has been shown in [10] that if the ratio of the
radius of two balls, n1 , n2 avoids certain exceptional values
one has the Pompeiu property for the pair n1 ,n 2 . It was shown
in [8] for the case of balls in R2 that if the ratio satisfies
a slightly stronger condition then one can find a pair of radial
distributions of compact support v1 , v2 such that the Bezout
equation

is satisfied for the chartacteristic functions x1 , x1 of the

balls n1 , n2 • Clearly this condition implies the Pompeiu

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 19

property since U = E'(R 2 ) in this case. Similar statements

hold in the case of any symmetric space of rank 1.

PROBLEM 2. Let X be a symmetric space of rank n,

o1 , ••• ,On+ 1 geodesic balls, x 1 , •••• xn+ 1 .their respective

characteristic functions. Find fairly general conditions on the
radii that imply that the Bezout equation

has a solution for some right K-invariant distributions of com-

pact support in G.

As an example of the kind of argument we have in mind, let

us discuss the case of a complex group. In this case the spheri-
cal functions have a specially simple form ([18, Theorem 1.13]).
Writing G ~ KAK we have for a 6 A,

n(p) L(det o)eioA(log a)

.A(a•O) = -
n(A) L(det o)eiop(log a)'

where the summation extends to the Weyl group, and n(A) is the
polynomial IT
aSE
+A(H ).
a
Without attempting to explain the nota-
tion n(p) is a fixed polynomial factorized into distinct linear
factors, whose zeros are, generically, the only linear varieties
on which the numerator vanishes. Now let be the measure dk
in X supported on the sets r = Kaj for some a 1 , .•• ,an+ 1 e
A. we can compute easily Fxj' namely

So, the variety of common zeros of Fxj is just the variety of

common zeros of the exponential sums in the numerator, excluding

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 20 CARLOS A. BERENSTEIN

the zeros of n(A). By [3] and [14] the zeros of such an expo-
nential sum are asymptotically near the hyperplanes of equations

A(a(log a) - a'(log a)) ~ 0 a f. a' ,

where a is one of the aj. By [9] if we choose a 1 , ••• , an e A

so that if we pick n such hyperplanes, one corresponding to
each point aj, then they intersect only at the origin, we have
that x1 , ••• •Xn are (jointly) slowly decreasing. Generically
we can assume their common zeros {A 1 } are all simple. It
follows [9] that any K-biinvariant solution f of X *f -= • • •
1
:a:

xn *f = 0 can be written as

We note that, by the functional equation satisfied by the spheri-

cal functions we have

and the convergence properties and uniqueness of this series

representation shows that Xn+ff = 0 is only possible when
f = 0 . we have chosen
1f a n+l so that { Fx1 = Fxn+1 a 0}

- ~. This proves that the family of sets rj with their natu-

ral measures has the Pompeiu property. In fact the arguments in
[9] indicate that there are v 1 , ••• ,vn+1 e E'0 (G) solving
problem 2 in this case.
Before ending this section let us point out that if X is
one of the classical domains [19], and if we assume enough growth
conditions on the functions f 6 C~(X) we want to check for

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 21

holomorphy, then, any bounding hypersurface r can be used as a

test for the Morera property. Note that the domain Sn from
Example 4 is one of the classical domains.

THEOREM [1]. Let X be one of the classical domains in en,

assume that f e c1 <x> ' 1!_

az k
e L2 (X,dll) (recall dll is the
invariant measure). Then if r is smooth, int r = 0 cc X and
(with the notation of Section 1)

0, k = 1, ••• ,n, Vg 6 G

and f e H(x).
The point is that (modulo technical details) the situation
is like in L2 (Rn,dx): if 0! t is a bounded function of
compact support and ~ e L2 then t*w 0 implies ~ = 0 since

implies that w= 0 a.e. in Rn (t extends to a non-zero entire

function).
To end this section let us mention that in the same paper
Agranovskii gives a different criterion for holomorphy in
classical domain X which does not require growth conditions.
Let F be a compact subset such that P(F), the closure in L~

of the space of restrictions of polynomials, is properly

contained in C(F). Then if f e C(X) and fogjF e P(F) for
all g 6 G it follows f 6 H(X).

4. MEAN - PERIODIC FUNCTIONS ON THE GROUP. In the group G it

is natural to consider closed subspaces V of C~(G) which are
invariant both on the left and right for the action of G. The
mean-periodic functions f are those such that the minimal

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 22 CARLOS A. BERENSTEIN

biinvariant V such that f 6 V is distinct from {0} and

C~(G). In other words, there are two non-zero distributions
~. v 6 E'(G) such that

~*f = f*v = 0.

This is a definition from [13], where an expansion theorem for

mean-periodic functions in SL(2,R) is given. Regretfully the
proof has a gap which to our knowledge has never been fixed.
Very little else is known about such functions, except for the
fact that some of the standard properties of mean-periodicity
could be lost. For instance in Rn an integrable function
cannot be mean-periodic (this was used in the sketch of the proof
of the theorem of Agranovskii at the end of Section 3), while an
example of Sitaram [22] shows this is not the case in SL(2,R).

THEOREM [22]. Let G be a K-biinvariant Borel set in SL(2, R)

with finite positive measure. Then there exists a non-trivial

function f 6 L1 (SL(2, R)) such that

For the motion group M(2) Welt [25] has used the work of
[11] on the Pompeiu problem to study biinvariant subspaces V of
M(2). We think that the problem of mean-periodicity in groups
has been neglected for too long and merits looking into.
We end this paper mentioning a few open questions:

PROBLEMS. 1. Let G/K b.e a non-compact Riemannian symmetric

space of rank n, n 1 , ... ,nn+ 1 geodesic balls in G/K, find
fairly general conditions on their radii so that if f 6 C(G/K)
satisfies

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 23

J f(x)dx = 0 Vg 6 G, j = 1, .•• ,n+1

gOj
then f .. 0.

2. It it true that if 0 is a topological ball in a symmetric

space G/K of rank > and, if the boundary of 0 is not
real-analytic then for every f e C(G/K)

r f(x.)dx 0 Vg 6 G
Jgo
implies f & 0?

3. Give a correct proof of the expansion for mean periodic

functions in C~(SL(2, R)) suggested in Ehrenpreis-Mautner,
Trans. AMS 84 (1957), 1-55.

4-. Let rank G/K = 1. Does every G-invariant closed subspace of

C~(G/K) contain a plane wave?

BIBLIOGRAPHY
1. M. G. Agranovskii, "Tests for holomorphy in symmetric
regions," Siber. Math. J. 22 (1981), 171-179.
2. S. C. Bagchi and A. Sitaram, "Spherical mean-periodic
functions on semisimple Lie groups," Pacific J. Math. 84 (1979),
241-250.
3. C. A. Berenstein, "An estimate for the number of zeros
of analytic functions in n-dimensional cones," Advances in Com-
plex Function Theory, eds. W. E. Kirwan and L. Zalcman, Springer-
Verlag, 1976, 1-16.
4. C. A. Berenstein, "A test for holomorphy in the unit
ball of cn,n Proc. Amer. Math. Soc. 90 (1984), 88-89.
5. C. A. Berenstein and R. Gay, "Sur la [Link] spectrale
dans les espaces symmetriques," to appear in J. Analyse Math.
6. C. A. Berenstein and M. Shahshahani, "Harmonic analysis
and the Pompeiu problem," Amer. J. Math. 105 (1983), 1217-1229.
7. C. A. Berenstein and B. A. Taylor, "Interpolation
problems in en with applications to harmonic analysis," J.
Analyse Math. 18 (1980), 188-254.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 24 CARLOS A. BERENSTEIN

8. C. A. Berenstein and A. Yger, "Le probleme de la d•con-

volution," J. Funct. Anal. 54 ( 1983), 113-160.
9. C. A. Berenstein and A. Yger, "Ideals generated by
exponential~polynomials," to appear in Advances in Mathematics.
10. C. A. Berenstein and L. Zalcman, "The Pompeiu problem
in symmetric spaces," Comment. Math. Helvetici .2.2. (1980), 593-
621.
11. L. Brown, B. M. Schreiber and B. A. Taylor, "Spectral
synthesis and the Pompeiu problem," Ann. Inst. Fourier 23 (1973),
125-154.
12. M. Eguchi, M. Hashizume and K. Okamoto, "The Paley-
Wiener theor-em for distributions on symmetric spaces," Hiroshima
Math . J • 1 ( 1 973 ) , 10 9 -1 2 0 .
13. L. Ehrenpreis and F. I. Mautner, "Some properties of
the Fourier transorm on semisimple Lie groups II," Trans. Amer.
Math. Soc. 84 (1957), 1-55.
14. L. Gruman, "The zeros of functions of finite order in
cn,n Ann. Polonici Mathematic! 40 (1983), 161-177.
15. D. I. Gurevich, "Counterexamples to a problem of L.
Schwartz," Funct. Anal. Appl. 2 (1975), 116-120.
16. s. Helgason, Differential Geometry and Symmetric
Spaces, Academic Press, 1962.
17. S. Helgason, "A duality for symmetric spaces with
applications to group representations I," Advances in Math. 2
(1970), 1-154.
18. S. Helgason, "Analysis on Lie groups and homogeneous
spaces," CBSM .12!_ American Math. Soc., 1972.
19. L. K. Hua, "Harmonic analysis of functions of several
complex variables in the classical domains," Amer. Math. Soc.,
1963.
20. P. A. Kuchment, "Functions on symmetric spaces which
are periodic in the mean," Funct. Anal. Appl. 1.§. (1982), 213-214.
21. L. Schwartz, "Th,orie g,n,rale des functions moyenne-
periodiques," Ann. of Math. ~ (1947), 55-79.
22. A. Sitaram, "Some remarks on measures on non-compact
semisimple Lie groups," Pacific J. Math. 110 (1984), 429-434.
23. D. C. Struppa, "The fundamental principle for systems
of convolution equations, II Memoirs Amer. Math. Soc. /1273 n
(1983).
24. A. Wawrzynczyk, "Spectral analysis and synthesis on
symmetric spaces," Univ. Autonoma Metropolitana-Iztapalapa,
Mexico, 1984.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SPECTRAL SYNTHESIS ON SYMMETRIC SPACES 25

25. Y. Welt, "On Schwartz's theorem for the motion group,"

Ann. Inst. Fourier 30 (1980), 91-107.
26. L. Zalcman, "Offbeat integral geometry," Amer. Math.
Monthly 87 (1980), 161-175.
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF MARYLAND
COLLEGE PARK, MARYLAND 20742

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

THE FINITE RADON TRANSFORM

ETHAN D. BOLKER

1. INTRODUCTION. DEFINITIONS AND CANONICAL EXAMPLES. Several

years ago Victor G~illemin asked me to think about the Radon
transform on finite sets and to look for structure there which he
could use to motivate theorems about the classical Radon trans-
form and its relatives. This paper sketches some of what I
discovered, with substantial help from Victor, Andy Gleason and
Shlomo Sternberg. Our study of finite Radon transforms shed a
surprising amount of light on the continuous ones and revealed
connections to several areas of interest to combinatorists:
geometry over finite fields, the Kirkman schoolgirl problem, and
the representation theory of the symmetric group.
Let X be a finite set and Y a set of subsets of X. For
any finite set A write C(A) for the finite dimensional vector
space of complex valued functions on A (C is acceptable nota-
tion since all functions are continuous) and define the Radon
transform R : C(X) + C(Y) by

Rp(Y) - L p(x) a L p(x), (1)

x6y xSFY

where we have written FY for {xlxSy}.

These finite transforms are in principle more symmetrical
than the continuous ones: no analytic difficulties obstruct
@ 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page
27
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 28 ETHAN D. BOLKER

defining an adjoint to R. If we set Gx = {ylxSy} we can

identify x with Gx, view X as a set of subsets of Y and
define the transform S: C(Y) + C(X) by setting

S~(x) = L ~(y).
y6Gx

If we- equip each space C(A) with the inner product

L cp(a)~(a) (2)
aS A

then C(A) becomes a Hilbert space. The transform definitions

above then imply S = Rt. The symmetry would be even clearer
were we to mimic the double fibration view of the continuous
transforms, but we prefer this less formal, more intuitive
approach here.
We shall focus on three main classes of examples, each of

.•
which yields finite versions of the classical Radon transforms.
First, we let X be an n-element set and Y = ( ~J the set of
k-element subsets of X. We call the corresponding transform the
k-set transform. Second, we fix a finite field F Fq with q
elements, let X be affine n-space over F, and Y the set

[ xk] of k-planes in X. Finally, we redo the second example,

using projective rather than affine n-space. These are the
finite affine and projective k-plane transforms.

2. WHEN IS THE RADON TRANSFORM INJECTIVE? Although we shall soon

see why this question is much too hard in its most general form
there are natural combinatorial conditions which guarantee in-
jectivity in each of the three canonical examples and have
suggestive analytic analogues.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 29

For any set A write #A for the cardinality of A and

6A for the characteristic function of A. The delta function
6x is just 6{x}· Then it follows from (1) that for A c X

(3)

and hence too that

(4)

Thus the matrix of R relative to the natural coordinate systems

{6x}, {6Y} for C(X) and C(Y) is just the 0-1 matrix express-
ing the incidence relation. Th~ question "Is R injective?"
becomes "Does the rank of the matrix for R equal #X?" Since
this matrix is an arbitrary 0-1 matrix the question is [Link]
too general a context. We can say something interesting only by
demanding some regularity. The useful constraints turn out to be
conditions on how the sets Gx live together in Y. We suppose
first that the cardinality of Gx is independent of X:

II(G )
X
= a. for all X 6 X, (5)

and, second, that

( 6)

independent of x and x', as long as x! x'. We will often

refer to these as the B-conditions. A combinatorist would recog-
nize them as the assertion that Y is a block design on X.
There are many ways to prove that the B-conditions guarantee
the invertibility of R - see, for example, [Ryser] or [Kung].
We present one whose structure suggests an invertibility proof in
some continuous cases. Suppose the B-conditions are satisfied.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 30 ETHAN D. BOLKER

Then

SRox(x') = SoG (x')

X (by 4)

( 3, applied to S)
if X = x'
{: if X ,; x'

Hence

(7)

Where the second e on the right is a constant function on X.

Thus

( 8)

We can turn this into an inversion formula for R. Since an

arbitrary ~ 6 C(X) can be written

we see that

a~e SR~ - a~e L ~(x). (9)

X

This expresses ~ in terms of R~ and the total mass

v<~) = <~.ox) a L ~<x).

X

If we can compute v(~) in terms of R~ we can construct an

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 31

inversion formula by substituting in (9). Now applying (3) to S

=-
a. JJ(R~).

Hence

1 aJJ(R~)
( 1 0)
~*a.-a sR~- a.(a.-a)·

To see this inversion formula in operator terms set

a
b - a.(a.-a)

and let L be the operator mapping $ 6 C(Y) to the constant

function JJ($) in C(X). Then (10) says aS+ bL is a left
inverse for R. In particular, we have proved

THEOREM 1. When the B-conditions are satisfied, R is injective

and Equation (10) provides an inversion formula.

In order to apply this theorem to our three canonical exam-

.ples we begin a study of how Radon transforms interact with group
actions. Suppose a group H acts on X. That action extends to
actions on Y and on X x Y in the obvious way. Suppose
further that the action of H on X x Y preserves the incidence
relation x 6 y. If H acts transitively on X then given x
and x' 6 X we can find h 6 H with hx = x'. Then hGx- Gx'
which establishes a bijection between and Gx , , thus

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 32 ETHAN D. BOLKER

guaranteeing condition (5). If the action of H is doubly

transitive on X then the same argument shows condition (6) is
satisfied. In the k-set and finite k-plane transforms above we
have such a doubly transitive group.
For the k-set transform the group is the symmetric group Sn
of all bijections on X. When X is affine n-space, H is the
group of all affine transformations x- ax+ b, where
a 6 G~(n,F) and b 6 Fn = X. When X is projective n-space the
group H = P~(n,F) acts doubly transitively. Therefore

COROLLARY 2. The k-set transform and the affine and projective

finite k-plane transforms are injective.

3. CHARACTERIZING THE RANGE. SPREADS. Attempts to character-

ize the range form a major theme in the study of the classical
integral transform. Typically, the image of R is found to be
the kernel of an ideal of partial differential operators [John],
[Grinberg]. It seems hard to find differential operators on
C(Y) when Y is finite. Since, for such spaces all operators
are integral operators, we shall search for them instead.
Consider X = F22 , the affine plane over the two element
field; let Y = [f] be the set of lines in X. Since each line
contains two points and two points determine a line, Y can
equally well be viewed as ( ~ J· Thus IIY = 6. Figure 1 is a
common picture of Y.

01 a 11

00 10
a

Figure 1

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 33

Note that we can break Y up 3 ways into families of

mutually parallel lines:

u = {a,a'}, v = {b,b'}, w = {c,c'}.

Call each of u, v, w a s;eread and let w = {u,v,w} be the set

of spreads. Then the incidence relation between lines and
spreads determines a Radon transform T: C(Y) .. C(W). Now
suppose <P 6 C(X). Then R(jl 6 C(Y) and

TR(jl(u) = R(jl(a) + R(jl (a' ) = 2 (jl(x) v,((jl). ( 11 )

x6X

The same computation shows

TR<j>(v) = TR(jl(w) v,((jl)

so that T~ is constant for each ~ = R(jl in the range of R.

To see that this condition characterizes the range we count
dimensions. Write for the constant function ~W e C(W)
and <1> for the subspace of constant functions. Let
p : C(W) + C(W)/<1> be the natural projection, and T' poT.
We have just proved that in the sequence

o .. C(X) ~ C(Y) I'ccw)/<1> .. o ( 12)

the composition RT' 0. so

im R c ker T' • ( 1 3)

To establish equality in (13) we prove (12) is exact. We

already know R is injective. That T and hence T' is sur-
jective is obvious. Then simply observe that

dim C(X) +dim C(W)/<1> = 4 + (3-1) = 6 =dim Y

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 34 ETHAN D. BOLKER

to prove (12) exact at C(Y)

This simple example generalizes. Let Y be any set
of subsets of the finite set X and R the corresponding
Radon transform. A spread is a subset u of Y such that
{FY 1 y e u} is a partition of X. When W is a set of spreads
we can construct the Radon transform T : C(Y) ~ C(W). With T'
defined as above, the argument we have given shows that RT' = 0,
so (13) is true. We shall say that W is large enough when

im R ker T' (14)

and that there are enough spreads when the set of all spreads is
large enough.

THEOREM 3. Let X = Fn, affine n-space over the q-element

field, and Y G~ 1J, the set of hyperplanes in X. Let R be
the transform defined by the usual incidence relation between
points and hyperplanes. Then there are enough spreads.

To prove the theorem we use the following counting lemmas,

which establish well known analogues for the lattice of subspaces
of a geometry over a finite field of well known facts about the
lattice of subsets of a finite set.

LEMMA 4. Fn contains

ordered sets of k independent vectors.

PROOF. After j - independent vectors have been chosen, the

jth can be any of the qn vectors in Fn except one of the
qj- 1 in the space spanned by the first j - 1. II

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 35

LEMMA 5. Fn has

ordered bases,

( 15)

k-dimensional subspaces, and

k-dimensional flats (k-planes), organized into parallel

pencils of q n-k flats each.

PROOF. The first assertion is just Lemma 4. To prove (15),

observe that the numerator is the number of ordered independent
sets which span k-dimensional subspaces, and the denominator is
the number of bases per subspace. The last assertion is a conse-
quence of the fact that if V is a k-dimensional subspace then
Fn/V .. Fn-k parameterizes the k-planes parallel to V. II
Now we can prove Theorem 3. A spread is just a pencil of
parallel hyperplanes. To show there are enough spreads we show
(12) is exact. Corollary 2 establishes exactness at C(X). That
T is surjective is easy to see: if y e u e W then u is the
only pencil containing y. Hence Toy = ou. Thus T and hence
T' is surjective. Now count dimensions:

dim C(X) + dim C(W)/<1> = qn + rLn-1_

n -,
0
- 1

n + nn-1
= q ~- 1
q-1

a
Ll
q( q-1)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 36 ETHAN D. BOLKER

A second transform whose range we can characterize using

spreads is the line transform in projective 3-space over a finite
field F with q elements. Then X= p3(F), which we can
think of as the set of lines through the origin in F4 . Hence by
Lemma 5,

q 3+ 1 - 1
q -

When q = 2, each line through the origin in contains just

one non-zero vector, so X can be identified with the set of 15
non-zero vectors in F24 . In general, lines in correspond
to planes through the origin in F 4 , so the set y of lines
satisfies

IIY =
~]o
(q4 - 1)(q 4 - g)
(q
4 - 1) ( q2 - q)

(q2 + 1) (q2 + q + 1).

In particular, ltY = 35 when n = 2. The ordinary incidence

relation of points and lines in p3(F) is mirrored in F4 by
the inclusion of lines through the origin in planes. In p3 (F) ,
there are no parallel lines, but two lines can be skew and hence
disjoint. A spread is a partition of X into a family of skew
lines. David Mumford showed us the following argument.

THEOREM 6. Spreads exist in p3(F)

PROOF. Let L be a quadratic field extension of F·, there is

always exactly one when F is finite. Define a complex struc-
ture to be an identification of L2 with F4 as F-vector spaces

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 37

Clearly many exist; choose one. Then the L-lines through 0 are
F-planes and hence lines in X. Any two such L-lines meet only
at 0, and hence correspond to skew lines in X. But the set of
L-lines through 0 clearly partitions F 4 - {0} and hence X.
Thus we have found a spread. II
This spread has interesting geometric properties. Suppose
that L = F(e). Then each z 6 L can be written uniquely in the
form z = x + By, with x, y 6 F. If we fix m6 F and con-
sider all the L-lines (x + By)L in F4 for which x/y = m it
is not hard to show that the corresponding F-lines form one
ruling of a quadric surface whose quadratic form has signature
+, +, -, -.

This argument works even for the classical infinite fields:

we can use the complex numbers for L when F is the real field
and the quaternions when we start with the cOmplex numbers. Can
anything analytically interesting be done with that information?
Can the sets of lines and spreads be topologized well enough to
construct the Radon transform T? The signature of the quadratic
form is precisely the signature of the ultrahyperbolic partial
differential operator whose kernel is the image of the classical
line transform. Is that just a coincidence?
But even without answers to these analytic questions, spreads
will give us answers to questions about finite transforms. We
call the spreads which come from complex structures complex
spreads. Let W be the set of complex spreads. We wish to
prove that W is large enough to characterize the range of the
Radon transform. The dimension counting argument we used in
Theorem 3 will not do here, because W is too large for the map
T: C(Y) + C(W) to be surjective. One can use Lemma 5 to count

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 38 ETHAN D. BOLKER

complex spreads: there turn out to be (q 4 -q)(q 4 -q3)/2. When

q = 2 that yields 56 complex spreads, while dim C(Y) = #Y = 35.

We rely instead on group theory to prove that there are enough

spreads.

THEOREM 7. (Schur's Lemma) Let the finite grQup G act on the

finite sets S, T. Then G naturally acts on S x T and on

C(S) and C(T). The action on C(S) is given by g~(s) = ~(g- 1 s)

for ~ e C(S) and g e G. Then

dim HomG(C(S),C(T)) # {orbits of the G action on S x T}.

Here HomG(C(S),C(T)) is the space of intertwining operators:

linear maps from C(S) to C(T) which commute with the group

action.

PROOF. The most general linear transformation K from C(S) to

C(T) is given by a matrix K(t,s) whose rows (resp. columns)

are indexed by T(resp. S):

K~(t) = L K(t,s)~(s).
ses

Now
(g(K~))(t) * L K~(g- 1 t,s)~(s)
ses

L K(g- 1 t,g- 1 s)~(g- 1 s)

ses
while
(K(g~))(t) ~ L K(t,s)~(g- 1 s).
ses

These two expressions are equal for all ~ and g if and only

if

K(t,s) K(gt,gs)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 39

for all g € G. Thus K must be constant on the orbits of the

action of G on S x T. The dimension of the space of such K
thus equals the number of orbits. II

COROLLARY 8. Let G act on S. Suppose that

is a decomposition of C(S) into a direct sum of k1 irreduc-

ibles of type v1 , etc., where the Vi are mutually inequivalent
irreducible representations. Then

( 1 6)

is the number of orbits of G acting on S x S.

PROOF. The expression in (16) equals dim HomG(C(S),C(S)).

We can now use these lemmas to show, following an argument

of Andrew Gleason, that there are enough spreads.

THEOREM 9. There are enough complex spreads to characterize the

range of the projective line transform over a finite field.

PROOF. We must show that the sequence in (12) is exact. Since

the projective group G is doubly transitive there are two
orbits for the action of G on X x X. One is the diagonal, the
other is its complement. Thus by Corollary 8

C(X) = C Ql V,

where we can identify the two irreducible components as the one

dimensional space C of constant functions and the space V of
functions with mass 0. The action of G on Y x Y has three
orbits, containing of those pairs of lines (y,y') which are

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 40 ETHAN D. BOLKER

respectively equal, coplanar but distinct (and hence meeting in a

point), or skew. Since 12 + 12 + 12 is the only way to write
3 as a sum of squares, C(Y) is a sum of three distinct
G-irreducible components. Since R : C(X) + C(Y) is injective,
we can identify two of these as isomorphic to C and V, so

C(Y) C ~ V ~ V'
- ( im R) ~ V'.

Now ker T' is a G-invariant subspace which contains im R, so

either ker T' = 1m R, or ker T' is all of C(Y). The latter
is impossible since T' is not identically 0. II
The fact that the Radon transform T is no longer surjec-
tive raises a tantalizing question: can we characterize its
range? That question leads us to a classical combinatorial
problem first posed by T. P. Kirkman in 1847:

A schoolmistress is in the habit of taking her girls

for a daily walk. The girls are fifteen in number and
are arranged in five rows of three each so that each
girl might have two companions. The problem is to
dispose them so that for seven consecutive days no girl
will walk with any of her school fellows in any triple
more than once.

The reference is [Ball/Coxeter, p. 287]. Suppose we ident-

ify each schoolgirl with one of the 15 points in X=
There are ( 1{) = 455 possible rows of 3 available to the school-
mistress when she sits down to make her schedule for the week,
but we shall restrict our attention to the 35 which form projec-
tive lines - the elements of Y. Then a daily schedule is
nothing but a spread. Theorem 9 tells us that the 56 complex
spreads in W are enough to characterize the range of the Radon
transform R : C(X) + C(Y). But are there enough to solve

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 41

Kirkman's problem? That would require 7 daily schedules of 5

lines each, using no line twice. Since there are 35 lines, such

a solution is simply but a spread of spreads. In fact, spreads

of spreads do exist. Ball/Coxeter found one in order to solve

Kirkman's problem. But our problem remains. Let w2 be the set

of spreads of spreads - the solutions to Kirkman's problem which

respect the projective structure. We know w2 is not empty, but

is it large enough? That is, if we let T2

the natural Radon transform coming from the incidence of daily in

weekly schedules, does im T = ker T2 ' in Figure 2? I do not

know.

R T

"
0-+- C(X)-4- C(Y) -

T'
),
C(W)/<1>

Figure 2

The questions continue, even though we lack answers. We

think we know #W 2 = 240. Can we characterize im T2 using

spreads of spreads of spreads, sets of 8 weekly schedules which

together exhaust all 8·7 = 56 daily schedules? Call such a set

a quarter. Are there any quarters? If so, are there enough?

Continuing, spreads of spreads of spreads of spreads, if there

are any, would there be sets of 30 quarters which used all

30·8 = 240 weekly scheduies? Thirty quarters make an education -

and so on. Moreover, these questions are open over fields

larger than F2 . There, we do not even know about the existence

of spreads of spreads.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 42 ETHAN D. BOLKER

5. CHARACTERIZING THE RANGE; REPRESENTATION THEORY. Lest we be

carried away by our successful search for spreads, we note that
there are many examples where no spreads exist. In particular,
there are none in the k-set transform on an n set unless k
divides n. Even then, there may not be enough spreads: look at
the 3-set transform on a 6 element set. Our major tool for
circumventing these difficulties will be the local spread. When-
ever we can find two subsets w1 , w2 of Y each of which is a
partition of the same subset of X we can conclude that for
This idea will allow us to study simulta-
neously both the finite k-set transform and the representation
theory of the symmetric group, and the analogous structures over
finite fields.
Write Yk for (~). In particular, Y0 = 0 and Y1 = X.
We shall study the transform R~ C(Yj) + C(Yk) defined by

for j s k, A e Yj and B e Yk.

If we think in terms of the simplicial complex of all

subsets of X then C(Yk) is the space of (k-1)-chains. (The
unfortunate shift of in the index is a consequence of the
convention which says that a set of cardinality has dimension
0.) The map resembles the coboundary operator from
k-1-chains to k-chains, except that all the signs in (17) are
positive. Thus a2 f. 0 and we do not have cohomology· in the
usual sense. The following lemma describes what we have instead.

LEMMA 10. For i < j < k,

Rj R~ ( 1 8)
k J

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 43

PROOF. The calculation is not difficult. The coefficient on the

right in (18) is the number of j element subsets between a k-
element set and one of its i element subsets. 11

The analogy between the Radon transform and the boundary

operator has geometric content, too. Write z~ for the image in
J
i
C(Yj) of Rj, the space of coboundaries. Combinatorial topol-
ogy uses cycles to study boundaries: the support of the simplest
k-cycle is the boundary of the k+1-simplex. Since in our theory
we have no cancellation built into the boundary operator, the
simplest k-cycle will be the boundary (in the geometric sense) of
the cross polytope in dimension k + 1 (that's the octahedron in
dimension 3). The 2k facets of the cross polytope can be
nicely divided into two classes each of which partitions the k-1-
faces, and gives us the local spreads we need.
Suppose 2k :s; n. Let P - Pk = {x 1 , .•• ,xk,x 1 , ,xk}
be a 2k element subset of X. Call each subset F of P
which for each i contains just one of xi, xi a facet. Let b
be the number of points xi in F. Call F even (odd) when b
is even (odd) and define sign (F) to be (-1)b. Finally, define
the cross Eol;y:toEe wk by

L sign (F)6F ( 19)

F a facet of Pk

Figure 3

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 44 ETHAN D. BOLKER

Figure 3 is the octahedron w3 with even facets shaded, showing

how the odd and even triangles each partition the edges.

LEMMA 11. Wk ~ Z~- 1 •

PROOF. Since is spanned by the functions k-1

Rk c5 A'
A e Yk_ 1 , it suffices to show that for all such A,

Rk-1c5 ~wk.
k A

But (19) and (17) show

2 sign (F). (20)

facets F=>A

I f no facet of pk contains A, this sum is 0. I f the k - 1

element set A is contained in some facet F, then F A u {x},

where for some i, X = xi or xi. Swapping xi and xi yields
a second facet F' containing A. Since F and F' have
opposite signs, their contributions cancel in (20), which is thus
always Q. II
Now s n acts transitively on the set of all 2k-sets Pk
in X, and hence transitively on all the cross polytopes
Wk e C(Yk). Define the Specht module Mk to be the subspace of
C(Yk) generated by the cross polytopes. Then any Wk is a
cyclic vector for Mk, which is Sn invariant and orthogonal to
z~- 1 • We shall prove soon that Mk = (Z~- 1 )~

LEMMA 12. If i < j and 2i s n then in if

and only if i + j > n.

PROOF. if and only if for every

But

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 45

L sign (F).
facets F=>A

Suppose first that i + j s n. Then we can find a j element

subset A of X which contains all of x1 • ... ,xi but none of

x1 ' ...
,xi. Then F = {x1 • ... ,xi} is the only facet
contained in A. Since it is even, the right member of (21) is
1• not 0.

Suppose instead that i + j > n, and that there is a facet

contained in the j-set A. Then for some t, A must contain both
Hence interchanging xt and maps the even
facets contained in A to the odd ones, forcing the sum in (21)
to be 0. II

THEOREM 13. For k S 2n,

1. Mk is an irreducible component of the representation of

sn on C(Yk),
2. The Specht modules M1 , •.. ,Mk are distinct
irreducibles,
3. The decomposition of C(Yk) into irreducibles is

• • • 'i (22)

(~)~=the sum of the first k terms in (22).

PROOF. We need only assemble the pieces. For k = 1, examine

C(Y 1 ) • C(X). There (1) is the one dimensional invariant
subspace of constant functions. Since the Sn action on X x X
has two orbits, and 12 + 12 = 2 is the only way to write 2 as
a sum of squares, Corollary 8 shows C(X) is a sum of two
distinct irreducible representations, of which we've found one.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 46 ETHAN D. BOLKER

For P1 {x,x} c X we have

so M1 is the second piece.

Now suppose the theorem true for values j < k. Since
j + k < 2k s n, Lemma 12 shows R~Mj is not identically 0.
Since Mj is irreducible, R~Mj is irreducible and isomorphic
to Mj. Thus our inductive hypothesis tells us we have found k
distinct irreducible components of C(Yk). Lemma 11 shows each
of these lies in zkk-1 . The action of s on yk yk has k + 1
n
X

orbits: the orbit of the pair (A, A') is determined by

#(An A'), which may be 0, 1, ... ,k. Then Corollary 8 shows
there must be k+1 irreducible components. Lemma 11 shows that Mk
is orthogonal to the. k components we already have, so it must
be the last one, and must be (Z~- 1 ) J. • II
A little more work, which we shall not include here, shows
that the range of is the sum of the first j terms in (22).
Thus, in particular, we have characterized the range of the k-set
transform
Combinatorists will recognize Mk as the irreducible repre-
sentation of Sn which corresponds to the tableau shown in

--
Figure 4.
k n-2k

I I I I I I
Figure 4

It is natural to ask about the other representations. They too

can be constructed using finite Radon transforms, though the task

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 47

is more complicated because multiplicities enter the picture.

Shlomo Sternberg has carried it out, translating the algebra in
[James] into the language of the Radon transform.
The tableau exemplified by Figure ~ corresponds to the
Grassmannian in the continuous case. The analogues of the more
general tableaux are the flag manifolds. It would be interesting
to see what the analogues of these and James' constructions
look like in the continuous case. Grinberg has begun a study
of Radon transforms on flag manifolds.
We close this paper with a sketch of a method for mimic
the preceding discussion of the k set transform for the k
plane transform in order to study the representations of the
group G~(n,F). It is well known that each of those linear
groups has an irreducible representation corresponding to each
irreducible representation of Sn - see, for example,
[Steinberg]. We shall indicate here how to construct some of
them by finding analogues of the cross polytopes, and hence the
Specht modules. Since the proofs I have are computational rather
than elegant, I shall omit them. The reader is urged to search
for geometric ones.
We define as before, using subspaces of projective
n-space over F rather than subsets of an n-set. Then with a
shift of indices, we can think of Yk as the set of k dimen-
sional subspaces of the n dimensional vector space V = Fn
over R. The analogue of Lemma 10 is easy to prove. Suppose
2k :ii n. Let

be a basis of the dual space V*. Let v0 be the 2k-dimensional

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 48 ETHAN D. BOLKER

subspace of V orthogonal to each zi. On V consider the

quadratic form defined by

k
Q(V) = L xi(v)xi(v). (23)
i=1

We find the local spreads we seek in the rulings of the quadratic

surface H = {v 6 vo I Q(V) = 0}. Let G be a facet of the set

{xi' xi} viewed as a cross polytope. Then for each i let

ui - xi if xi 6 G and -
xi otherwise; similarly define ui by
reversing the roles of xi and xi. Suppose A is an n x n
matrix over F. Then define vectors v 1 , ..• ,vk in v0 by
demanding that for i, j = 1, ••• ,k

Finally, write L(G,A) for the span of {v 1 , ..• , vk}.

THEOREM 14. L(G,A) is k-dimensional, and is a ruling of H

(that is, lies in H) if and only if A is skew symmetric.
Every k-dimensional subspace of v0 which lies in H is L(G,A)
for some facet G and skew symmetric matrix A. G and A are
not unique, but fi(G n {xi})mod 2 is.

Theorem 14 implies that we can divide the rulings of H

into two classes, determined by the parity of #(G n {xi}).

LEMMA 15. Every k-1 dimensional subspace of v0 which lies in

H lies on exactly two rulings, which have opposite signs. We
can then define the analogue of the cross polytope as

wk = I <SL
L a ruling of H

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE FINITE RADON TRANSFORM 49

and copy the p~oofs of Lemmas 11 and 12 and Theo~em 13 to con-

st~uct the desi~ed ~ep~esentations of G t(n,F).

The ~eade~ who takes the t~ouble to wo~k out the geomet~y of

the ~ulings when n = 4 and k = 2 will find connections to the

global sp~eads with which we studied the line t~ansfo~m, since

the positive lin~s which ~ule a hype~boloid of one sheet a~e pa~t

of a global sp~ead of mutually skew lines. The~e is much still

to unde~stand about these finite t~ansfo~ms, and much analysis to

do conce~ning thei~ continuous analogues.

BIBLIOGRAPHY
1. Ball, W. W. R, and Coxete~, H. S. M., Mathematical
Rec~eations and Essays, 12th Ed., To~onto, 1974.

2. G~inbe~g, E., "Sphe~ical ha~monics and integ~al geomet~y

on p~ojective spaces," T~ans. Arne~. Math. Soc. 279 (1983) 187-203
3. G~inbe~g, E., "Euclidean Radon t~ansfo~ms: ~anges and
extensions, These P~oceedings,

4. James, G. D., The Rep~esentation Theo~y of the Symmet~ic

G~oups, Lectu~e Notes in Mathematics, vol. 682, Sp~inge~-Ve~lag
(1978).

5. James, G. D., The Rep~esentations of Gene~al Linea~

G~oups, London Mathematical Society Lectu~e Note Se~ies 94,
Camb~idge, 1984.
6. James, G. D. and Ke~be~, A., The Rep~esentation Theo~y
of the Symmet~ic G~o6p, Encyclopedia of Mathematics and its
Applications, Vol. 1 , Addison-Wesley, Reading MA, 1981.

7. John, F., "The ult~ahype~bolic diffe~ential equation

with fou~ independent va~iables, Duke Jou~nal of Mathematics 4
(1938) 300-322.
8. Kazhdan, D., "Integ~al geomet~y and ~ep~esentation
theo~y," Rep~esentation Theo~y of Lie G~oups, London Mathematical
Society Lectu~e Note Se~ies ~ Camb~idge, 317-324.
9. Kung, J., "The Radon t~ansfo~m of a combinato~ial
geomet~y," Jou~nal of Combinato~ial Theo~y Se~ies A £§. (1979) 97-
102.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 50 ETHAN D. BOLKER

10. Lehrer, G., "On incidence structures in finite classical

groups," Mathematische Zeitschrift 147 (1976) 287-299.
11. Ryser, H. J., Combinatorial Mathematics, Carus
Mathematical Monograph No. 14, Mathematical Association of
America, 1963.
11. Steinberg, R., "A geometric approach to the representa-
tions of the full linear group over a Galois field," Trans.
Amer. Math. Soc. l l (1951) 274-282.

DEPARTMENT OF MATHEMATICS & COMPUTER SCIENCE

UNIVERSITY OF MASSACHUSETTS/BOSTON
BOSTON, MASSACHUSETTS 02125

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

HYPERFURCTIONS IN REPRESENTATION
THEORY AND MATHEMATICAL PHYSICS

Edward G. Dunne

I wish to explain here certain results from mathematical

physics (primarily twistor theory) and representation theory for
semisimple Lie groups. In particular, I want to point out how
they relate to one another in specific instances and how ques-
tions arising in one may find their answers in the other.
The first success of Penrose's twistor geometry was the
construction of massless fields by taking contour integrals of
homomorphic data on CP3, specifically, sections of line bundles
over an open se t P+ [14]. This process was formalized and
put into the language of complex geometry and integral geometry
in [4]. On the level of quantization, however, there was no
intrinsic Hilbert space structure on the holomorphic data which
was generating the Hilbert space of massless fields. By using
the double fibration, one could put a Hilbert space structure
there by pulling back the structure on massless fields [15].
This method is somewhat unsatisfactory in that it seems to be
begging the question. This problem in twistor theory turns out
to be a specific case of a more general problem in representation
theory: How can one geometrically realize "singular" unitary
representations of a semisimple Lie group? That is, is it pos-
sible to model the construction of the discrete series as
@ 1987 American Mathematical Society
0271-4132/87 [Link] + $.25 per page

51
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 52 EDWARD G. DUNNE

cohomology spaces on an appropriate homogeneous space as in [17,18]

for other classes of representations? By using holomorphic homo-
geneous spaces and partially harmonic differential forms, such a
realization was given for all unitary representations except for a
set of Plancherel measure zero in [23] . These are the regular
representations. Now, there remained the realization for the sin-
gular unitary representations. The quantization process wanted in
twister theory involves such representations for the group SU(2,2),
the four-fold cover of the conformal group. Such a process was
described in [15] . The general problem for representations which
are limits of the holomorphic discrete series is solved in [16]
using indefinite harmonic analysis. Thus, the problem from twister
theory was also resolved since the relevant representations are
covered by the Rawnsley-Schmid-Wolf result.

Twister theory generated another interesting problem related

to the one discussed above. The differential equations for zero-
rest-mass (z.r.m.) fields are hyperbolic. Thus, they will have
solutions which are not smooth and may have singularities. It is
necessary, therefore, to consider generalized solutions. If one
takes an arbitrary distributional solution on real (affine)
Minkowski space, it may not extend as a distribution to the
conformal compactification of Minkowski space. It will, however,
have a (non-unique) extension as a hyperfunction solution of the
conformal extension of the z.r.m. equations. This fact illus-
trates the need for hyperfunctions when considering the conformal
invariance of these differential equations. Now, with this
larger space of functions, one must ask whether the group theo-
retic results are still valid. That is, do the isomorphisms and
correspondences still hold on the level of representations of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 53

SU(2,2)? If so, is it possible to deduce them by purely repre-

sentation theoretic techniques? It turns out that everything
does carry over. Furthermore, from this point of view, the
naturality of hyperfunctions is quite evident. In the last
section below, some generalizations of these constructions will
be given in the context of integral geometry.

1. GLOBALIZATIONS OF HARISH-CHANDRA MODULES. In order to

explain the representation theory correctly, I first need to talk
about globalizations of Harish-Chandra modules. Let G be a
semisimple (or reductive) Lie group with maximal compact subgroup
K. Denote the Lie algebras of G and K by _g_ 0 and ~ 0 and
their complexifications by _g_ = (g 0 )C and ~"" (ko)c. (This
convention will be used throughout.) A Harish-Chandra module, or
(~, K)-module, is a complex vector space V which carries a ~

and a K action such that

1) the two actions are compatible
2) V is a direct sum of irreducible K-modules, each
irreducible occuring with finite multiplicity,
3) V is finitely generated as a U(~)-module, where U(~)is

the universal enveloping algebra of ~·

If (p,VP) is a unitary representation, then (Vp)K • {v e VP: p(K)v

is finite dimensional} • (the space of K-finite vectors) is a
Harish-Chandra module. This is true even with much weaker condi-
tions: (p,VP) is an "admissable" representation, VP is a
Frechet space.
Suppose V is a Harish-Chandra module. Then is it true
that for some (p,Vp)? Such a representation of G

is called a globalization of V. Casselman [2] and Bernstein

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 54 EDWARD G. DUNNE

have shown that any Harish-Chandra module has a globalization.

It is even possible to make vp a Hilbert space, though you
cannot guarantee that p is unitary.
Since a globalization is essentially a choice of topology in
which to take the completion of V, any Harish-Chandra module
has several globalizations. Is there, however, a canonical
choice? If VP is to be a Hilbert space, there is no such
choice. If Vp is allowed to be Frechet, Schmid [19] has shown
that there is a canonical choice. This globalization, Vmax'
has the property that any other globalization VP can be mapped
continuously, G-equivariantly, and injectively into Vmax by a
.map which restricts to the identity on the original Harish-
Chandra module, V. Furthermore, Vmax can be interpretted as a
space of hyperfunctions. What I will explain later is how to put
the hyperfunction theory of massless fields into this context.
We will see that this is actually a special case of Schmid's
theorem where the construction can be done by hand. Finally,
using integral geometry, the explicit construction will be given
for limits of the holomorphic discrete series for groups other
than 0(2,2), the relevant reductive group for twistor theory.

2. TWISTORS AHD THE Z.R.M. EQUATIONS. Let T denote c4 with

a fixed Hermitian form ~ of signature (+ + - -). This pair is
twistor space. Associated to T are the flag manifolds:
M z G2 , 4 = two-dimensional (complex) subspaces of !• r c3
the space of one-dimensional (complex) subspaces of !, and
F = ! 12 = the space of pairs (p,z) where z is a two-dimen-
sional subspace of T and p is a one-dimensional subspace of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 55

T contained in z. The Hermitian form ~ defines distinguished

subsets:

p+ = {p e P ~lp»O}
p {p e P ~lp«O}

fo ~ {p e P ~lp = 0}
F+ a
{p,z) e F ~lz»O}

and so on for f-, fa• ~-. and ~0 . The space ~0 is the

conformal compactification of real (affine) Minkowski space.
since is called complexified compactified
Minkowski space. Let U(2,2) be the subgroup of GL(4,C) pre-
serving ~ on T. Then ~+, ~-. ~0 , etc. are homogeneous
spaces under U(2,2) = G. Notice that Gc a GL(4,C). Explicit-
ly, we have:

K - U(2)xU(2) = a maximal compact subgroup of G

H U(1)xU(1,2) not compact

p
!!! GC/P

GC/P p
M PM "
~ ~ A, B, D, 0 2x2 matrices

!!I
p11 p12 p13 p14
0 p22 p23 p24
pp !I
0 p32 p33 p34
0 p42 p43 p4
F12 • GC/(PM n Pp)
K • G n PM H = G n Pp

Furthermore, ~ 0 is a compact orbit of the group G, hence is a

homogeneous space under K. To be exact, ~ 0 - K!K 0 where K0

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 56 EDWARD G. DUNNE

is the diagonal of the direct product U(2)xU(2): K0 = {(k,k)

k e U(2)}. In addition ~ 0 is the Shilov boundary of each of
M+ and M-.

The fundamental diagram for twister theory is the double

fibration:

(Fig. 1)

where a and B are the natural projections. These maps are G-

equivariant. Hence they respect sub- and super-scripts:

and so on.
The zero-rest-mass equations represent the kernels of dif-
ferential operators between certain spinor bundles on M. These
bundles can be described in the context of representation theory
as follows. Let s *1 , 0 be the representation of PM where A

acts by the dual of the standard representation of GL(2,C) on

c2 and B and D act trivially. Let s~.o be the representa-
tion of PM where D acts as the dual of the conjugate trans-
pose representation of GL(2,C) on and A and B act
trivially. Denote the associated homogeneous vector bundles on
GC/PM = M by *
s,,o and s,* ,o· These are the fundamental spinor
bundles from which the others are built. Let Sym n (S *
1,0 ) z
*
Sn,O
and Sym n (s *
0 ,1) = s *0 ,n be the homogeneous vector bundles asso-
ciated to the nth symmetric power of s~.o and S~~ 1 , respect-
ively. Let S* * *
p,q • Sp,O g S O,q· The notation is chosen in

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 57

analogy with the (p,q)-forms of complex geometry. Then the

zero-rest-mass operator of helicity n/2 is a differential
operator

with symbol map at the identity coset:

(z
A' •••
A, ,D

I••• ,De BQ ... Qe DQ(~(e A))

A,B,

where ~ e Hom s,* '0' so'

* 1) s *
TeK(!1_ is a basis
and {e 0 e1}
'
for s,* '0. On !!o' using the Peter-Weyl theorem for L2 (KILO),
vn can be written in terms of the differential d:

Vn = L (Xi Q1 - 1QXi) g <<lgxr)od)

i

where Xi is a basis of ~(2,C) and Xi* is the dual basis,

with respect to the Killing form. By thinking of *
s n,O as
homogeneous functions, d is interpreted as the standard dif-
ferential. This gives an invariant formulation for Vn.
Let 0 denote "holomorphic" and O(n) be holomorphic sec-
tions of the nth twisted line bundles on CP3. The following is
from [4]:

THEOREM: H1 (P+, O(-n-2) • Ke"' 0 (V '· M+) •

n - 4

The isomorphism is via the Penrose transform (nintegration over

the fibers") which explicitly generates solutions. Note, at this
point there is not yet a nice theorem for massless fields on
real compact Minkowski space. To state such a theorem, we will
need hyperfunctions.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 58 EDWARD G. DUNNE

3. HYPERFUNCTIONS. Loosely speaking, for a real analytic mani-

fold M, the space of hyperfunctions B(M) is the dual of the
space of analytic functions on M. This is a valid definition
only if M is compact. Alternatively, Sato defined hyperfunc-
tiona as boundary values of holomorphic functions on X - M,
where X • Me. This definition is necessary if M is non-
compact. For compact M, it can be shown to be equivalent to
the first definition. A good interpretation of Sato's notion is
in terms of relative cohomology. Thus, if M is n-dimensional,
n
(M) = HM(X,O). There are similar relations if 0 is replaced
by other sheaves.
It will be better, perhaps, to consider an example at this
point, rather than going into the details of hyperfunctions
(which can be found in [5] and [12]). Let M= s1 , the unit
circle. Then X is the Riemann sphere. Suppose ~ is a gener-
alized function of s1 with Fourier expansion

(I am purposefully avoiding a precise definition of "generalized

function" for the moment.) It is known that ~ e A(S 1 ), the
space of analytic functions on s 1. if and only i f there exist
some l'i > 0 and c > 0 such that l~nl s Ce-l'ilnl. The dual of
this criterion is: ~ e A'(S 1 ) : .. B(S 1 ) if and only if for all
€ > 0, there is a c€ > 0 such that l~nl s D€ eelnl • This
gives a characterization of hyperfunctions by their Fourier
series. Notice that this condition is not very restrictive, thus
B(S 1) is big. For general compact groups, use the Peter-Weyl
theorem to write

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 59

the generalized Fourier expansion. Then <I> e B(K) if and only

if II <1>>.11 ~ c e£1>-1 for all £ > o. For <I> 6 B(S 1 ), write
£
<I> .. <I>++ <I>_ where

= ! <I> ine and = ! <I> ine

<I>
+ ne <I>_ ne
nli:O n<O

f+(z) = ! cpneine defines a holomorphic function on the unit

nli:O
disk. Similarly, we can define f (z) which is holomorphic
outside the unit disk. Thus, <1> is the sum of the boundary
values of f+ and f_. The same procedure works in the opposite
direction. Simple arguments show that any holomorphic function
on the disk has hyperfunction boundary values.
Now that we have hyperfunctions, the isomorphism theorem for
real compact Minkowski space can be stated:

THEOREM': (Wells; Bailey-Ehrenpreis-Wells)

and there is a splitting of any solution u e Ker 8 into future

+
and past components: where u represents bound-
ary values of a holomorphic function on M+ and similarly for
u •

4. THE REPRESENTATION POINT OF VIEW. Let us now look at Theorem'

from the perspective of representation theory. The approach is
to establish the isomorphism on the level of Harish-Chandra
modules, then try to globalize the isomorphism. First, notice
that H~ (f, 0( -n-2)) splits as H1 (f+ ,O(-n-2)) + H1 (f-' o<-n-2)).
-0
This splitting is G-equivariant since f, p•, and p are
G-orbits in P where G acts by left translation.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 60 EDWARD G. DUNNE

THEOREM 1: [3] H1 (f±, 0(-n-2))K s (Ker(Vn; ~±))K as Harish-

Chandra modules.
IDEA OF THE PROOF: Write vn invariantly. Then the associated
Laplacian, Vn = Vn* Vn can be written in terms of Casimir oper-
ators. In this way, finding the K-types is reduced to an alge-
braic problem. Once this is done, one sees that the Harish-
Chandra modules have the same K-spectrum. That is, the irreduc-
ible K-modules in their decompositions occur with the same multi-
plicities. Furthermore, these are Harish-Chandra modules with a
lowest K-type. Since H1 (f±,O(-n-2))K is irreducible (by RSW)),
we can conclude that the Harish-Chandra modules are isomorphic.
Now there are two possible paths to take. First, from
Sato's theory, the relative cohomology H2P (P,O(-n-2)) cor-
-0-
responds to hyperfunction sections. By appealing to Schmid's
theorem, choosing vmax for the globalizations of the left and
right hand sides gives:

Alternatively, it is possible to do this explicitly by using the

double fibration. The fundamental picture for twisters becomes:

where a- 1 o means the topological pullback sheaf, whose

sections are constant on the fibers. The maps induced by a and
6 can easily be seen to be continuous and G-equivariant. That
6 is an isomorphism can be shown by using the Leray spectral
sequence and Bott's Borel-Weil theorem. By checking the highest

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 61

weight of the typical fiber, one sees that both H 0 (~+, Ker(Vn))
and H0 (M-,Ker(V )) are in the limits of the holomorphic discrete
- n
series. Such representations canbe holomorphically trivialized as
O(M±) Q V~± where V~± are the appropriate representations of
the isotropy group, K. The behavior of the boundary value map
is described in terms of this trivialization as follows:

PROPOSITION: If D is a bounded symmetric domain and r its

Shilov boundary, there is a one-to-one topological correspondence
between O(D) and Br(r) where Br(r) is a space of hyperfunc-
tions with prescribed Fourier e_xpansions.

Thus, O(M±) Q V ± BI(M 0 ) Q V ±. The G-action extends to

- ~
=
- ~

the boundary in a natural, but non-trivial, way. Putting all the

pieces together gives:

THEOREM 2: [3] H~ (f.,O(-n-2)) • Ker 8(Vn; ~ 0 ) as

representations of U(2,2).

As a corollary, we see that the splitting of hyperfunction

solutions into future and past components corresponds to the
splitting of the representation into irreducibles. Thus, this
decomposition is also G-invariant.

5. INTEGRAL GEOMETRY. Let G be a connected reductive matrix

group. Let ' be an involutive automorphism of G and
H • (G') 0 , the connected component of the subgroup of elements
fixed by '· Choose a Cartan involution 9 of G such that 9
commutes with ' and K = G9 is a maximal compact subgroup of
G. Assume that X - G/K is a Hermitian symmetric space. This
guarantees the existence of a discrete series for G. Let XC

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 62 EDWARD G. DUNNE

be the compact dual of X. Then X and its Shilov boundary,

x0 , are G-orbits in Xc. It is necessary also to assume that
G/MH K) has an invariant complex structure such that the
fibrations

G/(HnK) + G/H and G/(HnK) + G/K

are holomorphic. It then follows that G/H + K/L is holo-

morphic. Here L • HnK, a maximal compact subroup of H. Label
the homogeneous spaces:

Y • G/H Z • G/L X ~ GIK.

The picture of the double fibration is:

(Fig. 3)

where a and a are the natural maps. Given these choices of

H and K, one can show that X and Y are homogeneous spaces
in duality, in the sense of [8]. The Penrose correspondence
is the expression of this duality for ~ and P±.
Let L + G/H be a Hermitian line bundle associated to a
unitary character $ of H such that x= $IL satisfies the
vanishing condition (4.26) of [16]. Then the Harish-Chandra
module Hs(G/H,O(L))K for s = dimcK/L corresponds to a limit
of the holomorphic discrete series. I want to show now how to
realize its maximal globalization. The procedure is essentially
the same as for twisters.
Recall the double fibration for X, Y, and Z of Figure 3.
We move through the diagram by starting with the data on Y,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 63

moving it through Z to holomorphic data on X. Finally, we

take boundary values along x0 of the data on s. Thus, let
a- 10(L) be the topological pull-back of O(L). The sections of
a- 10(L) are constant along the fibers of a. Consider the map
a*:Hs(G/H,O(L)) +Hs(G/L,a- 1 0(L)). Clearly this map is continuous,
G-equivariant, and injective. Furthermore, by the definition of
a- 1 0(L)) it is onto. Thus a* is an isomorphism. (This implicitly

uses that H/L is Hermitian symmetric, hence contractible.) Con-

sider, now, the direct image sheaves s2a- 1 0(L) which move the data
from G/L to G/K. The qth direct image has fiber Hq(K/L,a- 1 0(L)),

K/L being the fiber of 8. By Bott's Borel-Weil theorem,

Hq(K/L,a- 1 0(L)) iszero in all but one dimension, say q 0 . In dim-

ension q0 , it is the K-module V~ of highest weight ~=w(ljl+pc)-pc

where w is the element of the Weyl group of K which makes ~

dominant and sends the positive 1-roots to positive ~-roots. Then,

lwl = s and, by Bott's Borel-Weil, lwl = qo. Thus, s!a -lo ( L)

has fiber v~. Let v~ be the vector bundle on G/K associated

to v~· The Leray spectral sequence for the fibration B :G/L+ G/K
collapses giving the isomorphism:

By standard results, this isomorphism is continuous and G-equi-

variant. As before, take the holomorphic trivialization

An examination of the arguments used so far shows that they are

all valid for the underlying Harish-Chandra modules of K-finite
vectors. Let B(V~) denote the space of boundary values along
x0 of elements of H0 (X,V~). As shown in [7], G acts on

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 64 EDWARD G. DUNNE

B(V~). Furthermore, B(V~) is infinitessimally equivalent to

H0 (X,V~). By the proposition from the previous section, the
boundary value map is continuous and bijective. Thus, B{V~) is
0
a globalization of H (G/H,O(L))K. Since i t has the hyperfunction
topology, B( V ) is the maximal globalization by Schmid's
~

theorem.

BIBLIOGRAPHY
1. T. Bailey, L. Ehrenpreis, and R. 0. Wells, Jr., "Weak solu-
tions of the massless field equations," Proc. R. Soc. Lond. A384
(1982), 403-425.
2. W. Casselman and D. Milicic, "Asymptotic behavior of matrix
coefficients of admissable representations," Duke Math. J. ~
(1982), 869-930.
3. E. Dunne, "Hyperfunction solutions of the zero-rest-mass
equations and representation theory," Ph.D. Thesis, Harvard,
1984.
4. M. Eastwood, R. Penrose, and R. 0. Wells, Jr., "Cohomology
and massless fields," Commun. Math. Phys. 78 (1981), 305-351.
5. V. Guillemin, M. Kashiwara, and T. Kawai, Seminar on Micro-
local Analysis, Princeton University Press, Princeton,--1979.
6. Harish-Chandra, "Representations of semisimple Lie groups,
IV, V, VI," Amer. J. Math. 77 (1955), 743-777; 78 (1956), 1-41,
464-628. -- --
1. H. Hecht, "The characters of some representations of Harish-
Chandra," Math. Ann. 219 (1976), 213-226.
8. S. Helgason, The Radon Transform, Progress in Mathematics
No. 5, Birkhausei";Boston, 1980.
9. M. Kashiwara and M. Vergne, "Functions on the Shilov boundary
of the generalized half-plane," in Non-commutative Harmonic Anal-
ysis, Lecture Notes in Mathematics No. 728, Springer-Verlag, New
York, 1979.
10. M. Kashiwara and M. Vergne, "K-types and singular spectrum,"
in Non-commutative Harmonic Analysis, Lecture Notes in Mathe-
matics No. 728, Springer-Verlag, New York, 1979.
11. A. w. Knapp and K. Okamoto, "Limits of holomorphic discrete
series," J. Func. Anal. 2 (1972), 375-409.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPERFUNCTIONS IN REPRESENTATION THEORY 65

12. H. Komatsu, "Introduction to hyperfunctions," in Hyperfunc-

tions and Pseudo-Differential Equations, Lecture Notes in Mathe-
matics No. 287, Springer-Verlag, New York, 1973.
13. R. Penrose, "Twister algebra," J. Math. Phys. (N.Y.) 1!
(1967}, 345-366.
14. R. Penrose, "Solutions of the zerQ rest mass equations," J.
Math. Phys. (N.Y.) ..!..Q. (1969), 38-39.
15. R. Penrose and M. A. H. MacCallum, "Twister theory: an
approach to quantization of fields and space-time," Physics
Reports (Section C of Physics Letters),.§. (1972), 241-316.
16. J. Rawnsley, W. Schmid, and J. Wolf, "Singular unitary repre-
sentations and indefinite harmonic theory," J. Func. Anal. .2.!.
(1983), 1-114.
17. W. Schmid, "Homogeneous complex manifolds and representations
of semisimple Lie groups," Ph.D. Thesis, University of
California, Berkeley, 1967.
18. W. Schmid, "L2 cohomology and the discrete series," Ann. of
Math. 103 (1976), 375-394.
19. W. Schmid, "Boundary value problems for group invariant
differential equations," to appear in the Proceedings of the
Cartan Symposium, Lyon, 1984.
20. R. 0. Wells, Jr., "Complex manifolds in mathematical
physics," Bull. Amer. Math. Soc. (New Series) 1. (1979), 296-336.
21. R. 0. Wells, Jr., "Hyperfunction solutions of the zero-rest-
mass field equations," Commun. Math. Phys. 78 (1981), 567-600.
22. J. Wolf, "The action of a real semisimple Lie group on a
complex flag manifold, I: Orbit structure and holomorphic arc
components," Bull. Amer. Math. Soc. 75 (1969), 1121-1237.
23. J. Wolf, "The action of a real semisimple Lie group on a
complex fl~g manifold, II: Unitary representations on partially
holomorphic cohomology spaces," Mem. Amer. Math. Soc. 138 (1974).
DEPARTMENT OF MATHEMATICS
RICE UNIVERSITY
HOUSTON, TEXAS 77251

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

THE EXPONENTIAL RADON TRANSFORM

DAVID V. FINCH1 AND ALEXANDER HERTLE

INTRODUCTION. In this note we prove invertibility and a support

theorem for exponential Radon transforms, a special class of n-
dimensional generalized Radon transforms.
Let v e C(Rn X sn- 1 ) be given. The generalized Radon
transform is the operator defined by

Rvf(w,p) = J f(x)v(x,w)dx (0 .1)

x• w=p

Throughout this note we assume that v is the exponential of a

function linear in x. That is, there exists a continuous vector
field
v : sn- 1 + Rn, such that
v(x,w) = exp(x•v(w)) (0.2)

where denotes the scalar product on Rn. We then call Rv

an exponential Radon transform.
With v as in (0.2) we prove injectivity of Rv on the
space L1 (R,v) of exponentially decreasing L1 functions given
by
L1 (Rn,v) = {f e L1 (Rn) : J lf(x) I exp(n II x II )dx < ""
for some n > II v II .. } •

1supported in part by NSF Grant MCS-8301691.

@ 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page
67
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 68 DAVID V. FINCH and ALEXANDER HERTLE

We also prove a support theorem for R for functions in

v.
L~(Rn). For n = 2 these results were proved in [1]. When
n = 2 we offer an inversion formula similar to that derived by
Markoe [2] for the constantly attenuated Radon transform. Last-
ly, we remark that Quinto [4] has inverted an important subclass
of ( 0. 1 ) on Rn, namely those R which are rotation invariant.
v.
However, (0.1) with (0.2) is not rotation invariant in general.

1. For f 6 L1 (Rn,v.) we denote by f its Fourier transform. It

is well-known that f extends holomorphically into a neighbor-
hood of the complex cylinder ~v~ oo, where for any a > 0

za = { z e en II Im ( z) II => aJ• (1 •1)

Recalling that v. is the form (0.2), we obtain a Fourier trans-

form relation (observed first by Natterer [3] in R2 ) between f
and RV.f. If f 6 L1 (Rn,v.) then

( 1 • 2)

for w e sn- 1 and z e e, where the Fourier transform of R

v. f
is taken with respect to p. In light of this relation we let

A
v.
= lwSSn-
J 1 ( ew + i v ( w) ) c en.

LEMMA (1.1). Im (en-B( II vII .. )) c Av. n Im en, where B( II vii .. >

is the closed ball of radius II vII .. centered at the origin in
en.

PROOF: Writing v(w) = v(w) + a(w)w with v(w)·w = 0, and

setting v.- = exp (x . \1( w)) it is clear that A
v. = A'il. More-
over, since II vII ;$ II vII ... if the lemma holds for \1 it
holds for v, so it suffices to prove the result for \1

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE EXPONENTIAL RADON TRANSFORM 69

satisfying v(w)•w = 0. In suppose

sa = tw + v(w). Then s2 = t2 + I v(w) I 2 and so

and
[a± = ±(1 - s - 2 I v(w) I 2 >1 12 w + s- 1 v(w)].

For S > ~ v ~~ the maps F±s : Sn- 1 + Sn- 1

are homotopic, (+) to the identity, (-) to the antipodal map,

and so must be surjective. Thus for s > ~ v ~~· sa= tw + v(w)
for some ( t, w) .

THEOREM 1.2. is injective on L1 (Rn,v). Moreover, if 0 is

Rv
a nonempty open subset of s n-1 ' then R f is determined by its
v
values on 0 x R, provided v is differentiable at some point
in 0 or f 6 L 1 ( Rn).
c

PROOF: Since f 6 L1 (Rn,v), f is holomorphic in Za for

some a> II" II~· We will show that l JcRw
weo
+ v(w)) c Im en
contains a non-empty open subset of (za\ z I " I ~) n Im en'
which is trivially a set of uniqueness for functions holomorphic
in Za. When 0 = Sn- 1 this follows directly from Lemma (1 .1).
If 0 is a proper subset of sn- 1 and f 6 L1 (Rn) then f is
c
holomorphic on en, and LJ (Rw + v(w)) contains Rn-[tl~.e llvllco
neighborhood of l_ J +
(R w)]
w60
which certainly contains a non-empty
nc
open set. If v is differentiable at some w0 6 0, then
G : R x 0 + Rn by G(t,w) = tw + v(w) is differentiable on
R x (w 0 ). It is easy to see that the differential of G at
(t,w 0 ) is singular for at most finitely many t (the roots of a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 70 DAVID Vo FINCH and ALEXANDER HERTLE

polynomial), hence we may find t arbitrarily close to zero

and a neighborhood of such (t,w0 ) which is mapped
onto a neighborhood of G(t,w 0 ), [6, p. 408]. Since
11 G( t, w0 ) 11
2 = t2 + I v( w0 ) I 2 • if v( w0 ) • w0 = 0, we may
arrange this neighborhood of G(t,w 0 ) to lie in za As in the
proof of Lemma (1 .1) the assumption v(wo)·w 0 = 0 presents no
restriction.

REMARKS 1.3. 1. The theorem clearly extends to distributions

whose Fourier transform is holomorphic in Za. 2. When n = 2,
a limiting argument as in [2, p. 720 bottom] shows that R~f is
uniquely determined by its values on A x R for any infinite set
A c S1 , f e L1( Rn ,p ) • Furthermore one need on~y ·
assume that v
is bounded.

We now restrict attention to functions of compact support.

THEOREM 1.-. Let f e L~(Rn) and suppose Rpf has support in

{ I pI :S r } 0 Then f has support in { I x I S r }.

PROOF: By (1.2) and the classical Paley-Wiener theorem we obtain

the uniform estimate

"'
If ( t;) I < C exp ( r II Im z; I ) for ( 1 • 3)

Since f L2 e
c' it is clear that f is c"" and that
Let e e sn-l. Then
"' 2 satisfies I £ere) I
L L
00

fiRn
8 n o
flee
for T SR, .and I f(io6) I < C exp(rl oJ for lol > II v I oo by

(1 .3) and Lemma (1.1). Applying the Phragmen-Lindeloef theorem

"'
[7, p. 177] in quadrants of C yields lf(z6)l < C exp(riim zl).
The Paley-Wiener theorem of Smith- Salmon-Wagner [5, Thm 14.3] is
now applicable, and shows that f is supported in { I x II S r}.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE EXPONENTIAL RADON TRANSFORM 71

REMARKS 1.5. 1.) Theorem ( 1.4) is stated in L2 terms to make

direct use of [5, Thm 14.3]. By regularization the result
extends to E '(Rn) and thus, of course, L~(Rn). 2.) Assuming
that is c1 and n = 2, the proof used in [1] yields a
"'
support theorem for certain weighted L1 spaces.

2. We assume throughout this section that n = 2 and that "'

is c1 • If w = (cose, sine), let w.L (-sine, cose), and let
cjl(w) = v(w) •w.l • Then

where

We will present an inversion formula for R~ based on the Cauchy

integral formula appled to the intersection of with certain
complex lines in c 2•
Given the complex line
has non-empty intersection with if there
is A 6 C so that ((x 0 y0 ) + A(1 ,w))· J = icp(w). Equivalent-
ly, provided (1 ,w)•tf f. 0, taking

icp(w)- (x 0 ,y 0 )·w.L
( 2. 1 )
(1 ,w)•w.L

Regarding A : s 1 • C by (2.1), we claim that for some w we

A A

may determine f(x o• y0 ) from fj ImageAA. If the numerator of

(2.1) vanishes then (x 0 ,y 0 ) 6 A~ so f(x 0 ,y 0 ) is known. Thus
we may suppose A(w) ~ 0 and compute the index of A with
respect to the origin. Taking Y1 (w) • icp(w) - (x 0 ,y 0 )· w.L and

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 72 DAVID V. FINCH and ALEXANDER HERTLE

(1 ,w)• wJ., Ind>.. (0) = Ind (0) ~Indy (0). If w = iv, 0 ;.

y1 2
then IndY. (0) = ± 1 depending on the sign of v. Thus
2
whatever Indy (O)is, v may be choosen so that Indy(O) ~ 0, and
1
f(x 0 ,y 0 ) may be recovered via Cauchy's formula. Since Indy(O)
may be hard to compute, we note that if we let >..± denote the
contours described above with w = ± i, then our analysis shows

(2.2)
J f((x 0 ,y 0 ) + z(1 ,i))dz/z].
Y+

REMARK 2.1. Formula (2.2) has a more convenient analytical form

in the case of constant attenuation ($(w) = cnst.) than
Markoe's formula [2, 2.6], since it does not involve splitting f
into odd and even parts. Markoe's formula is superior in that it
only requires f to be holomorphic in the interior of Z ~ v ~ ~

and continuous in the closure instead of holomorphic in za. for

some a. > ~ v II .., •

BIBLIOGRAPHY

1. A. Hertle, "On the injectivity of the attenuated Radon

transform," Proc. A.M.S. ~ (1984), 201-205.
2. A. Markoe, "Fourier inversion of the attenuated x-ray
transform," SIAM J. Math. Anal . .12 (1984), 718-722.
3. F. Natterer, "On the inversion of the attenuated Radon
transform," Numer. Math. 32 (1979), 431-438.
4. E. T. Quinto, "The invertibility of rotation invariant
Radon transforms," J. Math. Anal. Appl. 2l (1983), 510-522.
5. K. T. Smith, D. C. Solmon, s. L. Wagner, "Practical and
Mathem·atical aspects of the problem of reconstructing objects
from radiographs," Bull A.M.S • .§1 (1977), 1227-1270.
6. K. T. Smith, Primer of Modern Analysis, Springer, New
York (1983-).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE EXPONENTIAL RADON TRANSFORM 73

7. E. C. Titchmarsh, The Theory of Functions, Clarendon

Press, Oxford ( 1 932). -
DEPARTMENT OF MATHEMATICS
OREGON STATE UNIVERSITY
CORVALLIS, OR 97331
and
FACHBEREICH MATHEMATIK
UNIVERSITAT MAINZ
SAARSTRASSE 21
D-6500 MAINZ, WEST GERMANY

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]

Contemporary Mathematics
Volume 63, 1987

INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS

S. G. GINDIKIN

1. INTRODUCTION. After differential geometry had been created,

a mathmatical theory claiming to be entitled "integral geometry"
should have appeared, sooner o.r later. The first candidate to
be thus christened had been proposed in the 1930's by Blaschke.
The corresponding realm of problems is connected with geometric
probabilities and ascends to the legendary Buffon problem. At
the end of the 1950's I. M. Gelfand [1 ,2] proposed to employ the
term "integral geometry" in another sense. Here I will discuss
some characteristic formulations of problems arising in this
other manner.

2. INTEGRAL GEOMETRY AND REPRESENTATION THEORY. It goes without

saying that first formulations of problems of integral geometry

(in I. M. Gelfand's sense) have arisen as a result of thinking
about the deduction of an analogue of the Plancherel formula for
the group SL(2;C). This result is contained in I. M. Gelfand
and M. A. Naimark's 1947 paper [3] and it has been actually
reduced to the following problem. In the group G = SL(2;C) of
complex 2x2 matrices g = (~ ~) with determinant 1, one con-
siders a unipotent subgroup Z = {(6 ~)} and curves of the form
where g1 , g2 are fixed elements of G. The family
of these curves depends on 3 parameters (notice that dimcG • 3).
© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

75
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 76 S. G. GINDIKIN

These curves were later called horocycles (recall that horocycles

on the Lobachevski plane are trajectories of unipotent subgroups
of the motion group).
Let us indicate the explicit parametrization in the set of
horocycles H. Almost all of them are presentable in the form

a = A + az, 15 A- 1 + bz, Y =z, 6 = aA - 1 + bA + bz, (1 )

where z is a parameter on the horocycle and A, a, b are

parameters of the family. But there are also horocycles depend-
ing on one parameter A only:

a = A, 15 = A '
-1
y o, e z. (2)

To a function f(g) on G (e.g. with a compact support) inte-

grals along horocycles f(A,a,b) are assigned. The study of
unitary representations of SL(2;C) is essentially equivalent to
the study of the integral transformation f + f, and obtaining
an analogue of the Plancherel formula is equivalent to the inver-
sion of this operator. It turned out that it is possible to
produce an explicit inversion formula which is of the form

f(g) = J LLfw,
Y(g)

where L is a first order differential operator, Y(g) is the

set of horocycles passing through g (dimcY(g) = 1), and where
w is the canonical measure (it is convenient to asssume that LL
maps the space of functions on the big manifold into the space of
( 1 , 1 ) -forms on Y(g) ) .
Computations in [3] follow exactly this scheme, though
geometric language is not used. A reformulation of certain
important problems of representation theory of complex semisimple

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 77

Lie groups in the language of integral geometry is contained in

[1]. Let G be such a group, and let Z be a maximal unipotent
subgroup. The principal role of Z in representation theory has
already been made apparent earlier, but a number of facts become
more transparent after the shift to geometric language. Let X
be a homogeneous manifold of G; orbits of Z and of its con-
jugate subgroups will be called horospheres on X. Then in a
number of important cases the problem of decomposition of the
representation in the space L2 (G) into irreducible representa-
tions is effectively solved by studying the integral transforma-
tion f + f, assigning to a function on X its integrals along
horospheres. At least this is the case when X coincides with
G (the Plancherel formula for G) or is a symmetric space G/U,
where U is the maximal compact subgroup. The inversion formula
in these cases is written explicitly and preserves the structure
first noted for SL(2;C), only the differential operator L
becomes an operator of high order. Having in mind what will
follow let us underline that the inversion formula is local: to
recover the function at ~ point ~ must know only integrals
along horospheres close to this point.
It turned out that the relation described above between the
representation theory of complex semisimple Lie groups and inte-
gral geometry is not extendable to the case of real semisimple
Lie groups. Here even for the case of the whole G the hero-
spherical mapping f + f has a kernel decomposable into repre-
sentations of discrete series. On the symmetric space G/U (U
being the maximal compact subgroup) the horospherical mapping has
no kernel but (as the example of the Lobachevski plane already
shows) the inversion formula is generally speaking not known

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 78 S. G. GINDIKIN

locally. For concrete spaces it can be found nevertheless in

explicit form though its general appearance is not sufficiently
clarified (4]. It seems plausible that certain essential connec-
tions of integral geometry with real groups remain to be
uncovered.
It became clear during the study of problems of integral
geometry that problems of this kind had already arisen earlier.
First of all there is the Radon problem on restitution of a
function in Rn from its integrals along hyperplanes and an
earlier Minkowski-Funk problem on restitution of an even function
on a sphere from the integrals along big circles. However, these
problems are equivalent in the framework of the projective
version of the Radon transformation (5].
The Plancherel formula for complex semisimple Lie groups
belongs to comparatively rare situations when a many-dimensional
analytical problem is solved exactly. In this connection it
would have been natural to investigate the structures that enable
this possibility. First of all a homogeneous structure is con-
spicuous and it is already customary that the existence of other
explicit analytical formulas are due to a group symmetry, e.g. in
the special functions theory. Integral geometry on homogeneous
manifolds had collected by now a number of interesting results.
But is, in fact, a homogeneous structure really necessary for
obtaining results like the Plancherel formula for groups or
equivalently inversion formulas for horospherical mapping?

3. ADMISSIBLE COMPLEXES OF LINES. Even the study of the first

example connected with SL(2;C) hints that the group structure
may be of no importance. The fact is that SL(2;C) can be

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 79

perceived as a hyperboloid G- {(a,e,Y,6): a6- eY • 1} in c4

and horocycles are just its linear elements. We may try to take
this geometric picture as a basis for geometric generalizations
(and we will return to this possibility) but formulas (1) hint at
still another, at first glance more simple, way to generalize.
Consider the projection G + C3 , (a,e,Y,6) + (a,Y,6). For Y I 0
the preimage of each point consists of one point; for Y o the
image consists of the hyperbola {a6 = 1} and the whole line is
projected onto each of its points. We can assume that G is
obtained from a a-process that glues in the surface {Y = 0}
instead of the hyperbola. The generators (elements) of the form
(1) pass into lines that intersect the hyperbola and all such
lines arise in this manner. The generators (2) pass into points.
As a result the following reformulation of the integral
geometry problem arises. We have to recover a function in c3
(of class C~) if its integrals along all lines intersecting a
hyperbola are known. Is it perhaps possible to replace a hyper-
bola by another algebraic curve? A. A. Kirillov showed that, if
integrals along all lines in en intersecting an algebraic curve
are given, then there exists a local formula which recovers a
function. Notice that this family of lines depends on n
parameters.
The dimension of the manifold H of all lines in en
equals 2(n-1). It is customary to call submanifolds of dimen-
sion n complexes. Consider the integral transformation f + f
assigning to a function f(x) its integrals along lines. A
complex K c H is admissible if f can be recovered from the
restriction of f onto K via a local inversion formula. It
turned out that in c3 in addition to complexes of lines

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 80 S. G. GINDIKIN

intersecting a fixed curve there are also admissible complexes of

lines which are tangent to a fixed surface and that there are no
other admissible complexes [6]. I. M. Gel~and and M. I. Graev
[7] showed that admissible complexes of general position are
defined by the conditions that they are tangent to a certain
number of hypersurfaces and intersect a certain number of sur-
faces of codimension 2. K. Malyusz [8] has considerably weakened
the requirement for general position; in his case admissible
complexes are also defined by tangent-intersection conditions but
they can be interlaced in a quite complicated fashion.
Functions f which belong to the image of the trans-
formation f + f satisfy a system of differential equations
with constant coefficients. John [9] noted that if one
integrates a function f(x, ,x2,x3) defined in n3 along lines
x 1 = a 1x3 + s1 , x2 = a 2x 3 + s2 then the resulting function will
satisfy the ultrahyperbolic equation

!1u 0, !1 (3)

and any solution of (3) belongs to the image of the map f + f.

In c3 the image is described by the system !1u = 0, 6u 0
(integration is performed along holomorphic lines). In the multi-
dimensional case the image is described by the system of these
equations in each pair of indices. The problem of inversion of
the operator f + f can be interpreted as a boundary value
problem for this system and admissible complexes are character-
ized by the fact that for them the "Goursat problem" is solvable:
a solution is recovered from its restriction onto K (without
derivatives), and a certain locality condition is verified.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 81

Let us elucidate how the problem of description of admissi-

ble complexes is solved for the example in c3 . The admissibil-
ity condition is written as a system of non-linear equations
which can be solved exactly. Following Plucker, one can consider
a manifold of lines in CP 3 (it is convenient to pass to the
projective point of view) as a non-degenerate quadric Q in cP 5
(in Plucker coordinates). On Q, there is a canonical flat
conformal structure: isotropy cones vq are defined by inter-
sections of tangent planes with the quadric Q. If q 6 Q, then
Vq is the tangent cone to the canonical submanifold of lines
intersecting the line q (its equation in local coordinates (3)
is da 1 ds 2 - da 2 ds 1 = 0). On Vq, there are two families of
two-dimensional planes parameterized by points of a projective
line. Planes from the first family a(A) = {A 1 da 1 + A2 ds 1
= A1 da 2 + A2 dB 2 = 0} are tangent to the submanifold of lines
passing through fixed points of the line q and planes from the
second family B(v) = £~ 1 da 1 + v 2 da 2 = ~ 1 ds 1 + ~ 2 ds 2 = 0} are
tangent to submanifolds of lines belonging to fixed planes in
cp3 containing q. The study of all possible local inversion
formulas shows that. the admissibility of K c Q is equivalent to
the fact that tangent subspaces TqK for all q 6 K contain a
plane from the famuly B(v). Clearly this is a non-linear dif-
ferential equation for K.
Further, it is a simple fact from linear algebra that the
condition Tq K => B(~) for some ~ is equivalent to the condi-
tion TqK :::> a( A) for some A. Notice that dim TqK n vq -2 and
for an admissible complex the intersection consists of the pair
of planes a(A 0 , BCv 0 ) intersecting along a line. Roughly
speaking, the admissibility condition is that for generic lines

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 82 [Link]

q 6 K and for almost all points z 6 q lines from the complex

passing through z belong infinitesimally to the same plane
passing through q. This is in a sense the maximal degeneracy
condition. It had been encountered in classical differential
geometry (complexes with the parameter of rotation being identi-
cal zero [10]). This condition turns out to be equivalent to the
condition that K infinitesimally contains all lines passing
through a point z 6 K.
As a result, for an admissible complex a (critical) point is
fixed on a generic line. Critical points fill in either a curve
r or a surface S. We investigate separately the behavior of
the line~ from the admissible complex with respect to r or S.
Here we encounter a remarkable manifestation of the projective
duality principle: the condition of infinitesimal degeneracy of
the complex of lines is equivalent to the global condition of
singular behavior of lines of the complex with respect to a
certain set (of critical points) in the projective planes.
The study of admissible complexes in cp3 does not require
specifically new analytical methods: the classical Hamilton-
Jacobi method is enough. The reason is that the admissibility
condition described above is a kind of a certain Hamilton-Jacobi
equation on Q, i.e. H(q; grad u(q)) = 0. In fact, at each
point q 6 K one condition is imposed on the tangent space TqK.
Rays (projections on Q of bicharacteristics) for this equation
are 0-geodesics for the conformal structure on Q. They are
parameterized by flags (point and a plane containing it) in CP3.
A ray consists of lines q 6 Q passing through this point and
belonging to this plane. In other words, the set of character-
istics is identified with PT*(CP3). The subset of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 83

bicharacteristics into which the set of solutions of the equation

H(q; grad (u)) = 0 (an admissible complex) is fibered should be a
conic Lagrange manifold in T*(CP3) with respect to the canon-
ical symplectic structure. As is known, they are defined either
by a curve r or by a surface S in CP3 and consist of pairs
(z, 1). where z e r (resp. z e S), 1 e T*z (CP3) • 1 ( f,;) - 0 for
any F.: e Tzr (resp. TZS). We drop trivial solutions corre-
sponding to points of CP3. The general problem ~n admissible
complexes in en requires a slight modification of the Hamilton-
Jacobi method.

4. Hamiltonian families of lines. It seems to us that integral

geometry is not the unique source of problems leading to admiss-
ible complexes of lines. Here is another example of such a
problem. Let H{x,p) be a Hamiltonian on a3 homogeneous in
p. One can ask for which H are rays (projections onto a3 of
solutions of the system x -
:m
ap• p = -
aH
ax belonging to the
surface H = 0) straight lines. This is equivalent to the fact
that any solution of the equation H(x;grad u(x)) = 0 is a
ruled surface. It turns out that in this case the manifold of
rays is necessarily an admissible complex of lines and, converse-
ly, any admissible complex is a set of rays for a Hamiltonian.
If K consists of lines intersecting the curve r, then the
equation H(x,p) - 0, for a fixed x, defines director vectors
of planes passing through x and containing a tangent to r. If
K consists of lines tangent to the surface S, then tangent
planes to S passing through x are similarly defined.
Let us clarify this statement. It turns out that (three-
dimensional) manifolds K of curves which can be realized as

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 84 S. G. GINDIKIN

manifolds of rays of some Hamiltonians admit the following char-

acterization. If q 6 K and Kq is a two-dimensional sub-
manifold of rays intersecting q in generic points, then Kq is
smooth at q. In other words Kq 's are a priori conic at q

and it is required that the tangent cone to Kq in TqK be a

plane. The discussion above implies that this condition is
equivalent to admissibility of K when K consists of lines.
The usage of the manifold of characteristics with a special
infinitesimal structure in it is instructive in this argument.

5. Admissible manifolds of curves. In what follows we may make

a digression from the historical sequence of events and tell how
the passage from integral geometry for lines to the integral
geometry for curves is performed. Several examples of manifolds
of curves possessing a local inversion formula appeared in con-
nection with homogeneous manifolds. Now we are interested in
arbitrary n-parameter families of analytical curves X on a
complex manifold M of dimension n for which a function f on
M (of class C~) can be recovered via an inversion formula from
integrals of f along curves of the family X with respect to
some densities. These families will be called admissible and it
turns out that they admit a sufficiently effective description.
This description is obtained in two steps. The first step
has been performed in [11] and in this step we find out what form
a local inversion formula can have on a manifold of curves. It
turns out that it is a restriction of an explicit universal
(1 ,1)-form on the (infinite dimensional) manifold of all curves
and densities on them. Before we pass to precise formulations
let us make a convention that we will be interested not only in

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 85

inversion formulas on n-dimensional manifolds of curves K (com-

plexes) but also inversion formulas on manifolds of higher dimen-
sion k > n. It will later be clear from what we mean by an
inversion formula in this overdefined situation.
Let K be a complex manifold of analytical curves E on M
endowed with holomorphic parametrizations and densities;
(E,w) 6 K, dimcK 3 k > n = dimcM· One and the same curve E can
enter K with different densities. We will identify tangent
vectors to K with variations <~E.~w>. where ~E is a varia-
tion of E in K which is identified via a parametrization with
a section of
™IE (the restrfction onto E of the tangent
bundle to M); ~w will be interpreted as a density on c1

(once more using a parametrization).

Consider the integral transformation I
the form

rrcE,w> f fw,
E
f 6 C~(M), (4)

i.e. the integral of f along E with respect to w. Let Kz

be a submanifold of curves in K passing through z 6 M. We
will consider differential operators of biorder (1 ,1)

acting from the space of smooth functions on K to the space of

(1,1)-forms on Kz and depending on z as on a parameter.
The operator P is admissible if for almost any z 6 M
(i) dPI = 0, i.e. the image of I is transformed by P
into the set of closed forms;
(ii) there exists an operator L defined on the subspace of
functions in C~(M) which vanish at z and acting on the space

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 86 S. G. GINDIKIN

of 1-forms on Kz such that Pif = dLf if f(z) = 0, i.e. i f

f(z) = 0 then Pif is an exact form.
It turns out that if dim K > dim M, then (i) implies (ii).
given an admissible operator P and a two-dimensional cycle Y
in Kz we have

y
f Pif c(Y)f(z), ( 5)

where c(Y) does not depend on f. If c(Y) I 0 for a cycle

Y, then we recover f(z) and if this is true for all z eM,
then we get an inversion formula. The notion of an admissible
operator is an axiomatization of the structure which is natural
to suppose to be intrinsic for the local inverting operator for
the transformation f + If. The problem of description of admis-
sible operators is sapid only when there are no cycles such that
c(Y) ~ 0 (or there are no cycles at all) and an inversion
formula is impossible to obtain. This is the problem of a formal
integral geometry.
As is shown in [11] admissible operators are parametrized by
functions vE,~,z(w) on curves of K depending also on a fixed
point z e E holomorphic outside of z and having in z a
first order pole. The main restriction on v is that the resi-
due of vE,~,z~ at z should depend only on z (and does not
depend on E and ~). Denote the residue by c( z), z e M. The
admissible operator p corresponding to v is of the form
v
where
(6)
F e c"'(K)

where v~ is considered as a mapping from K to the space of

densities on c1 and d(v$) is its differential.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 87

One sees, using proportion of ~. that definition (6) is

correct since 6E(z) s 0 on Kz. It is not possible, generally
speaking, to define an operator of the form (6) on K since the
vector ~ = (s~6E,d(~E)) will not be tangent to K. the mani-
fold K is admissible if there exists a family E,ljl,z with
~

c(z) ~ 0 such that ~ = (~6E,d(~lji)[6E,61jl]} is tangent to K,

i.e. it is possible to define an operator of the form (6).
On an admissible manifold K, if Kz contains appropriate
cycles, there exists a local inversion formula If + f.
It turns out that the existence of the family ~ such that
K is admissible imposes extremely rigid constraints on K (and
in case of global curves on individual curves also). These con-
straints are so strong that it is possible to describe admissible
manifolds of curves. The description had been obtained by J.
Bernstein and the author [12, 13]. The main fact is that func-
tions ~E ,,,
,.,..,z define on each curve E 6 K the canonical struc-
ture of the projective line. Moreover, after a suitable modifi-
cation of M, the admissible family of curves in a neighborhood
of each of its curves E turns into a complete family of rational
curves. It means that all sections of the normal bundle to E
are obtained from curves of the family K (all infinitesimal
deformations of E are extendable up to global ones). We have
already seen above an illustration of this possibility. Lines
intersecting a hyperbola do not constitute a complete family of
rational curves in CP 3 (the manifold of all lines is such a
family). However, after a birational mapping of cp3 to the
hyperboloid G we get already the complete family of elements on
G. Conversely, any complete family of rational curves is admiss-
ible. Thus, the problem of admissible manifold of curves in the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 88 S. G. GINDIKIN

~of integral geometry is equivalent to the problem of com-

plete families of rational curves.
Let us emphasize that the property of the families of curves
K on M to be realized as a complete family of rational curves
on a certain manifold M is defined by the structure of the
family in a small neighborhood on M. Namely, in analogy with the
conformal structure on the manifold of lines Q induced by the
intersection relation, let us, in the general case, consider on
K for the curve Eq' q e K, the set Kq c K of curves
Er, r 6 K, intersecting Eq. Let Vq c TqK be the tangent cone
to Kq. The main condition for the existence of a "rational"
realization of K is the requirement that cones Vq should be
linearly equivalent to one of a finite number (for a fixed dimen-
sioh of K) of canonical cones described explicity. This is a
simple reformulation of the Grothendieck theorem on the structure
of vector bundles on the projective line. As a rule, this condi-
tion is also sufficient for existence of a realization. The
exception is the case when in the first infinitesimal neighbor-
hood of the curve all curves that intersect it belong t~ a two-
dimensional surface (the normal bundle is 0(1) + 0 + 0 + , •• + 0).
In this case the family is realized only as a family of locally
rational curves, and to extend the family up to a complete family
of global rational curves a fulfillment of a condition on the
second infinitesimal neighborhood of the curve (a variant of the
classical Desargues axiom from the projective planimetry) is
needed. In ~he general case this condition is automatically
satisfied (the generalized Desargues theorem).
A question arises: how can one construct examples of com-
plete families of rational curves? A more precise formulation of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 89

this question is as follows. Suppose we are given a complete

family of rational curves, say a manifold of all lines in cp3
(4-parameter) or the manifold of all second order curves in CP 2
(5-parameter) or the manifold of second order curves in cp3
(8-parameter). What admissible submanifolds do they have? They
will not be complete manifolds of rational curves on the initial
manifold M but can become such after a suitable sequence of
a-processes (as was the care during the passage from cp3 to the
hyperboloid G) or on a ramified covering of M. We know al-
ready the answer for an admissible submanifold of lines. It
turns out that the structure of the answer is similar in the
general case, also.
In general position admissible submanifolds of a complete
manifold of rational curves are defined by tangency conditions to
a set of submanifolds s 1 , .•• ,Sr of codimension and by
in~ersection conditions with certain submanifolds r 1, 'r s '·
codim
.
r.J = 2. We have already encountered special cases of this
statement in the description of admissible complexes of lines.
Another example, a general two-parameter family of second order
curves on the plane for which the problem of integral geometry
has a local inversion formula, is singled out among all curves by
these conditions which state that either curves of the family are
tangent to a fixed curve or that all of them pass through a fixed
point. So the family of all circles tangent to a given curve is
admissible (recall, that circles are characterized by the fact
that they pass through a pair of cyclic points). At the same
time for the family of curves of fixed radius there is no local
inversion formula. Here we deal with the complex version of the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 90 S. G. GINDIKIN

notion of a circle (e.g. it is a subfamily of conic sections

passing through a pair of cyclic points).
Let us indicate briefly how to formulate results without the
assumption of general position. For a manifold M to which
rational curves of a complete family K belong we construct a
tower of a-processes M + M such that at each step instead of
submanifolds of different codimensions submanifolds of codimen-
sion are glued in. Furthermore, on M, one fixes
submanifolds s 1 , •.• ,sr, codim Sj = 1, and positive integers
j 1 , ... ,jr. The curves of the family K which are lifted .onto
M and are there tangent to the submanifold St of order jt
are considered. This is the general form of admissible subfam-
ilies in K. Notice that even for the description of admissible
subfamilies of lines in CPn we need to go beyond the limits of
CPn and consider rational curves on its modifications. The
general answer is obtained only in this more general realm. To
trace all possible combinations of intersections and tangencies
directly in CPn without applying the language of a-processes is
quite difficult.

6. Analytical problems connected with manifolds of rational

curves. The problem of definition of manifolds of curves for
which the problem of integral geometry admits a local inversion
formula is not a unique analytical problem leading to complete
manifolds of rational curves. On the contrary, this situation
is perhaps a fundamental one in exactly solvable analytical
problems.
One more example is given by a family of curves on three-
dimensional manifolds which are families of rays for some

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 91

Hamiltonians (p~ojections of ze~o-bicha~acte~istics). Such fami-

lies will be called Hamiltonian ones. Above we have conside~ed

Hamiltonian families of lines. The discussion above conce~ning

Hamiltonian families of cu~ves implies that on cu~ves f~om a

Hamiltonian family the~e is always a canonical st~uctu~e of

locally p~ojective cu~ves and, if a natu~al condition ("infini-

tesimal Desa~guesness") is satisfied, they a~e canonically ex-

tendable to a complete family of global ~ational cu~ves. When

the Hamiltonian is quad~atic in momenta this additional condition

coincides with the condition of confo~mal flatness of the met~ic.

In pa~ticula~. we can desc~ibe Hamiltonian families of second

o~de~ cu~ves in cp3 (the infinitesimal Desa~guesness condition

will be automatically ve~ified fo~ them). These families a~e

distinguished in the gene~al position by 5 conditions of inte~-

section-tangency. A desc~iption of the co~~esponding

Hamiltonians is equivalent to the p~oblem of the desc~iption of

Hamilton-Jacobi equations H(z;g~ad u(z)) = 0; the set of its

solutions is fibered into conic sections (rays).

Manifolds of ~ational cu~ves a~e also in close connection

with nonlinea~ equations which are integ~ated by the inve~se

scatte~ing p~oblem method. Most often these equations a~e w~it-

ten as a compatibility condition of a linea~ system of fi~st

o~de~ differential equations with a ~ational pa~amete~. To each

such linea~ system we may assign a complete family of ~ational

cu~ves. Thus Penrose [14] associates with each selfdual con-

fo~mal 4-met~ic the family of ~ational cu~ves depending on 4

pa~amete~s on a th~ee-dimensional manifold (the no~mal bundle is

0(1) lil 0(1)). This is the construction of a nonlinea~ twisto~

manifold [14]. The construction of such families of ~ational

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 92 S. G. GINDIKIN

curves is the main step in the Penrose construction of a right-

flat metric (self-dual solutions of the vacuum Einstein equa-
tion). J. Bernstein and the author applied the above tangency-

intersection constructions to construct right-flat metrics, in

particular, on certain 4-dimensional submanifolds in the manifold
of conic sections in CP3 [12].
The twistor construction of Penrose adm·its a generalization
onto nonlinear equations of a sufficiently general form inte-
grated by the inverse scattering problem method. Let A(t) = 0
be a linear system on a maniflod K that has indefinite coef-
ficients with a rational parameter t. We consider a nonlinear
equation for the coefficients of the system which states that the
system is completely integrable for any t. Each integrable
system A(t) = 0 induces a solution of a nonlinear equation, If
we consider the manifold M of integral manifolds of the linear
system, then to a point q e K on M there corresponds a
rational curve parameterizing integral surfaces through q, As a
result the construction of integrable systems with a rational
parameter is equivalent to the construction of certain systems of
rational curves [13, 15].
Let us make a preliminary summing up. It seems to us that
results concerning integral geometry for manifolds of curves are
quite final. It is clear today that the property (noted by I. M.
Gelfand and M. A. Naimark) of generators (elements) on a hyper-
boloid in c4 (that there is an explicit local inversion formula
of the corresponding problem of integral geometry) is generalized
onto complete families of rational curves on an algebraic mani-
fold. Such a family can appear in a somewhat distorted shape due
to an unlucky modification of the manifold (as it happened with

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 93

lines intersecting the hyperbola). It is remarkable that this is

the most general situation where the local inversion formula
exists.
Very little is known about similar problems of integral
geometry for families of submanifolds of higher dimension. Let
us mention the paper [16] where it is shown that under certain
conditions in the many-dimensional problem of integral geometry
the existence of the local inversion formul leads with necessity
to infinitesimal structures on a family of submanifolds similar
to the structure which manifolds of rational curves possess ( the
P-structure [12]).

Integral geometry for planes. Even for integral geometry

connected with integration along k-planes for k > 1 much is
unknown. Let us consider the real problem. Assign to
f(x) 6 S(Rn) its integral f(h) along a k-plane h. Let
H = Hn,k be a set of k-planes, Hx be the set of planes through
x. The simplest case (k = n- 1) is the Radon transformation;
there exists a local inversion formula for odd n and a nonlocal
one for even n.
For k <h - the inversion problem becomes overdetermined
since dim H > n. In [17] the following remarkable method to
scan all possible inversion formulas is described (the descrip-
tion of local inversion formulas for complex curves given above
followed this pattern). There exists an explicitly described
differential operator P: C~(H) + nk(H ) acting from the space
X

of functions on H into the space of k-forms on Hx such that

the image of the operator f + f coincides with the set of
preimages of closed forms on Hx with respect to P. If Y is

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 94 S. G. GINDIKIN

a cycle in Hx, then J f(h) = c(Y)f(x). for odd k we have

y
c(Y) = 0 for all Y and for even k there exist cycles Y
such that c(Y) I 0 and we get an inversion formula.
Let us give the corresponding formulas in the projective
form. Notice that the map f + f, in particular the Radon
transformation, is very close to the Fourier transformation.
However, the map f + f possesses one important distinction: it
can be invariantly connected with thr projective structure, not
only with the affine one (like the Fourier transformation).
Projective formulas have certain advantages.
Let x = (x 0 , .•. , xn) and I;= (1; 0 , ,n) be homo-
geneous coordinates on the projective space pn = RPn and its
X X
dual
between and We will integrate sections of the one-
dimensional bundle 0(-n) over Pn
x (i.e. just homogeneous func-
tions f(xo, .•. ,xn) of class c~ of degree -n) along planes
in P~. In the affine chart x0 = 1 we can deal with ordinary
functions f(1 ,x 1 , ... ,xn). It is useful to also consider
other one-dimensional bundles O(s) but we will not give here
corresponding formulas. For simplicity's sake we will assume n
to be odd. Denote by [a 0 ,a 1 , ... ,an] the determinant of the
matrix with columns a 0 , ..• ,an (notice that among aj's there
can occur columns of 1-forms which skew commute; in particuilar
the presence of several copies of the same column does not imply
vanishing of the determinant; $[~] means that the column $ is
repeated i times).
First of all let us express the projective-invariant from of
the inversion formula for the Radon transformation (n is odd):

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 95

f(y) 1- . - -
= ----'- - J J f(x)[x,dx[n]][E,;,df,;[n]]
4(2ni)n- 1 n!(n-1) P~ P~
(7)
x d(<E,;,x>)o(n- 1 )(<E,;,x>)

It goes without saying that this integral can be restricted onto

the quadratic {<E,;,x> =} in (the set of flags con-
sisting of a point and a hyperplane passing through it). Actual-
ly we have passed in (7) to the "residue" on this quadric. We
have

f( x)[x, dx[n]]
l<p,x>l
(8)

where <p,x> F 0 and P~- 1 (x) is the subspace of E,; such that
<E,;,x> ~ 0. The integrand considered on {<E,;,x> = 0} does not
depend on p. If an affinization of is fixed and the sup-
port of f belongs to art affine chart, then p can be chosen so
that {<p,x> = 0} is the infinite plane. It is characteristic
that to express the formula on the quadric an additional struc-
ture is needed. If for f e O(s) and s = -n the integrand
does not depend on p then the dependency on p of the corres-
pending form for s F -n is essential, although the integrand
accrues by an exact form after a variation of the field p(x,E,;).
Formulas (7) and (8) can be considered as an expansion of
the &-functions o(x 1 ) .•• o(xn) in Rn (the chart x 0 = 1)
with respect to plane waves o(n- 1 )(<E,;,x>). This Radon expan-
sion is actually equivalent to the known Feynmann formula for the
decompostion of (x 1 ... xn)- 1 with respect to <E,;,x>-n which,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 96 S. G. GINDIKIN

however, had been obtained earlier by Fantappie in his study of

analytical functionals.
Let q :> n-1 be even. We will define a q-plane in pn by
X
means of an (n-q+1)-frame Tl = in P n and a
c,;
vector r = Namely, a q-plane L(q,r)
consists of x e Pn such that <n,x> = cr for some c e R. Set

f(n,r) J f(x)<Z";,dx> ( 9)
L(n,r) [n, r;J

where <r;,dx> =A <r;j,dx> and r; is a q-complement to n; (9)

does not depend on the choice of r;. However, f depends not
only on L(n,r) but on n also (we may assume that on L(n,r)
an "infinite" (q-1)-plane M(n) = {x : <nj,x> = 0, j :> n-q} is
fixed).
We have

f(n,r)
4(21ri)n-q- 1 (n-q-1) !(n-q-1)
( 1 0)

x Jn Jn-q-1
P P (x n)
X Z '

n-q-1
where
<c;,x> =
<p,x>
0 and
" 0 and
c,; = Aon
Pc,;
0 +
(x,n) consists of c,; such that
...
+ An-qn n-q for A s pn-q. further .
y e L(n,r) is fixed so that <n,y> = r, i.e. <c;,y> = (A,r>
+ • • • + Formula (10) is obtained by applying
(8) in the quotient space with respect to M(n).
It is easy to see that (8) is obtained from (10) by applying
the differential operator

P =
n-q-1
- [ n, {a1 ar, dy J [q] J ( 11)
( 21fi)q(n-1) !q!

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 97
a .
where {a/ar,dn} L ---. dnJ, transforms the function
O::>j ::>n-q ar 1
depending on r to a column of 1-forms which should be repeated
in the determinant q times. One should have in mind that if
~ L Aini, <n,y> = r then <~,y> = <A,r> and {a/ar,dn}<~,y>
= L Aini • d~ modulo the space spanned by {nil and also that

(n-q-1)!q! c~.p,d~[n-1]] ( 1 2)
(n-1)! " "

Suppose there is a map assigning to ~ 6 P~- 1 (y) a plane

L(n,r) 9 y contained in {x:<~,x> = 0}. Let Y be the image of
this map. Then

JP f(n,r) = f(y). ( 1 3)
y

We consider formulas containing integration along other cycles.

In the similar problem on CPn where one integrates along ana-
lytic planes all arguments are similar but in the inversion
formula one should take the operator PP (here one must not
bother any more with the parity of the complex dimension). In
[17, 5] the operator P is given in coordinates with respect to
an orthogonal frame when a q-plane is defined by its q + 1
points.
Let K be a complex (n-dimensional manifold) of q-planes
L(n,r), and let Kx be the submanifold of planes through
X 6 Pn. The complex K is admissible if P<PIK can be re-
X
covered from cp IK for almost any X 6 Pn. If we do not come
across a degenerate situation, then from an admissible complex
one can recover an inversion formula (at least in the complex
case). We have already encountered this notion in case of com-
plexes of lines where it is possible to get a complete

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 98 S. G. GINDIKIN

description of admissible complexes. In case of complexes of

q-planes for 1 < q < n-1 only several examples of admissible
complexes are known. The complex of horospheres in GL(k;C) is
among them, thanks to which it is possible to obtain the
Plancherel formula by means of integral geometry [18]. Our
knowledge of the general structure of admissible complexes is
scanty though the problem of their description or at least exhib-
iting sapid series of examples seems to be interesting and far
from hopeless.
Returning to the real case, recall that we have restricted
ourselves to the case of odd n and even q. The first
restriction is not principal: it suffices to replace everywhere
~(k- 1 )(<~,y>) by (<~,y>- iO)-k, where (t- iO)-k = t-k
. k-1
+ ln(- 1 ) ~(k- 1 )(t) and either even or odd summand will work
k-1
depending on the parity. On the other hand, the evenness of q
is the principal constraint. Otherwise the passage from an
analogue of (10) to the analogue of (8) is given by an operator
which is non-local in r. It is not difficult to compute ex-
plicitly the corresponding operator. The non-local analogue of P
which arises and transforms functions to densities had been
considered in [20] (see also [21]). One cannot but have a feel-
ing that the situation with non-local formulas in the real inte-
gral geometry still requires serious elucidations.

8. Complex integral geometry. Though we have already discussed

problems of integral geometry in complex spaces en, CPn they
were essentially problems which make use only of the real struc-
ture of the space and we have integrated functions smooth in the
real sense. Now let us pass to formulations which actually make

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 99

use of the complex structure. Here is one of the possibilities.

We have already discussed the John presentations of solutions of
the ultrahyperbolic equation (3) in the form of integrals along
lines in R3. Is there an analogous presentation for other
signatures: for the wave equation or for the Laplace operator in
R4? The history of geometry [22] has a parallel scheme: as we
have already recalled Plucker had found out that the manifold of
lines in RP3 is canonically isomorphic to the quadric of signa-
ture (3,3) in RP 5 • Klein and Lie found out geometric realiza-
tions for quadrics with other signatures in RP 5 • For example,
the geometry of Lie spheres corresponds to the signature (4,2).
These geometric examples have a direct bearing on questions we
are interested in.
In the framework of his twistor program Penrose proposed to
realize real quadrics in RP 5 as certain submanifolds of complex
·lines in cp3 in accordance with the embedding of real quadrics
in the complex quadric lines Q c CP 5 • Then considering a real
hypersurface H :

( 1 4)

in CP3 with homogeneous coordinates (z 0 ,z 1 ,z 2 ,z 3 ) we see that

it contains a family of complex lines M depending on 4 real
parameters. Corresponding points on Q fill in exactly the real
quadric of signature (4,2). On M a conformal flat metric of
signature (3,1) is induced so that M can be interpreted as a
conformal compactification of the Minkowski space.
It is natural to try to connect with this construction a
problem of integral geometry leading to the wave equation.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 100 S. G. GINDIKIN

The most delicate moment is the choice of an analytic object

to be integrated along lines on H. Modifying somewhat the
initial ideas of Penrose [23] let us consider (0,1)-forms on H
with coefficients in 0(-2) (i.e. in homogeneous coordinates
homogeneous forms of bidegree (-2,0)) and satisfying the tan-
gential Cauchy-Riemann equation ab¢ 0 on H (with respect to
the CR-structure induced on H). If (t 0 ,t 1 ) are homogeneous
coordinates on the line Eq, where q 6 M, we can integrate
$A(t 0 dt 1 -t 1 dt 0 ) along Eq. Considering integrals along all
lines Eq, q 6 M, we get on M a solution of the wave equation
and all solutions on the whole M are obtained this way.
Similarly, if we consider lines in CP3 passing through
pairs of points of the form (z 0 ,z 1 ,z 2 ,z 3 ), (-z 1 ,z 0 ,-z 3 ,z 2 ),
then on Q a real quadric of signature (5,1) is distinguished,
on which a conformal-Euclidean flate metric connected naturally
with the Laplacian has been induced.
It is somewhat simpler to consider not the integral trans-
formation of boundary Cauchy-Riemann cohomology on H but
a-cohomology in domains H± distinguished by H(z) ~ 0, where
H is the form (14) (Penrose had actually studied this very
transformation). Lines belonging to H± fill in on Q two
domains T± which are Stein manifolds, and M is their common
skeleton. Domains T± are biholomorphically equivalent to
future (past) tubes, i.e. to radial tubular domains whose cones
consist of one branch of the light cone. Integrating a-closed
(0,1)-forms with coefficients in 0(-2) along lines in H± we
get solutions of the wave equation holomorphic in T±. Inf this
process only a-exact forms vanish, i.e. H(0, 1 )(H±;0(-2)) is

isomorphically mapped onto the space of holomorphic solutions on

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 101

the wave equation on T±. General solutions on M are presented

as jumps of holomorphic solutions in T±. It is worthwhile to
support these arguments with explicit formulas which we will
produce in a more general situation.
A domain V c CPn is called (n-q)-linearly concave if
through each of [Link] a q-plane passes which is entirely
contained in V. Suppose, in addition, V possess a smooth
boundary and the domain CPn - jj is (q+1)-linearly concave.
Through each point of av an "inner" q-plane of support and an
"exterior" (n-1-q)-plane of support pass (non-unique!). Domains
H± are 2-linearly concave (q = 1).
According to the general conception starting from results of
Andreotti-Grauert-Norget the object of study for the complex
analysis in such a domain is q-dimensional cohomology with coef-
ficients in ~ coherent sheaf, since q is the unique dimension
in which the cohomology space is infinite dimensional. Let us
restrict ourselves for the sake of simplicity to cohomology with
coefficients in 0 ( -n).

Let f be a a-closed (O,q)-form with coefficients in

0(-n), i.e. f(tz,dz) = t-nf(z,dz). Consider analytic q-planes
L(n, r) belonging entirely to V and define the integral of f
along L(n,r) by the formula (9). We should only have in mind
that now f is a (O,q)-form and the integrand is a (q,q)-form.
A

Denote by V the set of (n,r) such that L(n,r) c V. Then we

get a holomorphic function f(n,r) on V. The image of the map
f + f is described by the same system of differential equations
as the image of the corresponding real transformation.
The complex analogue of (10) is the [Link] which recovers
f(n,r) from boundary values of f [25]:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 102 S. G. GINDIKIN

<n,w> r and the integrand does not depend on n

where a p 6 CPE;.
Thus p can be chosen arbitrarily taking into account the depen-
dence on z 6 aV, provided <p,z> ~ 0. Furthermore, for any
z s av an (n-q-1 )-plane ll.(z) 3 z which has support depending
smoothly on z in CPn-D is fixed, and then E;(z,n) is chosen
so that is a linear combination of no , • • • , nn-q and the
hyperplane {<E;,u> • 0} contains ll.(z). Finally, let a q-frame
in CP~ be chosen so that ll.(z) is defined by equations
<E;,u> = 0, <~,u> = 0. The formula (15) can be viewed as a
generalization of the Fantappie kernel in which the right-hand
side reduces to q • n-1 . The connection of the Fantappie kernel
with the complex integral geometry had been established in [24].
The proof of (15) by means of deformation of the contour is
reduced to the Leray-Martineau integral formula for linearly
convex domains [25].
A complex analogue of the passage from (10) to (8) is very
interesting. Let us formally apply the operator P = ___!!:.!_p to
n-q-1
( 15) (see ( 11 ) ) :

f(n,r) --------~~2 ________ J fA[z,dz[n]]

(2ni)n- 1 (n-1-q)!q! av <p,z><E;,w>n-q[~.n]
( 16)
A[E;,p,~,dE;[n-q- 1 ]JA[n,dE;[q]J

where <p,z> ~ 0, <n,w> = r and ~.~ are chosen as above. One

A

can show that f is a holomorphic closed q-form on V contain-

ing differentials only with respect to n (it is essential that
E; can be excluded from (16) and then [~,p.~,d~[n-q- 1 ] will

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 103

contain differentials only with respect to i(z) whereas

[n,d~[q]] will contain those with respect to n).
Let us consider the fibration V + V and let r be a
section. It turns out that

( 1 7)

is a a-exact form (we take the restriction of f onto r and

the (0,1)-component of the resulting form on V). In other
words, the form

( 1 8)

is a-exact. We thus obtain a method of recovering the cohomology

class of f from f. This inversion formula (18) had been
obtained in [24]. On the other hand, taking (17) into account,
formula (16) enables one to recover a form cohomological to f
in V from boundary values of f. We get an integral formula
for cohomology with holomorphic kernel: a holomorphic "continua-
A

tion" of the cohomology class onto a bundle V over V is

constructed.
The usage of the holomorphic form f instead of the a-
closed one, f, is very effective in various questions. For
instance; this is how the problem on extendability of the class
of f across av is solved in these terms. The class of f is
extendable across z e av if the holomorphic form f is extend-
able across the corresponding point of the boundary. In particu-
lar, if f has no singularities on the boundary, then f is
extendable across all points of the boundary. Notice that f is
a-exact if and only if f vanishes identically.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 104 S. G. GINDIKIN

The analogy between complex and real formulas of integral

geometry seems to be of principal nature and deserves a thorough
investigation. In a sense, real formulas can be considered as
boundary values of complex ones, and in a number of cases it is
really possible to get them this way.
Let us summarize. We wanted to give several characteristic
results in the field which 25 years ago I. M. Gelfand suggested
to call integral geometry. Today it would be difficult to dis-
tinguish its problems or to claim with assurance that we deal
with a mature trend in mathematics. As earlier, integral geom-
etry consists of a number of quite brilliant examples which
establish a certain pattern of situations. Their characteristic
feature is the presence of integral transformations with the
maximal employment of geometric structures on manifolds. Homo-
geneous structures connected with initial formulations of
problems constitute only one of the examples of such structures.
The arguments given above seem to illustrate sufficiently, how,
starting from problems on homogeneous manifolds, certain more
general (and now no longer homogeneous) geometric structures were
developed which are respnnsible for solvability of problems of
integral geometry.
From the analysis point of view a characteristic feature of
these examples is the nature of exactly solvable problems and
explicit formulas. In these problems it turned out to be equiva-
lent to locality of the formulas. Still another approach to the
formulation of problems of integral geometry is possible when, for
instance, one studies the integral representation f + f consist-
ing in integrating of functions along a family of submanifolds,
and one is only interested in the existence of the inverse

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 105

operators in the maximally general situation without attempts to

get an explicit inversion formula. These formulas are connected
with elliptic equations, integral Fourier operators, inverse
problems and much has been obtained this way in recent years.
But this is a different domain of problems. We wanted to stress
that the search for exactly solvable problems and the selection
of geometric structures responsible for their solvability are
very important features of integral geometry in which geometry
and analysis are interlaced in a non-separable embrace.

BIBLiOGRAPHY

1. I. M. Gelfand, M. I. Graev, "Geometry of homogeneous

spaces, representations of groups in homogeneous spaces and
related problems of integral geometry," I, Proc. Moscow Math.
Soc.,~ (1959), 321-390 (in Russian).
2. I. M. Gelfand, "Integral geometry and its connection
with representation theory," Russian Mathematical Surveys, 15
(1960), 143-151 (in Russian). -
3. I. M. Gelfand, M. A. Naimark, "Unitary representations
of the Lorentz group," Math. USSR Izv., Ser. Math., 11 (1947),
411-504 (in Russian). -
4. s. G. Gindikin, F. I. Karpelevich, "On a problem of
integral geometry," Collection to the memory of N. G.
Chebotaryov, Kazan (1964), 30-43 (in Russian), English
translation: "One problem of integral geometry," Sel. Math.
Sov., Vol. 1, _g_ (1981), 169-184.
5. I. M. Gelfand, S. G. Gindikin, M. I. Graev, "Integral
geometry in affine and projective spaces," Modern Problems of
Mathematics, 16 VINITI, USSR Acad. of Sci., Moscow (1980), 53-226
(in Russian).-
6. I. M. Gelfand, M. I. Graev, N. Ya. Vilenkin, "Integral
geometry and related problems ofrepresentation theory," Moscow
(1962) (in Russian).
7. I. M. Gelfand, M. I. Graev, "Complexes of lin~s in cn,n
Funct. Anal. Appl., _g_ (1968), 39-52 (in Russian).
8. K. Malyusz, "Structure of admissible complexes of lines
in cpn~n Proc. Moscow Math. Soc., 39 (1979), 181-211 (in
Russian). -

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 106 S. G. GINDIKIN

9. F. John, "The ultrahyperbolic differential equation with

four independent variables," Duke Math. J. i (1938), 300-322.
10. J. Favard, Cours de Geometrie Differentielle Locale,
Paris, Gauthier-Villars-(1957) .
. 11. I. M. Gelfand, s. G. Gindikin, Z. Ya Shapiro, "Local
problem of integral geometry in the space of curves," Funct.
Anal. Appl., .1J. (1979), 2, 11-31 (in Russian).
12. S. G. Gindikin, "Integral geometry and twisters," Lect.
Notes Math., 970 (1980), 2-~2.
13. S. G. Gindikin, "Reduction of manifolds of rational
curves and related problems of differential equations theory,"
Funct. Anal. Appl. ~ (198~). ~. 1~-39 (in Russian).
14. R. Penrose, "Nonlinear gravitons and curved twister
theory," Gen. Rel. and Grav., I (1976), 31-52.
15. S. G. Gindikin, "Manifolds of rational curves and non-
linear equations," Group Theoretical Methods in Physics, Nauka,
Moscow, £ (1983), 32~-332 (in Russian). English translation by
Gordon and Breech.
16. I. M. Gelfand, G. S. Shmelev, "Geometric structures of
double fibrations and their connection with several problems of
integral geometry," Funct. Anal. Appl., 17 (1983), 2, 7-22 (in
Russian). --
17. I. M. Gelfand, M. I. Graev, Z, Ya. Shapiro, "Integral
geometry on k~dimensional planes," Funct. Anal. Appl., 1 (1967),
1, 15-31 (in Russian).
18. I. M. Gelfand, M. I. Graev, "Complexes of k-dimensional
planes in en and Plancherel formula for GL(n,C)," Math. USSR
Doklady, 179 (1968), 3, 522-525 (in Russian).
19. I. M. Gelfand, G. E. Shilov, "Generalized functions and
actions over them," Fizmatgiz, Moscow (1958) (in Russian).
20. I. M. Gelfand, s. G. Gindikin, "Nonlocal inversion
formulas in real integral geometry," Funct. Anal. Appl. 11 (1977)
3, 12-19 (in Russian). --
21. I. M. Gelfand, M. I. Graev, R. Rochou, "Nonlocal inver-
sion formulas in a problem of integral geometry connected with
p-dimensional planes in real projective space," Funct. Anal.
Appl. _ll (1982), 3, 49-51 (in Russian).
22. s. G. Gindikin, "The complex universe of Roger Penrose,"
Math. Intellingencer, 2 (1983), 1, 27-35.
23. S. G. Gindikin, G. M. Henkin, "Radon transformation and
complex integral geometry," Modern Problems of Mathematics, ll•
Moscow, VINITI (1981), 57-111 (in Russian).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AS GEOMETRY AND AS ANALYSIS 107

24. S. G. Gindikin, G. M. Henkin, "Integral geometry for a-

cohomology in q-linearly concave domains in cpn,n Funct. Anal.
Appl., l l (1978), 4, 6-23 (in Russian).
25. s. G. Gindikin, "Integral formulas and integral geometry
fori-cohomology in cpn,n Funct. Anal. Appl., 18 (1984), 2, 26-3
(in Russian). -
MOSCOW STATE UNIVERSITY
BUILDING A
MOSCOW (GSP234119899) USSR

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS

ERIC L. GRINBERG

1. THE k-PLANE TRANSFORM. The classical Radon transform R

takes functions on X = R3 to functions on Y, the Affine
Grassmannian of lines in R3. If f(x) is a suitable function
on X and 1 is a line in Y then

Rf(1) L. f.

If 1 is not parallel to the xy plane, then we can parametrize

1 by

1(t) = (x(t), y(t), z(t))

where

x(t) =at + a. y(t) = yt + 6, z(t) = t.

Thus (a, B, Y, 61 are local coordinates for Y and

dim(Y) = 4. Since dim(X) = 3 = dim(Y) - 1, we do not expect R
to be onto. And indeed, in these coordinates we have

Rf(a,B,Y,6) = fteRf(at+B,Yt+6,t)dt,

and differentiation under the integral sign yields

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

109
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 110 ERIC L. GRINBERG

Conversly, Fritz John has shown ([7]) that this ultrahyperbolic

partial differential equation 'characterizes' the range of R.
This example is the starting point of several papers in Integral
Geometry (for example see [1], or [4]). Since the proof of
John's result is not usually presented, we begin by giving a
slightly different proof. In fact, we will consider an entire
class of Radon-like transforms simultaneously. As we shall see,
the precise meaning of 'characterizes' depends upon the class of
functions under consideration.
If we write z = A generic
affine k-plane in Rn can be parametrized as follows:

y Ax + c

where A is an (n - k) x k matrix and c 6 Rn-k. We define

the k-plane transform R from suitable functions f(x) on
X s Rn to functions on the Affine Grassmannian of k-planes in
Rn via

Rf(A,c) J k f(x,Ax+c) dx.

x6R

John's example above corresponds to the case k = 1. Following

Debriard and Gaveau [1] or I. M. Gelfand and his collaborators
(e.g. [2]) we define the partial Fourier transform

From now on we assume that f belongs to s, the Schwartz class

of rapidly decreasing functions.

1\
LEMMA 1. The integral defining Rf converges absolutely, and,
if f is the Fourier transform of f then

L\ t
= f(-A
A
Rf(A,~) ~.~).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 111

Here (-At~.~) 6 Rk x R(n-k) is viewed as a vector in Rn.

PROOF. lle -I~At~·x e~~~·y f( x, y) dx dy

JJe-/~At~·x e 1 ~~·(y+Ax) f(x,y+Ax) dy dx

JJe-l~~·y Jf(x,Ax+y) dx dy
JJe-~~~·y Rf(A,y) dy

~(A,~).

Absolute convergence follows from Fubini 1 s theorem. Q. E. D.

As an immediate consequence we see that R is inj~ctive on,
say, the class of rapidly decreasing functions: for generic
~ 6 Rn there exist A and ~ with ~ = (-At~.~), so f is
determined by Rf. Indeed, the Fourier inversion formula com-
bined with the Lemma above gives an inversion formula for R. We
will not pursue inversion formulae here but rather aim for a
range characterization of R (but see Debriard and Gaveau [1]).
Our first step towards a characterization of the range of R
is the following:

LEMMA 2. Let B be an (n-k) X k matrix and let eij denote

the·k X (n-k) matrix with a I 1I in entry ij and zeros else-
where. Then for generic ~ 6 Rn-k the kernel K of the opera-

tor B -+ Bt~ is spanned by the vectors

(1 ~ i ~ k, 1 ~ j, i. ~ (n-k))

PROOF. Assume ~ # 0. To compute dim(K) we may first replace

~ and B by C~ and CB (respectively) where C is an in-
vertible k x k matrix with C~ (1, 0, ... , 0). In this
special case we see that dim(K) (k - 1) x (n - k). It is then

evident in general that if none of the components of the vector ~

are zero then the collection {Vij£} contains (k- 1) x (n- k)

independent elements which belong to K. Q. E. D.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 112 ERIC L. GRINBERG

COROLLARY (I. M. Gelfand et al. [2]). If g(A, c) is a smooth

function in the range of R then g satisfies the system of
(ultrahyperbolic) P. D. E.

o. (1)

for ~ ~. j ~ n - k and 1 ~ i ~ k.

PROOF. If g = Rf, by lemma 1, g(A,~) g(A + B, ~) for

matrices B with Bt ~ 0. Thus

using the notation of lemma 2, and where the aiji are arbitrary

scalars. Differentiating with respect to aiji and then con-

verting the ~-multiplications of g to a-differentiations of g
we obtain the equation (1). Q. E. D.

Next, we formulate some moment conditions similar to those

given by Helgason [6]:

LEMMA 3. Let g(A, c) be a function in the range of R. Let

Jg(A,c)<c,w>k de. (2)

Then Pk(A, w) is a homogeneous degree k polynomial in w and

1 ' 2' ••• )

PROOF. JRf(A,c)<c,w>k de JJf(x,Ax+c)<c,w>k dxdc

JJf(x,Ax+c)<c,~>k dcdx
JJf(x,c)<c-Ax,w>k dcdx
JJf(x,c)(<c,w> - <x,Atw>)k dxdc.

This last expression is a homogeneous degree k polynomial in w

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 113

and Atw with coefficients which depend on f but not on A

or w, Q. E. D.
To better understand the space of functions on k-planes, and
to define the notion of rapid decrease we compute a distance
function:

LEMMA 4. The distance d(A, c) of the k-plane {y Ax + c} to

the origin is defined by

d(A,c) 2 = <[I + AAtJ- 1 c,c>,

where < > denotes the Euclidean inner product in Rn-k •

'

Note tnat the matrix 1 + AAt is a positive definite symmetric,

hence invertible, matrix.

The proof of the lemma is a standard computation in linear

algebra. The main feature of the formula is that a plane may
have a large c parameter and still be rather close to the
origin, so c alone is not a good measure of distance.
We now define a set of basic differential operators which we
will use to formulate the notion of rapidly decreasing functions.
It is most convenient to work on the Fourier transform level.
Thus we assume that g(A, c) is rapidly decreasing in c for
each fixed A, and we let ICA, ~) be its partial ~ourier

transform. Geometrically, we can think of ICA, ~) as a func-

tion of (n - k) planes through the origin, with At playing
the role of the slope matrix: g(A, ~) is the restriction of
f(n, ~) to the n - k pl~ne (n = -At~) (which is dual to the
k-plane y = Ax).
First, if i ranges between and k and j ranges
between and n- k we have the operators a/aaji' From the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 114 ERIC L. GRINBERG

formula in lemma 1 we see that it is not unreasonable to expect

that ag/oaji decrease about as rapidly as g does. However,
we can actually require that ag/aaji decreases even faster than
g (roughly by a factor of I A Ill ) . We write
A

The operator (acting on g(A, ~)) detects an extension

property of our functions g(A, c). Indeed, we can think of
g(A, c) as a function on an open dense subset of the Grassmann
manifold Gr(k, n) of all k-planes (not necessarily through the
origin) in Rn. Specifically, g is defined on the set of all
k-planes which are graphs over Rk x {0}. However, the Radon
transform is naturally defined on all k-planes. Hence we will be
interested in functions g(A, c) which extend smoothly to
Gr(k, n). If g is such a function then Da g(A, ~) has to
li
remain bounded for fixed ~ as A tends to infinity in norm
(this can be verified in general by using local charts on
Gr(k, n); below we will verify this property for functions g of
the form Rf).
Next, we have the operator D~
1

D I I + AA t I -21/2 0
~i a~i·

We combine these operators into a vector valued differential

operator and write D~ to denote (D~ , .••
). The opera- ,D~
n-k 1
tor D~ tests how g(A, c) behaves under infinitesimal motion
of k-planes away from the origin.
Finally, we require a collection of operators which test the
behavior of g(A, c) under infinitesimal rotations. Let

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 115

As written, L acts on the space variables (A, c); here i,

i 0 , j, j 0 vary over appropriate ranges. L represents an
infinitesimal action induced by a rotation in Rn. Specifically,
consider a rotation by an angle e in the (x. Y. ) plane:
1o' Jo

x! x. cos(e) + y. sin(e),
1o lo Jo
y~ a-x. sin(e) + y. cos(e),
Jo 1o Jo

where all other coordinates remain unchanged. This rotation

induces an action on k-planes in Rn. This action can be
expressed in the (A, c) coordinates in a straightforward (but
tedious) fashion. The infinitesimal generator of this action is

We have removed the a/aa term because it is already controlled

by the operators a On the Fourier transform level has
aa:-i·
J.
L

the simpler form

where ni=-IJ.a J,1

. . •f,; J.
.•

DEFINITION. Let g(A, c) be a smooth function. Assume that for

each fixed A, g(A, c) is rapidly decreasing in c. We will
say that g is a rapidly decreasing function of k-planes
if any monomial in the above basic differential operators

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 116 ERIC L. GRINBERG

(a/aa .. , D , D~, L. . ) applied to g yields a function

Jl ali ~ 1 0'Jo
which decreases faster than any polynomial in 1/(1 + 6(A, F;)),
where 6(A, ~) is the Fourier analog of d(A, c), defined by

6(A, ~) 2 11At~ll 2 = IIF;II 2 •

REMARK. This notation of rapid decrease is natural because it is

derived from the action of the group of rigid motions in Rn. It
may seem that we have too large a set of basic operators. Actu-
ally, we could eliminate the {Da .. l and the {L . . } i f we
Jl Jo 1 o
were to work with the Grassmannian of all k-planes. The addi-
tional operators are needed to control the approach to the vari-
ety of planes which are not graphs over Rk x 0. We have chosen
to work with only a dense open set of planes so as to preserve
the form of the partial differential equations involved. Of
course, we expect that Radon transforms preserve our notion of
rapid decrease.

LEMMA 5. Let f 6 S(Rn). Then Rf(A, c) is a rapidly

decreasing function of k-planes in Rn.

PROOF. It is easiest to verify this on the Fourier transform

level, using lemma 1 • Let g(A, f;) = ~(A, 0 = f(-Atf;, f;) . If

n = -At~ then 6(A, 02 = n I 2 +
II ~ I 2• Thus the polynomial
decrease properties of g(A, ~) defined in terms of the function
6(A, ~) correspond directly to the usual notion of polynomial
decrease of f(n, f;) as a function of the Euclidean variables
(n, F;). We now examine the differentiations in the basic opera-
tors. The operator DF; acts as a sum of n and ~ differen-
tiations on f with factors which are at most 0 <11AII2). The
II IAA t II -1 /2 factor in the definition of
+
2 Df; controls these
and hence D~g enjoys all the decrease properties which g has.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 117

The a-derivatives ag/aa~i are easily controlled using the

identity ag/aaji =-~jar/ani. Similarly, Dalig still
decreases because a direct calculation shows that Da g(A, ~)
A R.i
= -n aaf (-At~, ~). Finally, it remains to check L . . . Since
1 ni loJo
L was constructed from a rotation in the (x. , y. ), i t
1o Jo
follows that

This can also be computed directly. In any case, the observa-

tions made regarding each of our thr~e type of basic operators
clearly show that g has the required decrease properties.
Q. E. D.

THEOREM 1. Let g(A, c) be a rapidly decreasing function of

k-planes in Rn. Then g is in the range of the Radon trans-
form if and only if g satisfies the ultrahyperbolic system of
P. D. E. (1) and the moment conditions (2)

PROOF. Clearly, if g is in the range of R then g satisfies

the required conditions. Conversely, let f(A, c) be rapidly
decreasing. The function h

is smooth in the region ~ # 0. The moment conditions (2) imply

that h extends smoothly to all of Rk x Rn-k. The proof is
quite similar to Helgason's proof [6] of the corresponding fact
in the context of compactly supported functions (for general k)
and the context of rapidly decreasing functions (for k = n- 1):
for any positive integer m one can break up the Fourier inte-
gral defining g(A, ~) into a term which is polynomial of degree

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 118 ERIC L. GRINBERG

m in ~ and At~. and a term which vanishes to order m at

~ 0:

L~=O ~! Jg(A,~)<c,~>mdc
+ Jg(A,c)em(<c,~>)dc

where em(t) vanishes to order m at t = 0. Substituting

-(~nt)/(~t~) for A in the right hand side of the above equa-
tion, we observe that the left summand is polynomial in n and
~ while the right summand is smooth of class at least em , since

em(<c,~>) vanishes to order m at ~ = 0. This shows that h

extends smoothly. We still need to show that the extension is

a rapidly decreasing function. For simplicity, let us do this
for the case n = 2, k = first. We drop all indicies and
write A, ~. n, etc. If (A, c) = (-(~nt)/(~to, 0 then
6(A, ~) 2 = n 2 + ~2 . Thus h certainly goes to zero fast
enough. We only need to check deriviatives. First, examine
~-differentiation:

ah
~ (n,O

Note that a = -n/~ here, so ah/o~ is

= n-1 ( a2 • _E!
aa + n • *)
a"
~

Now the operator L acts as a2 .E.! + n .E.! and by assumption,

aa a~
Lg has all the decrease properties that g has. In the region

In I iii: we have established control of ahla~. To treat the

case In I < 1 • we may assume that 1~1 > 1• since we already
know that h is smooth and the remaining reg~on {lnl ~ 1,1~1 ~ 1}
is compact. When lnl < 1 and 1~1 > 1, a is bounded and

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 119

hence so are all ~-derivatives (and of course a-derivatives) of

~. This concludes the verification of boundedness for ~-deriva-

tives of h.
Next, we examine n-derivatives of h:
~~ (n,~) = ~- 1 i! (-nl~.~).
In the r~gion 1~1 ~ 1 we immediately have the required decrease
for ahlan. Reasoning as before, we only need to look at the

case In I ~ and 1~1 < 1. Writing ah/an as

an
ah
<n,O n
-1
• a •

we see that since has the required decrease, so does

ah/an. Higher order derivatives are handled in a similar fashion.
We have thus shown that h is rapidly decreasing in the case
k = 1, n = 2. The higher dimensional case can be proved along
the same lines, but it is somewhat more complicated. The inter-
ested reader is referred to Appendix II at the end of the paper.
Since h(n, ~) is rapidly decreasing, there is a rapidly
decreasing function f(x, y) with f = h. Now

r<-At~.o
h(-At~·O
~(-(~~tA)/(~t~).~)
= ~(B,~)

where B = (-~~tA)/~t~. Since g satisfies the differential

equations (1),

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 120 ERIC L. GRINBERG

for generic ~ and every n M n matrix E with At~ Et~.

If ~ # 0 then

A
so g(A, ~) Rf(A, ~) for almost every ~ and so g Rf,
Q. E. D.

It was pointed out by F. Gonzalez [3] that in the realm of

rapidly decreasing functions, the range of the Radon transform
cannot be characterized by moment conditions alone.

On the other hand, I. M. Gelfand and his collaborators have

announced ([2]) a range characterization in a slightly different
context using only P. D. E. for the case k < n- 1. (Since
there are no P. D. E. in the case k = n - 1, the moment condi-
tions are clearly necessary there (Helgason [6J». In the present
context, we note that the only place the moment conditions were
used is in showing that the function h is smooth at ~ = 0.
The moment conditions could be removed if one knew that a func-
tion g(A, ~) which is smooth in the entries of A and ~ and
depends set-theoretically only on ~ and At~ is actually a
smooth function of the entries of ~ and At~.

NOTE ADDED IN PROOF. The fact about smooth functions of variables

~ and At~ mentioned above is true and, as pointed out by
0. Gabber, easy to prove by constructing global sections of the
appropriate bundle. It shows that the moment conditions can be
removed from all the range characterizations in this paper that
involve a non-equidimensional situation (k < n). Recently,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 121

F. Richter and R. Sulanke have shown independently that the

partial differential equations alone characterize the full
Euclidean k-plane transform (k < n) (preprint, Humbolt
Universitat zu Berlin, 1986).

2. RESTRICTIONS OF RADON TRANSFORMS. In Section 1 we observed

that the k-plane transform is injective on the class of rapidly
decreasing functions in Rn. A completely parallel discussion
can be given for complex k-planes in en, hence the complex
k-plane transform is, in particular, injective. This transform
may be regarded as the restriction of the real 2k-plane transform
in R2 n to the variety of complex 2k-planes. The injectivity of
this restricted transform may be deduced from the following
general fact given in [4]:

THEOREM 2. Let G be a closed subgroup of the group of all

isometries of Rn. Assume that G acts transitively on Rn and
that Rn is still two-point homogeneous under G. Let X be a
fixed k-plane through the origin in and let R denote the
restriction of the k-plane transform to the class of planes of
the form gX where g S G. Then R is injective.
In fact, there is an inversion formula for R which looks
'the same' as the inversion formula for the full k-plane

tr:-ansform [4].In the case of the complex k-plane transform, G

consists of the complex rigid motions of R2n "' en and X is
just ck .. R2k R2n .. en. The Radon transform defined in [4]
differs from ours, but the difference is only a multiplicative
factor, so the corresponding result applies here. We will sketch
a proof of the theorem which is slightly different fr:-om the one
given in [4] and uses our previous results above.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 122 ERIC L. GRINBERG

First we observe that the assumptions on G imply that all

translations are in G. In view of lemma 1, it suffices to prove
that for generic ~ 6 Rk, as (A, c) ranges over the planes
{gXIg 6 G}, the vector -At~ ranges over all generic vectors in
Rn-k . This says that almost every point in Rn is expressible
t By assumption, the isotropy subgroup of G fix-
as (-A~. ~).

ing the origin acts transitively on the unit sphere sn-1 , hence
almost every point in Rn lies in a plane gX for some X,
hence almost every point in Rn is expressible as (x, Ax) and
almost every point irt Rn is orthogonal to a plane of the form
(A, 0) = {(x, y)IY =Ax} belonging to GX. Such a point is
necessarily of the form (-At~, ~) for some ~. Hence almost
t
every point in Rn can be expressed as (A ~. 0, completing
the proof. ~. E. D.
We now consider a particular restriction of the k-plane
transform which has generated some interest in the literature
( [ 1], [4]). We replace n by 2n and k by n and consider
the space of real n-planes in R2n. If (x, y) and (X t ' y')

are points in Rn X Rn .. R2n we have the s:t:m2lectic form

{(x,y), (x',y')} = x • y' - x' • y,

where is the usual dot product on Rn. Clearly, { , } is a

non-degenerate skew-symmetric form on R2 n. An n-dimensional

vector subspace of R2 n is called Lagrangian if the form { , }
restricts to zero on it. A plane of the form (A, 0) is
Lagrangian precisely when A is a symmetric n x n matrix.
Thus we define the Lagrangian Radon transform as the restriction
of the n-plane transform in R2 n to planes of the form (A, c)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 123

where A is symmetric. We observe that this restriction satis-

fies the hypothesis of theorem 2.
The group G in question consists of all motions in R2 n
which can be expressed as a translation followed by a unitary
rotation (the isotropy group of the origin is U(n) c 0(2n)).
The n-plane X is just Rn x 0 c R2 n. Hence the Lagrangian
Radon transform is injective. We now proceed to characterize its
range.

LEMMA 6 (DEBRIARD AND GAVEAU [1]). For generic ~ in Rn, the

kernel of the operation B + Bt~ on symmetric n x n matrices
B is spanned by the basis

where ~ k, 1 ~ n.
The proof follows along the same lines a lemma 2 above.

THEOREM 3 [1]. Let g(A, c) be a smooth function defined for

all A symmetric and all c. If g lies in the range of the
Lagrangian Radon transform then g satisfies the cubic system:

;. 0, (3)

for 1 ~ k, 1 ~ n.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 124 ERIC L. GRINBERG

We'll need to extend our notion of a rapidly decreasing function

of k-planes in Rn to a closed submanifold of planes (A, c).
To avoid technicalities, we will say that a function f(A, c)
defined on a subset of k-planes is rapidly decreasing if it
extends to a rapidly decreasing function on all (A, c).

COROLLARY. Let g(A, c) be a rapidly decreasing function of

(affine) Lagrangian planes in R2 n. Then g is in the range of
the Lagrangian Radon transform if and only if g satisfies the
system (3) and the moment conditions (1) restricted to symmetric
matrices A.
The proof is similar to the proof of theorem 1 above. The
only significant difference lies in the construction for a given
pair of vectors ~. n of a matrix B so that -Bt~ = n. In
theorem 1 we used B = -(~nt)/(~t~). Here we need a symmetric
matrix B. One possibility [1] is to take a diagonal matrix
D(n) whose jth diagonal entry is -nj/~j. The rest of the
proof is straightforward and is left to the reader.

3. A SINGULAR RESTRICTED RADON TRANSFORM. We consider the n-

plane transform on R2 n once again. If (x, y) and (x', y')
are vectors in Rn x Rn = R2 n we define the quadratic form

Q((x,y),(x',y')) = x • y' + x' • y

This is the symmetric form corresponding to the skew form { , }

of Section 2.
An n-plane X through the origin in R2 n is said to be
isotropic with respect to Q if the form Q restricts to zero
on X. A plane {y = Ax} is isotropic precisely when the matrix
A is skew symmetric. We thus consider the restricted n-plane

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 125

transform (also denoted by R) defined only on planes (A, c)

with A skew. As we shall soon see, despite the analogy with
the Lagrangian Radon transform, this operator is qualitatively
very different. In particular, it is not injective.

THEOREM 3. Let f(x, y) be a rapidly decreasing function on

R2 n. Then the following are equivalent:
1. The Radon transform Rf vanishes identically
2. f(n, ~) is zero on the variety {n • ~ = 0}
3. The partial differential equation

+ • • • + f

has a rapidly decreasing solution u. In particular, the kernel

of R is dense in hence R has no extension to which
is compatible with the conclusion of lemma 1 above (the trans-
forms considered in Sections 1 and 2 do have such L2 exten-
sions). To prove the theorem, we will need a division lemma in
the class of rapidly decreasing functions.

DEFINITION. Let U be an open subset of Rm and let g be a

smooth function in U. We will say that g decreases rapidly in
U if, for every linear partial differential operator P with
polynomial coefficients the function Pg is bounded on U.

LEMMA 7. Let U be and open subset of Rm+ 1 = R x Rm with the

property that if (a,b) 6 U and t > 0 then (ta,b) 6 U. If
g(a,b) is a rapidly decreasing function on U which vanishes
on the variety {a = 0} then there exists a rapidly decreasing
function h(a,b) with g(a,b) =a • h(a,b).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 126 ERIC L. GRINBERG

PROOF. The existence of a smooth h follows from Taylor's

theorem. We need to show that h is rapidly decreasing (along
with its derivatives). This is clear from the following explicit
representation of h:

h(a,b) I, iS.
0 oa
" (ta,b) dt,

Q. E. D.

Armed with the division lemma we proceed to prove the theo-

rem. First by lemma 1 above, Rf gives us information about
,..
f(n, ~) precisely for (n, ~) of the form n = A~ with A
skew. This consists almost exactly of the variety {n . ~ O}.
Therefore, Rf =0 i f and only if r vanishes on {n • ~ 0},

i.e., 1 and 2 are equivalent. Now 3 clearly implies 2, so it

only remains to show that 2 implies 3. By a linear change of
coordinates, we can describe our variety as n2 - ~2 = o. The
task then is to show that if g(n, ~) vanishes on { n2 - ~2}

then g = (n 2 - ~ 2 )h where h is rapidly decreasing. The

existance of a smooth h is guaranteed by the Malgrange division
theorem near the origin (see Appendix I), and by Taylor's theorem
away from the origin; clearly, h is unique. We need to show
that h is rapidly decreasing.
We define a region U~ in R2n by
2 2 2 2
{ n, > 0, n2+•••+nn-~,-···-~n < O}. The map

(n,O + (x,z,w)

w = <w,, .•. ,wn) = <~, ••••• ~n). maps u~ to the region

{(x, y, z)iz 2 - w2 < 0, x > z 2 - w2 } which satisfies the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 127

hypothesis of lemma 4,with x playing the role of a and (z,w)

playing the role of b. Since the above coordinate change has
a (non-vanishing) Jacobian which satisfies polynomial esti-
mates along with all its derivatives, the lemma shows that
h decreases rapidly in u,.
+
Similarly,
replacing the condition n1 > 0 by n1 < 0 we get a region U~

in which h decreases rapidly. Next, replacing the coordinate

+
n, by nj and by ~j (respectively), we get regions uj, uj.
+
vj, and vj on which h decreases rapidly. since these
regions cover the set {(n, 0 ln 1 {. 0, ... '~n {. 0}, h is
rapidly decreasing everywhere. Q. E. D.

Having described the kernel of R we proceed to

characterize the range.

DEFINITION. We will denote by Sij the n x n matrix with a 1

in entry (i, j), a (-1) in entry (j, i), and zeros elsewhere.

LEMMA 8. The kernel of the operation B + B~ in the class of

skew symmetric n x n matrices B is spanned by the basis

REMARK. Note that this says in particular that the operation is

injective in the case n s 2.
The proof of lemma 5 is similar to that of lemma 2.

COROLLARY. Let g(A, c) be a smooth function defined for A

skew and c arbitrary. If g lies in the range of R then g
satisfies the system of P. D. E.

+ (4)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 128 ERIC L. GRINBERG

The proof is similar to the proof of the corollary to lemma 2

above. We thus have a range characterization similar to theorem
1.

THEOREM 4. Let g(A, c) be a rapidly decreasing function de-

fined for A skew and all c. Then g is in the range of R
if and only if g satisfies the system of P. D. E. (4) and the
moment conditions (2) for A skew.

PROOF. The 'only if' portion of the theorem is clear from the
preceeding discussion. To prove the 'if' part, we let g(A, c)
be a function satisfying the hypothesis of the theorem. If we
could extend the definition of g(A, c) to all matrices A, so
that the appropriate moment conditions and differential equations
were preserved, then we could use theorem 1 to conclude that g
(restricted to skew matrices A) lies in the range of R. We
will perform the extension on the Fourier transform level.
Let g(A,~) be the partial Fourier transform of g, as
usual. Then the method of proof of theorem 1 shows that
h(n,O g(-~tn/~t~.~) extends to a smooth function on the
variety {n • ~ = 0} (i.e. all finite difference operators have
continuous limits). Let Pk(A, w) denote the kth moment of g
as in lemma 3 above. Consider the power series S(At~. ~) =

tk (k!)- 1 Pk(A, ~) viewed as a power series in ~ and At~. By

Emile Borel's extension theorem, there is a Cm function s(At~.~)
whose Taylor series at the origin is S(At~. ~). For each n
and ~ with n • ~ o put r( n, ~) = h( n, ~) - s ( n, ~). Then
r is em on {n ° ~ 0) and vanishes to infinite order at the
origin. By passing to a linear (global) change of coordinates we
may replace the variety { n • ~} with { II n I 2 - II ~ I 2 = 0} in

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 129

the previous sentence. Let

R<n. 0 = r< II r; 1 112 II" I - 1 12 n. I n 1 112 I r; I - 1 12 r;).

Then R(n, F;), viewed as a function on all of Rn x Rn, is

smooth everywhere since r is flat at the origin. This shows
that h(n, F;) extends to a smooth function on all of Rn x Rn,
since s(A, F;) clearly does. Finally, returning to the
{n • F; 0} picture, if ~(t) is a smooth compactly supported
function on R1 with ~(0) = 1, then h 1 (n,F;) = ~(n • F;)h(n, F;)
extends h(n, F;) and is rapidly decreasing. If f(x, y) is the
inverse Fourier transform of h1 then f is rapidly decreasing
and Rf = g. Q. E. D.
It would be interesting to give a more canonical construc-
tion of a right inverse for R. The construction in the proof
above relies on Borel's extension theorem which is non-canonical
and does not yield a continuous inverse. It would also be of
interest to investigate other restrictions of Radon transforms
(e.g. those related to quaternionic forms) and to characterize
their kernels and ranges. These characterizations will likely
involve new systems of P. D. E •• We hope to pursue this in
future papers.

APPENDIX I

LEMMA. Let f(x,y) be a smooth function in a neighborhood of

(0,0) in Rn x Rn. If f(x,y) = 0 whenever x2 y 2 = 0,
then f(x,y) • (x 2 - y 2 )g(x,y), where g is smooth near
(0,0).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 130 ERIC L. GRINBERG

PROOF. We'll write t for x1 , and w for Cx 2 , ..• ,xn).

According to the Malgrange-Mather division theorem [8] there are
smooth functions q(t, w, y) and r(t, w, y) defined near the
origin with r of the form a(w, y) + b(w, y)t such that

f(t,w,y) (t 2 + w2 - y 2 )q(t,w,y) + r(t,w,y).

In the region {w 2 < y 2 } the equation f(t, w, y) = 0 has, for

(w, y) fixed, two distinct solutions so, since r is linear in
t, it must vanish identically in this region. It follows that
r vanishes to infinite order at the origin t = w = y = 0 (the
region w2 < y 2 contains tubes around 2n curves terminating at
the origin with linearly independent tangents there). Conse-
quently, r is divisible by Ct 2 + w2 - y 2 ) with em quotient
s. We can therefore take g= q + s, Q. E. D.

APPENDIX I I

Completion of the proof of Theorem 1 in higher dimensions.

Let g(A, c) be a rapidly decreasing function of k-planes in
Rn. Here k and n ~ k are arbitrary. Put h(n,~)

= g(-~nt/~t~.~). We need to show that h is a Schwartz func-

tion on Rn. We have already shown in the course of proving
theorem 1 (using the moment conditions) that h is smooth. It is
also clear that h decays faster than any power of 1/(n 2 + ~2 )
because g decays faster than powers of 1/o(A, ~), where as
before, o(A,~) 2 = ~At~ ~ 2 + ~ ~ ~ 2 (vector norms are i 2

norms unless otherwise specified). We proceed to check

derivatives:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 131

We first observe that this is a smooth function of (n, ~)

because h is. Further, in the region { II ~ II > 1, II n II < 1}

II A II is small, so Clg/aaji is controlled and the sum involv-
ing the derivatives Clg/aaji is bounded by a polynomial in
II n 11 2 + II ~ 11 2 = o(A, o2 when A = -~nt/~t~. Finally, we
consider the case II n II > (the rest is compact). Choose an
index t with Inti large. Then we can rewrite 1b_ as
a~m

A direct calculation (using the substitution ni

shows that this is precisely

The leftmost summand is nicely bounded because Llm is a basic

operator acting on the rapidly decreasing function of k-planes
g, and because we are in a region where Inti is large. The
remaining terms appear singular but actually represent a smooth
function (because the rest of the terms in the equation are
smooth). By inspection, this function is dominated by a poly-
nomial in o(A, ~) as before. This takes care of ~

derivatives.
Next, we examine n derivatives:

~~ 1. = ~ -~
J
j II ~ II - 2 ~! .. ·
Jl

In the region II n II > 1 we choose an index t with I n1 1

large and write the previous equation as

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 132 ERIC L. GRINBERG

Since is a basic operator we have control in the region

II n II In the region II n II < 1 we can assume II c; II > 1

and then obtain polynomial estimates on the coefficient of .!L

aaji
as before. Thus we have checked the decrease of n derivatives.
Higher derivatives can be handled in a similar fashion. This
shows that h(n, t;) is a Schwartz function and completes the
proof of theorem 1.

BIBLIOGRAPHY

1. A. Debriard and B. Gaveau, "Formule d'inversion en

geometrie integrale Lagrangienne" (French), C. R. Acad. Sc. Paris,
296 (1983) Serle I, 423-425.
2. I. M. Gelfand, s. G. Gindikin, and M. I. Graev, "Integral
geometry in affine and projective spaces," Journal of Soviet
Math, vol. 18 No.3 (1982), 39-164, (trans. from Itogi i Nauk i
Tekhniki, Seriya Sovremennye Problemy Matematiki, Vol. 16 (1980),
53-226.
3. F. Gonzalez, "A Radon transform on Grassmann manifolds,"
Ph.D thesis, M. I. T., Cambridge, Mass. (1984).
4. E. L. Grinberg, "On images of Radon transforms," Duke J.
Math. vol. 52, no. 4 (1985), 939-972.
5. V. Guillemin and s. Sternberg, "Some problems in integral
geometry and some related problems in microlocal analysis," Amer.
J. Math lQl (1979), 915-955.
6. S. Helgason, "The Radon transform," Progress in Math~
(1980), Birkhauser, Boston.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 EUCLIDEAN RADON TRANSFORMS: RANGES AND RESTRICTIONS 133

1. F. John, "The ultrahyperbolic differential equation with

four variables," Duke J. Math. ! (1938), 300-322.
8. L. Nirenberg, "A proof of the Malgrange preparation
theorem," Proc. of Liverpool Singularities-Symposium I (C. T. C.
Walled.), Lee. Not. Math 192 (1970), Springer-Verlag, New York.
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF MICHIGAN
ANN ARBOR, MICHIGAN 48109-1003

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

PERSPECTIVES IN INTEGRAL GEOMETRY

VICTOR GUILLEMIN

1. INTRODUCTION. This talk will consist of two parts, part one a

brief (and, therefore, unfortunately a rather selective) survey
of developments in integral geometry from the mid-nineteenth
century to the present; and, part two, a discussion of integral
transforms from the point of view of double fibrations. (This
approach allows one to do integral geometry with some bizarre
looking objects such as twistors and sheaves. See Section 5.)
The type of integral geometry which I'll be talking about
for the next hour is sometimes called "Gelfand-Helgason style"
integral geometry. For a survey of what has been going on in
other areas of integral geometray, for instance, the "classical"
area (of Blaschke-Santal6 et al.), see the talks by Robert Bryant
and Ronny Wells.

2. CROFTON ET AL. Most of you are probably familiar with

Crofton's theorem on curves in the plane: Roughly speaking
Crofton's theorem says that the probability of a convex curve in
the plane intersecting a randomly chosen line is proportional to
the length of the curve. To formulate this theorem precisely we
need to provide the set, L, of all lines in R2 with a mea-
sure, and we do this as follows. every oriented line, ~. can
be uniquely described by an equation of the form
© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

135
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 136 VICTOR GUILLEMIN

where w6 s1 and r 6 R. Hence there is a bijection

L +-+- s1 x R
( 2. 1 )
R.++(w,r)

and we can take for our measure on L the measure dwdr. Now
let C be a convex curve and let C be the set of all lines
R. e L, such that R. n C ~ 0. Then Crofton's theorem says that

Jc dwdr s two times the length of c. (2.2)

For a proof of (2.2) see Santal6, [15], Section 1 .3.2.

Crofton proved a number of other theorems of the same nature
as (2.2), one of the most beautiful being the following theorem
on planes in R3. Let P be the set of all (oriented) affine
planes in R3. Then the equation of a plane, P 6 P, is de-
scribed, as above, by a pair (w,r) e s2 x R, i.e.

P = {x e R2 : r},

and this sets up a bijection

P +-+- s2 x R

P ~cw,r).

Let S be a closed convex surface in R3 and let S be the set

of all planes which intersect S non-trivially. Then

Js H dS • J; dwdr (2.3)

dS being the element of area on S and H the sectional

curvature. (See Blaschke, [1].)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 137

These formulas contain in embryo the idea of the Radon

transform. The correspondences between sets, described above,
(i.e. C +-+C and S +-+S) can easily be extended to correspon-
dences between functions; if f is a compactly supported contin-
uous function on R3, then its Radon transform, f, is a
function on P whose value at P 6 P is the integral of f
over P. In symbols

f(P) f(w, r) Jp f dP (2.4)

dP being the unit of area on P. In [14] Radon showed that the

correspondence, f ~ f, is one-one by showing that f can be
expressed in terms of f by means of the following inversion
formula:

-8'11" 2 f(x) - (2.5)

In the same paper he also showed that a similar (but more compli-
cated) formula was true in R2 n+ 1 ; and, in 1934, Fritz John
showed thatananalogue of (2.5) was true in all dimensions. John
also showed that the Radon transform and integral transforms of a
similar nature (such as the spherical mean transform) had some
elegant applications in the theory of partial differential equa-
tiona. We won't have time to describe these here, except to
mention the following result: Differentiating (2.5) one gets

a
-8'11" 2---f(x) = J a3
---f(w,r)l
A
[Link].
axi ~ 3
3 2 or r•w•x

Therefore, differentiating again and summing over i,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 138 VICTOR GUILLEMIN

a4
- 4 f(w,r) I dw.
ar r=w•x

This shows that the Radon Transform converts the relatively

complicated operator, 6, into the relatively simple operator,
a2 1ar 2 • Another way of saying this is that, under the Radon
transform, the operator which describes the time evolution of
solutions of the wave equation gets converted into the operator
"translation by t." For some applications of this result, see
[ 1 3].

3. THE HOROCYCLE TRANSFORM. In the late 40's Gelfand, Neumark,

Harish-Chandra, Segal and others initiated a program for extending
the classical Plancherel formula to non-compact semi-simple Lie
groups. The problem they were interested in is the following:
Let G be a non-compact semi-simple Lie group. (For instance,
G = SL(n,R), the group of invertible n x n matrices ot determi-
nant one.) Let H be a closed subgroup of G, and let G/H be
the H-coset space. Problem: Decompose the Hilbert space,
L2 (G/H), into a direct sum (or direct integral) of a-invariant
irreducible summands. By the mid-fifties, it was known that for
all G's one could do ·this for certain H's. For instance every
G contains a maximal uni-potent subgroup, N. (For
G • SL(n,R), N is the upper triangular matrices with one's along
the diagonal.) Thanks to Mackey's work on induced representa-
tions it was known rather early that L2 (G/N) can be de~omposed
into a direct integral of so-called unitary principal series
representations. In the mid-fifties, Bruhat showed that these
representations were either irreducible or, at worst,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 139

decomposible into a finite sum of irreducible summands; so this

settled the Plancherel question for GIN. (Incidentally, these
results apply equally well to any closed subgroup, H, contain-
ing N, a fact which we will use below.)
In the early 60's Gelfand and Graev on the one hand and
Helgason on the other devised an ingenious scheme for applying
these results to any H. Their idea was the following; Let X •
G/H and consider subsets of X of the form

gNx = {gnx, n 6 N}.

Such subsets are called horocycles. Some examples are:

1. Let X SL(2,R). (i.e. G = SL(2,R) and H =e.) A matrix

is in SL(2,R) if (x 1 ,x 2 ,x 3 ,x 4 ) lies on the quadric

x 1x 4 - x 2 x 3 • 1. The horocycles are just the affine lines in a4
which lie on this quadric.

2. Let G = SL(2,R) and let H = S0(2), i.e.

H = { [ cos
-sin
e sin
e cos :]' 0 • • • 2• }

Then X = G/H can be identified with the Poincare upper half-

plane. The horocycles are either circles tangent to the real
axis as in Figure 1, or horizontal lines as in Figure 2.

X X

Figure 1 Figure 2

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 140 VICTOR GUILLEMIN

Now let all X be the set of all horocycles. There is a

natural action of G on X. Namely, if a 6 G, ~ acts on X by
mapping the horocycle gNx to the horocycle ~gNx. Let us
inspect a typical G-orbit for this action. Let p be a point of
X representing the horocycle gNx. Then the group gNg -1 stab-
-1
ilizes p; so N stabilizes g p. This proves

PROPOSITION. The G orbits in X are of the form G/H where

H is a closed subgroup of G containing N.

In many instances it is possible to find a measurable cross-

section for the action of G on X. If one can do this, then
one can decompose L2 (X) into a direct sum (or direct integral)
of the L2 spaces associated with the orbits. However, by the
proposition above, each of these can be decomposed into a direct
sum (or direct integral) of unitary principle series representa-
tions; so one gets with relatively little effort a Plancherel
formula for L2 (X).
What I have just described is part one of the Gelfand-
Helgason program. Part two is to map L2 (X) into L2 (x) by
means of the horocycle transform. For simplicity we will assume
that X = G/H with H compact. If f 6 C~(X) and p 6 X the
horocycle transform of f, evaluated at p, is

f(p) = JNf(gnx)dn. (3.1)

(Here p = gNx and dn is Haar measure on N.)

Of course (3.1) only defines the horocycle transform of f
if f is compactly supported and continuous. The problem of
extending the horocycle transform to L2 (X) is by no means
trivial, and even when this is possible, it is not clear that it

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 141

maps bijectively onto L2 (X). (For example this is true

in Example 2 above but not in Example 1: the kernel is the sum
of the discrete series representations of SL(2,R).) Nonethe-
less, the horocycle transform is a formidable tool for doing
harmonic analysis on G/H's even for rather bizarre kinds of H
(e.g. H discrete). (References: in Gelfand, Graev, Vilenkin
[6] the examples mentioned above are discussed in detail. For
the general theory see [5] and [10].)

4. DOUBLE FIBRATIORS. Before turning our attention to more

recent developments, it will be convenient to introduce a little
terminology. In Figure 3 below, X, Y, and Z are manifolds
and n and p fiber mappings.

( 4 .1)

Figure 3

We will call this diagram a double fibration if the product map

n x p : Z + X x Y (4.2)

imbeds Z as a submanifold at X x Y. If this is the case then

(4.2) also imbeds the the fiber

F • p-1(y)
y

as a submanifold of X. Thus a double fibration gives us a

family of submanifolds, {Fy}• of X parametrized by the points
of Y. Notice that, by the same token, the fibers, Gx• of n
are a family of submanifolds of Y parametrized by the points of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 142 VICTOR GUILLEMIN

X. The F 's
y
are related to the G 's
X
by the incidence
relation, Z; i.e.

( x , y) 6 Z <~) X e Fy <=> y 6 GX •

Usually we will assume that one or the other of the mappings

in (4.1) is proper. In fact, in this section we will assume n
is proper. This implies that the G 's
X
are compact.
Given a Cw manifold and a Borel measure, ~. on its a-
field of Borel sets we will say that ~ is smooth if, for every
coordinate patch U, ~ is equal to a smooth function times
Lebesgue measure on U. We will say that ~ is nowhere
vanishing if for every coordinate patch this function is non-
vanishing.
Suppose now that we are given nowhere vanishing smooth
measures, ~ and v, on Z and Y. Then, if g is a compact-
ly supported Cw function on Z, the measure g~ is compactly
supported. Thus by elementary measure theory, the direct lower
star image, p*g~, is well-defined and is a compactly supported
measure on Y. Using a partition of unity on Z it is easy to
check that p*g~ is smooth; so we can express it globally as the
product of v with a compactly supported smooth function. We
will denote this function by p*g; i.e. by definition,

Now let f be a compactly supported smooth function on X.

Since n is proper, the inverse image, n *f, is compactly
supported; so we can apply p* to it and obtain p*n *f.
Therefore, the data, ~ and v, enable us to define an integral
transform

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 143

(4.4)

mapping C~(X) into C~(Y). If we allow X to have a nowhere

vanishing smooth measure as well (say o) we obtain an integral
transform

mapping in the opposite direction. This time, however, b•cause

of the properness of n, S carries C~(Y) into C~(X), (no
support conditions). It is easy to check that, for f 6 C~(X)
and g 6 C~(Y),

J f Sg do = J Rf g dv, (4.6)
X y

i.e. S is the transpose of R.

The main problem in integral geometry is the inversion
problem: construct a left inverse for R. One can consider Rt
as an approximate left inverse for R since RtR is injective
if R is; so we can, without loss of generality, focus our
attention on the inversion problem for RtR. We won't try to
solve this problem here; however, we will make a modest first
step in the direction of solving it by describing explicitly the
Schwartz kernel of the operator RtR. First, however, we will
need to review some elementary facts about products of measures:
let K, M, and N be smooth manifolds and n 1 : M + K and
n2 : N + K smooth mappings. The fiber product

p = MX N
K

is the set

{(m,n) eM X N: n,(m)- 1T2(n)}.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 144 VICTOR GUILLEMIN

If 'IT, and 1T2 are transversal, this set is a closed

submanifold of the usual product, M " N. Now let ]J and \/ be
smooth measures on M and N, and K a nowhere vanishing
smooth measure on K. Then, providing 'IT, and '1T2 are
transversal, one can construct from the data, ]J, v, and K, a
measure, 0, on P, by a canonical procedure. We won't
describe this construction here; however, to give you a sense of
how it goes, we will describe two extreme cases (the general case
being an amalgamation of these two cases).

CASE 1. K =a point. Then P is the usual Cartesian product of

M and N and o is the usual product measure, JJ x v.

CASE 2. N = a point. Then 1T2(N) is a point p in K and

p = 1T1-1 (p). The hypothesis that 1T1 and '1T2 are transversal
means simply that p is a regular value of 1T 1 • It is easy

to see that this implies that if f is a smooth compactly-

supported function on M, then the measure ('1T 1 )*fJJ is smooth
near p. Therefore, there exists a smooth function, f, on K
such that near p

By definition the measure o on P is the unique measure for

which, for all f 6 C~(M),

Jp f do f(p). ( 4. 7)

(In fact, by the Riesz representation theorem, (4.7) can be taken

as the definition of o.) For the construction of o in general
see [9].

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 145

Here are some applications of this construction: Let K, M,

and N be the manifolds X, Z, and { x} where X and z are
as in ( 4. 1 ) and X is a point of X. Then G .
p = so for all
x'
x, Gx is equipped with a smooth measure. Next take K, M, and N
to be Y, Gx' and Gx • Then p = G n so n G is
xo Gx ' Gx x1
0 1 1 0
equipped with a smooth measure providing Gx and Gx inter-
0 1
sect transversally inside of Y. Since the GX 's are compact
this means the volume of is well-defined. Let us set

Volume of (4.8)

THEOREM 1. The Schwartz kernel of the operator RtR is smooth

near (x 0 ,x 1 ) providing Gx and Gx intersect transversally.
0 1
Moreover, if this happens, the Schwartz kernel is equal to (4.8)
at (x 0 , x 1 ).

The proof of this theorem is relatively simple but we won't

give it here. Incidentally, using Sard's theorem it is easy to
show that intersects transversally for almost all
(x 0 ,x 1 ) 6 X x X. From this one easily deduces:

THEOREM 2. The Schwartz kernel of RtR is a locally integrable

function on X x X and is equal to (4.8) almost everywhere.

EXAMPLE. Let R be the classical Radon transform on R3. It is

clear that (4.8) is invariant under the group of Euclidean
motions, so it is just a function of the distance between x0
and x1 . A simple scaling argument shows that, for some positive
constant Y,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 146 VICTOR GUILLEMIN

However, 1 is the fundamental solution of Laplace's

1i1T lxo - x1 I
equation on a3., so

~RtR = 4nY Identity.

Since R intertwines ~ and a2 1ar 2 , this formula is equiva-

lent to (2.5).

PARENTHETICAL REMARK: In our historical survey (Section 1-3) we

have said nothing about what is nowadays called "classical"
integral geometry (in the sense of Blaschke and Santal6). No-
tice, however, that the main problem of integral g~ometry (in our
sense) comes down to computing the mixed volumes, (4.8). This is
typical of the kind of problem considered in classical integral
geometry. (Conclusion: these two types of integral geometry are
not as unrelated as is usually supposed.)

5. REV DIRECTIONS IN INTEGRAL GEOMETRY. Functions are contra-

variant objects: If M and N are manifolds and t : M+ N a
smooth map, then functions on N can be pulled back by t to
functions on M. With considerable effort, functions can be made
into covariant objects as well. If M and N are equipped with
smooth nowhere vanishing measures and t is a proper fiber
mapping, then (as we saw in Section 4) functions on M can be
pushed forward to N.
There are other objects in mathematics which are bi-variant
in this sense. For instance, differential forms on N can be
pulled back to M by t; and, if t is a proper fiber mapping,
differential forms on M can be pushed forward to N with a
shift in degree equal to the dimension of the fiber (see Bott-
Tu), [2]). Let us denote these two operations by t* and t*.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 147

Suppose now that the double fibration (4.1) has the property
that the second arrow is proper. Then by composing ~* with p*
we obtain an integral transform.

p*~* : k-forms on X + (k-f)-forms on Y.

(f is the dimension of the fiber of p) (5 .1)

Integral transforms of this type have begun to be investigated

recently by Gelfand, Graev, Gindinkin and others (see, for in-
stance, [7]). However, they are still not completely understood.
A sheaf is another type of mathematical object which has the
property of being bi-variant: If S is a sheaf on N, then its
inverse image ~*S is defined as the fiber product.

--> s

l
~*S

--> !
(S being regarded as a covering space of N.) There is also a
direct image operation on sheaves which is more sophisticated.
If
ger,
setting
S
k
is a sheaf of abelian groups on
0,1, .•• , one gete a pre-sheaf,
M,
----
then for each inte-
(~*)kS' on N by

for each open set U in N. The sheafification of this pre-

sheaf, which we will denote by (~*)kS' is called the k-th
direct image of the sheaf S. (See Godement, [8].)
Given a double fibration, (4.1), and a sheaf, S on X let
Sk be the sheaf (p*)k~*S.

Then from the sequence of mappings

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 148 VICTOR GUILLEMIN

we get a mapping

(5.2)

which can be viewed as a kind of sheafified integral transform.

An important example of a transform of this type occurs in
the work of Eastwood, Penrose and Wells on the zero rest-mass
field equations of quantum physics. Let PT = ep3 = complex
projective 3-space and let Me be the Grassmannian of complex
lines in eP 3 . Let Z be the point-line incidence relation.
Both PT and Me are familiar objects in physics: PT is
projectivized twistor space, and Me is just Minkowski space in
disguise.(*) Let L + eP 3 be the canonical line bundle and let
n
0(-n) be the sheaf of holomorphic sections of ~ L. The double
fibration

(5.3)

associates with the sheaf 0(-n), a sheaf 0(-n)' = (p*) 1 n*O(-n)

on Me. This sheaf turns out to be the sheaf of holomorphic
sections of a vector bundle, E(n) + Me· Moreover, if U is an
open subset of Me, then, with U' = p- 1 (U) and U'' = n(U'),
we get a double fibration
IJ'

U" u
(*) To be more accura t e, Me is compactified, complexified
Minkowski space.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 PERSPECTIVES IN INTEGRAL GEOMETRY 149

and, by (5.2) a map,

H1 ~ (U" ,0(-n)) ~ r(U,E(n)). (5.4)

Eastwood, Penrose and Wells show that for appropriate, U, the

image of (5.4) is the set of all solutions of the zero rest-mass
field equations of helicity n on U. (See [4] and [16]).
Another example of integral geometry applied to sheaves
occurs in some recent work of Brylinski. Let CPn be complex
projective n-space, (CPn)* the space of hyperplanes in CPn
and Z the incidence relation. Let V ~ CPn be a complex
projective variety and H a hyperplane. The cohomology groups
of V n H have been extensively studied by Lefshetz and others
for hyperplanes H which are in general position with respect
to V. However, not too much was known until recently about how
these cohomology groups vary as one varies H. Brylinski was
able to prove some very precise statements about the variation
with respect to H by showing that these cohomology groups could
be regarded as the Radon transforms with respect to the double
fibration.

of the constant sheaf of the variety V. See [3]. For other

applications in this direction see [12].

BIBLIOGRAPHY
1. W. Blaschke, Vorlesungen ueber Integralgeometrie, 3rd.
ed. Deutsch. Verlag Wiss. Berlin (1955).
2. R. Bott and L. Tu, Differential Forms in Algebraic
Topology, Springer Verlag, New York (1982~--

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 150 VICTOR GUILLEMIN

3. J. L. Brylinski, "Transformation de Fourier g~ometrique"

(to appear in Ast~risque).
4. M. Eastwood, R. Penrose and R. o. Wells, Jr.,
"Cohomology and massless fields," Comm. Math. Phys. 78 (1980)
305-351.
5. I. M. Geland and M. I. Graev, "The geometry of
homogeneous spaces, group representations in homogeneous spaces
and questions in integral geometry related to them," Trans.
Moscow Math. Soc. ~ (1959) 321-390.
6. I. M. Gelfand, M. I. Graev and N. Ya. Vilenkin, General-
ized Functions Vol. 5, Academic Press, New York (1966).
1. I. M. Gelfand, M. I. Graev and Z. Ya. Schapiro, "Differ-
ential forms and integral geometry," Funct. Anal. Appl. 3 (1969)
101-114. -
8. R. Godement, Topologie Algebrigue et Theorie des
Faisceaux 3rd. ed., Hermann, Paris (1973).
9. V. Guillemin and s. Sternberg, Geometric Asymptotics
AMS, Providence (1977).
10. S. Helgason, "Duality and Radon transforms for symmetric
spaces," Amer. J. Math. 85 (1963) 667-692.
11. F. John, Plane Waves and Spherical Means Applied to
Partial Differential Equations, Wiley (Interscience) New York
(1955).
12. M. Kashiwara and P. Schapira, "Microlocal study of
sheaves," Proc. Japan Acad. Vol. 59, Ser. A, No. 8 (1983)
349-354.
13. P. Lax and Ralph Phillips, Scattering Theory, Academic
Press, New York (1967).
14. J. Radon, "Ueber die Bestimmung von Funktionene durch
ihre Integralwerte laengs gewisser Mannigfaltigkeiten," Ber.
Verh. Sachs. Akad. Wiss. Leipzig. Math. Natur. Kl. 69 (1917)
262-277.
15. L. Santalo, Integral Geometry and Geometric Probability,
Addison~wesley, Reading, Mass. (1976).
16. R. 0. Wells Jr., "Complex manifolds and mathematical
physics," Bull AMS Vol. 1, No. 2, (1979) 296-336.
DEPARTMENT OF MATHEMATICS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
CAMBRIDGE, MASSACHUSETTS 02139

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

SOME RESULTS ON RADON TRANSFORMS, HUYGENS' PRINCIPLE

AND X-RAY TRANSFORMS

S. HELGASON 1

1. INTRODUCTION. This paper i8 partly a survey, partly an expo-

sition with proofs of some new results. These are an outgrowth
of a study of some problems in modern integral geometry. Basi-
cally, this subject is concerned with the problem of determining
a function on a manifold on the basis of its integrals over
certain submanifolds; the subject can be said to originate with
the papers Radon [23] and John [18a], both of which state tqe
problem in the title. Later results in the subject have had
substantial applications to differential equations, representa-
tion theory, and tomography.
To formalize the principal problems let X be a differenti-
able manifold in which certain submanifolds ~ depending on
parameters ~ 1 •.••• ~n are singled out. For a function f on
X its Radon transform f is defined by

f{E,;) - f( ~,. ... , ~

n
) & J f(x)da(x),
~
( 1. 1 )

da being some preassigned measure on ~. Let denote the set

of all these manifolds ~. so f is a function on

1supported in part by NSF Grant MCS-8202127.

@ 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

151
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 152 [Link]

The basic problems in this framework are the following:

A. Determine f from its integrals f(~).

B. Determine the range of the mapping f + f as f runs through

some naturally defined space of functions on X.
While Problem A is the one considered by Radon and John,
solutions to Problem B, even partial ones, have also been found
to be important for applications.
This paper is organized as follows. In Sections 2-3 we
describe known solutions to Problems A and B for the p-plane
transform and for the horocycle transform on a symmetric space.
In Section 4 we discuss a natural generalization of these exam-
ples, namely the Radon transform for a double fibration of a
homogeneous space; here we prove some simple general correspon-
dence theorems for function spaces. In Section 5 we describe
support theorems for the spherical mean value operator and deduce
a new converse of Huygens' principle for the conformally invariant
wave equation on a hyperbolic space. In Section 6 we prove a new
support theorem for the X-ray transform on a symmetric space.

Notation. If X is a topological space, C(X) denotes the

space of continuous functions on X and Cc(X) the subspace of
functions of compact support. If X is a manifold we use
Schwartz' notation V(X) for C~(X), E(X) for Cc(X), and
V'(X) and E'(X) for the distribution duals. For a closed set
s c X, Vs(X) stands for the set of f 6 V(X) with support
contained in s.

2. THE p-PLAME TRANSFORM. Let X a Rn and G(p,n) the

family of p-dimensional planes in Rn. Here p is a fixed

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 153

number, 0 < p < n. The p-plane transform of a function f on

Rn is given by

f(0 J f(x)dm(x), F; 6 -' ( 2. 1 )

F;

dm being the Euclidean volume element on the p-plane ~. For

p = n- 1, (2.1) was considered and inverted in Radon [23].
More generally we have (cf. [3'], [15a]) the inversion formula

f(x) (2.2)

Here L is the Laplacian on Rn, x + a is the translate by x

of the p-dimensional subspace a and .~ is a suitably normal-
ized invariant measure on the Grassmann manifold G
p,n of p-
dimensional subspaces of Rn.
Note [Link] exactly when p is even, the inversion formula
is local: the value of f at x0 is determined E.I. the inte-
grals of f over p-planes arbitrarily close to x0 .
Consider now the mapping Gp,n given by the n : G(p,n) ~

parallel translation of a plane to one through 0. The inverse

image n- 1 (a)
a 6 Gp,n is naturally identified with the
of
orthogonal complement a~. We write

F; = (a, x") if a = n(F;) and x" (2. 3)

We define the differential operator o on G(p,n) (the

Laplacian along the fibers) by

<P 6 E(G(p,n)), (2.4)

vertical bar denoting restriction. The operator f ~ f and its

dual

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 154 S. HELGASON

~(~) + ~(x) s JG ~(x+a)dp(a), (2.5)

p,n

intertwine L and o in the sense that

(Lf) (2.6)

Our inversion formula (2.2) can also be written

f(x) = J (oP1 2 r)(x+a)dp(a). (2 .7)

G
p,n

Since the function f satisfies certain consistency conditions

(Rn can be fibered by p-planes in many ways) one would expect
that the integration in (2.2) could be replaced by lower-dimen-
sional integrations. Such refinements are indeed possible as
shown in [5a].
For the range question B, interesting differences appear
between the ranges V(Rn)
rapidly decreasing functions on Rn. Let us write (2.1) in terms
of the convention (2.3). Then
,..
f(O = f(a,x") J f(x'+x")dx'. (2 .8)
a

We consider now the space VH(G(p,n)) of Cm function ~(~) •

on G(p,n) of compact support satisfying the following
moment condition:
(H) For each k € Z+ there exists ~ homogeneous kth degree
polynomial on Rn such that for each aSGp, n the polynomial

P
a,k (u") I a
,~.
.l "'a
(x") <x" ' u">kdx" ,

coincides with the restriction Pkla.l.

We have proved in [15j], Chapter I, §6, that

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 155

f + f is a bijection of V(Rn) onto VH(G(p,n)). (2.9)

An L2-version of this, more complicated to state, was given

by Solmon [25], § 6. For the hyperplane case p = n-1, and (2.9)
takes a more concrete form if we rewrite (2.8) as

f(w,p) = J f(x)dm(x), (2.10)

<x,w>•p
Then

V(R)" consists of the functions <P € V(Sn-l x R) such

that <1>(-w,-p) = <P(w,p) and for each k€ ~+, J<t>(w,p)pkdp
(2.11)
is ! homogeneous polynomial of degree k in the coor-
dinates of w, ([15d], Cor. 4.3).

While the exact analog of this does hold for the image
S (Rn)" under the h;n~eq~lane transform (i.e., the case P • n - 1)

it was shown by Gonzalez [10] that if p < n - 1 the natural

analog of (2.9) for S (Rn) fails.
To indicate the range for these cases, assume first that
p • 1, n • 3, so (2.1) is the integral over a line ~ in R3.
Assuming ~ intersects the plane x3 = 0 we write it in the
form

so

whence

- o. (2.12)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 156 S. HELGASON

Under mild decay conditions on f and on f, John [18b] proved

that the range is the solution space of the differential equation
(2.12). For extensions to arbitrary p < n-1 see [4], [5], and
[11] . The definition of S (G(p ,n)) being a bit awkward we do not
cite precise results, but remark that they imply that in the defi-
nition of VH(G(p,n)), condition (H) could be replaced by differen-
tial equations.

3. THE HOROCYCLE TRANSFORM. Let G be a connected semisimple

Lie group with finite center, G = KAN an !wasawa decomposition.
Let X = G/K be the symmetric space associated wi~h G and o
the origin in X. The orbits ~ in X of groups gNg- 1 conju-
gate to N are called horocycles. We shall now indicate the
solutions to Problem A and B for this case.
The group G permutes the horocycles ~ transitively.
Putting ~ 0 = N•o, the set of all can be written

G/MN (KIM) X A. (3 .1)

Let B, k, a, n and m denote the Lie algebras of the groups

introduced and for H6 a let 2p(H) Tr(adH\n), and denoting

adjoint representation of g and restriction. Let c(A)

denote Harish-Chandra's c-function on a* (the dual of a)
which by [14], I, § 13 and [9a] is explicitly given in terms of
the roots of g with respect to a. It extends meromorphically
to the complexification a*c·

The Radon transform (or the horocycle transform) of function

f on X is defined by

f(O - J f(x)dcr(x)_, ~ 6 ::, (3 2)

0

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 157

where do is the volume element on ~.

We can now describe the general inversion formula for the

horocycle transform obtained in [15,b,e]. Let A denote the
operator on S(A) given by

A 6 a*, (3.3)

denoting the Euclidean Fourier transform

The function c(A-ip)- 1 on a* has each of its derivatives

bounded by a polynomial and the pseudo-differential operator A
maps S(A) into itself. We extend it to an operator on func-
tions on (also denoted A) by just letting it act on the
second variable:

(A$)(kM,a) = Aa($(kM,a)). (3 .4)

Let A* be its adjoint with respect to the volume element d~

on -· Then the horocycle transform f + f is inverted .Qy

L
A

f(g•o) = (AA*f)(g·~)d~(~), g 6 G, (3. 5)

-o

where is the space of horocycles passing through o

(naturally identified with KIM) and d~ is the K-invariant
measure on =o normalized independently of f.
The operator AA* is (via (3.3)) explicitly given in terms
of the structure of G. While the formal similarity between
(2.7) and (3.5) is obvious, the analogy goes deeper. In the case
when p is even, oP 12 is a differential operator so inversion
formulas (2.2) and (2.7) are local as already mentioned. On the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 158 S. HELGASON

other hand, c(A)- 1 is a polynomial if and only if all Cartan

subgroups of G are conjugate so, by (3.3), exactly for these
cases is A (and A*) in (3.5) a differential operator. This
~the condition .2.!!. G which gives ~local inversion of the
horocycle transform: the value of f at x0 ~ determined ~

the integrals of f over horocycles arbitrarily close to x0 •

In these cases the inversion formula can also be stated.

f(g•o) = Dg(J" f(g·~)d~(~)), g 6 G, (3.6)

-o

where D is the lift to G of an explicit differential operator

on X, so here we have the analog to (2.2).

For the case when G is complex several variations of the

horocycle transform are considered in [61 , § 5 and for this case
a formula related to our (3.5) is given. For G = O(n,l) an
inversion formula is given in [7], Ch. V, § 2 and further examples
are discussed in [9bl .
The desirability of an explicit general inversion formula
like (3.5) is stressed in [8], § 1.
We shall now briefly indicate the solution to Problem B,
the range question, as obtained in [15f], § 8 and [1St]. Given
g 6 G define A(g) 6 a by g 6 N exp A(g) K. Let D(A) denote
the set of invariant differential operators on the Euclidean group
A. To each D 6 D(A) we assign a member LD of E(K/M) 9 D(A)
(differential operators on A with coefficients in E(K/M)) by
the following process: The function

A 6 a*,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 159

is a polynomial in A with coefficients in E(K/M). We convert

this into a differential operator L0 (kM) via the Euclidean
Fourier transform F + F on A, i.e.,

(3 .8)

We then have the following result:

A function ~ 6 V(3) • V(K/M x A) belongs to the (3 .9)

range V(X) of the horocycle transform if and only

if for each u 6 K and each D6 D(A) the function

a 6 A, (3.10)

is invariant under the Weyl ~ W.

In this condition, L0 (.) operates on V ( 3) ~ V (KIM x A) in

two ways, namely as a convolution operator in the first variable,
and a differential operator in the second variable.
The range characterization above implies that the range
V(X)A is closed in V(3) not only in the usual topology but also
in the weak topology o( V( 3), E( 3)). This in turn can be used to
prove (compare [15m], p. 226 and [151]) that

E' (X) is closed in E'(3), (3.11)

both in the strong topology of E'(=) and in the weak topology

Our range characterization (3.9) can be made more transpar-

ent by using Fourier decomposition along K. Let IS be an
irreducible unitary representation of K on VB and let vM
0
denote the space of vectors v 6 VB fixed under o(M). Let
d( 0) - dim VB, 1(\S) ~
M
dim VB. For f 6 V(X) put

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 160 S. HELGASON

f 15 (x) = d(ll) IK f(k- 1 •x)ll(k)dk, r15 (l;) - I ~

f 15 (x)dm(x). (3.12)

Then

f(x) f( l;)

where K is the unitary dual of K and KM is the set of ll e K

with 2.(1l) > 0. Since r 15 (ka·~ 0) = ll(k)f 15 (a·~ 0) the range

V(X) 11 is determined by the restrictions flljA·~o which we view

as maps from A to Then (3.9) yields the following
result ([152.]). Here the matrices ·R 15 are related to polynomial
matrices used in Kostant's work [20] in representation theory.

Let ll e KM. There is ~ certain 2.(1l) x 2-(ll)-

matrix R15 (D) with differential operator entries
such that as f runs through V(X) the function
(3.13)
runs through the set of functions
where is
W-invariant.

This range description is concrete enough so that when it is

specialized to the case when X has constant curvature K <0
the Euclidean result (2.11) is obtained by letting K • 0.

4. THE RADON TRANSFORM FOR A DOUBLE FIBRATION. In this section

we provP. some elementary general results about the Radon
transform for homogeneous spaces in duality as introduced in
[15c] (cf. also [15m], Ch. I).

Let G be a connected 1 Lie group, K and H two closed

subgroups. We assume that G, K, H and L = K n H are
1 To begin with, "local compactness" would suffice.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 161

unimodular and that KH is a closed subset of G. We assume

furthermore that

k6K, kH:_HK~ k6H: h6H, hKsKH~ h6K. (4.1)

Let X = GIK, == G/H: we refer to these as homogeneous spaces

in duality. Two elements x 6 X, ~ 6 = are said to be incident
[2] if as cosets in G they have a point in common. We put

v
X = {~ 6 - x and ~ incident}, (4.2)
~ {x 6 X ~ and x incident}. (4.3)

These are again homogeneous spaces. We recall that given bi-

v
invariant measures dg, dk, dh, and dR., each x and each ~

has (because of (4.1)) a canonically normalized invariant measure

dp and dm, respectively. Also
v
x c =• ~ c X are closed
v
subsets and the maps X ~ X, ~ ~ ~ are one-to-one. For the
double fibration

G/L

G/K
/~ G/H
L K n H (4.4)

v
the Radon transform f ~ f and its dual ~ ~ ~ are defined by

f(O (4. 5)

For the invariant measures dx on X = GIK, d~ on = = G/H,

which are determined by dg, dk, and dh, we have the compati-
bili ty relation

(4.6)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 162 [Link]

f S V(X) + f 6 E(3); ~ € V(2) + ~ € E(X) (4.7)

being continuous, we can, as suggested by (4.5)-(4.6), for

s 6 E'(X), o S E'(2) define s S V'(2), d 6 V(X) by

s(~) = s(~), d(f) = o(f), f 6 V(X), ~ S V(2).

This general setup is suggested by examples; for the p-plane

transform we have if M(n) is the isometry group of

X= M(n)/O(n), M(n)/M(p) x O(n-p) (4.8)

while for the horocycle transform in § 2 we have

X = B/K, G/MN (4.9)

so in each case we have a pair of coset spaces of the same group.

EXAMPLE I. Another interesting class of examples is formed by the

symmetric spaces

X = G/K, 2 = G/H

corresponding to two commuting involutions of the same semisimple

Lie group G (see [15(b), § 5] for the simplest case).
We now explain how the general setup can have value for a
specific example.

EXAMPLE II. Let X= G(p,n), the manifold of p-planes in Rn,

and let M = G(q,n) where p+q = n-1. In [15d] we have defined
the transform f + R f
p from functions on X to functions on
by

R
P
f(s) = J
s
(J x3a f(x) dv(x)) dm(a), s e 2, (4.10)

the measures m and v being as in (2.1) and (2.2). Thus in

(4.10) we are integrating over the set of p-planes x

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 163

intersecting the q-plane ~. For n odd the following inversion

formula holds.

(4.11)

where c is a constant independent of f, and is defined

in (2.4). However, in [15d] this formula is only proved for a
certain subspace of functions f. This is because the intersec-
tion of X 6 X and ~ 6 E is in (4.10) allowed to take place
under arbitrarily small angles and this gives rise to convergence
difficulties.
Since the spaces

X= M(n}/M(p) x O(n- p), E = M(n}/M(q) x O(n - q) (4.12)

are homogeneous spaces in duality, our group-theoretic setup in

(4.4) suggests replacing definition (4.10) by the general trans-
forms (4.5). Suppose the origins x 0 6 X, ~ 0 6 E in (4.12) have
been chosen such that the p-plane x0 intersects the q-plane ~ 0
and that the intersection takes place under a right angle. Then
(4.5) becomes

f(~) = J f(x)dm(x), ~(x) = J ~(~)d~(~). (4.13)

x~~.xn~f0 ~~x.~nx~0

This definition was taken up by Gonzalez [10]. For n odd he

proved the inversion formula

(4.14)

and in contrast to (4.11) this holds for all f e V(X). Thus the
group-theoretic viewpoint has led us to a better definition.
Coming back to the general framework, we shall now prove some
simple results for the transforms (4.5). Let p : G + GIK,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 164 S. HELGASON

n : G + G/H be the natural maps and D a right-invariant

differential operator on G. Let p(D) denote the differential
operator on X given by

(p(D)f)op z D(fop), f 6 E(X). (4.15)

Similarly we have a differential operator n(D) on

satisfying

(n(D)<I>) on D(4>on), 4> 6 E(3).

PROPOSITION 4.1. With D as above,

v v
(p( D) f) n(D)f, (n(D)4>) p(D)$.

PROOF. Let xo {K} ' ~0 {H} denote the origins in X and

:::, respectively. For g 6 G Let -r(g) denote the translation
zK + gzK on G/K. Let dhL denote the invariant measure on
H/L induced by dh and d2.. With ~ = gH we have, putting
Hg = gHg -1
etc.,

{zK : gH n zK ~ 0} {ghK h 6 H}

so

~ = -r(g)(H/H n K) = -r(g) • ~ 0 (4.15)

and the measure dm on ~ is the transform by -r(g) of the

measure dhL on ~ 0 H/L. Hence, i f f 6 V (X),

(fon)(g) f(gH) = fH/L f(ghK)dhL fH/L (fop)(gh)dhL• = (4.17)

Since H/L equals the orbit H•x 0 c X which equals the closed
subset ~ 0 , the integration in (4.17) takes place over a compact
set. Hence

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 165

D(fon)(g) = IH/L Dg((fop)(gh))dhL,

which by the right-invariance of D equals

IH/L (D(fop))(gh)dhL = IH/L ((p(D)f)op)(gh)dhL

= (p(D)f) (n(g)). Thus (n(D)f)on = (p(D)f)Aon

as desired.

PROPOSITION 4.2. Assume K compact. Then

(i) f + f is a continuous mapping of V(X) into V(=).
v
( 11) ~ + ~ is a continuous mapping of E( => into E(X).

PROOF. (i) G is connected and K compact so X has a G-

invariant Riemannian structure. In the corresponding metric d,
X is complete. Let FR be the closed ball in X with radius
R, center x0 , and put

Then tR is closed; also if ~ 6 tR then for a suitable g 6 G

so

Since the right hand side is compact (K, FR and are

compact) and tR closed, tR is compact.
Suppose f 6 V R(X). d(x 0 ,x) > R for
F
X 6 ~ so 0. Thus f + f maps V (X)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 166 S. HELGASON

into E(:!) n Cc(:!) = V(:!) and V R(X) into V R(:!). Using

F ~
(4.7), (i) follows.
For (ii) we have

¥<gK) = I K
$(gkH)dk, $ e C(:!). (4.18)

Assuming $n ~ 0 in E(:!) we must show that for each R > 0

v
and each differential operator D on X, D$n ~ 0 uniformly on
A

FR. If gK 6 FR then d(x 0 ,gk•!; 0 ) :S d(x 0 ,g•x 0 ) :S R so gkH

lies in the compact set ~R. Fix E ) 0 and a function
$ e V( :!) such that $ .. on ~R+r:.

ljln = $n on ~R+r:
so by (4.18), ¥n = ljln
v
on FR+r: whence
v
Dv
gn s Dljln on FR. Since $n ~ 0 in E( :!) we have ljln ~ 0 in
v
V(E) so by (4 '7)' 1jl
n
~ 0 in E(X). Thus D$ n ~ 0 uniformly
on as desired.

COROLLARY 4.3. Assume K compact.

v
(i) If s e E'(X) then the mapping $ e E(E) ~ s($) extends s
A

to an element of E'(:!).
(ii) The mapping a e E'(:!) ~ 6 e V'(X) extends to~ mapping
a e V'(:!) ~ ~ e V'(X) .2.x.
V A
a( f) = a (f) f 6 V( X).

In short, Prop. 4.2 implies,

E'(X) ~ E'(:!), V'(:!)v ~ V'(X).

For the classical Radon transform (where X • Rn,

G(n-1,n)) it is known ([15k,m,p.226]) that E'(Rn) is
closed in the weak topology o(E'(:!),E(:!)) and~ fortiori also
closed in the strong topology of E'(:!) as dual of E(:!). See
also [16a, Section 5] for an extension to more general

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 167

distributions. Using just this closure property, Hertle [16b]

deduced the surjectivity of the map ex : <P € E(G(n-l,n)) +<P eE(Rn)
as follows: The transpose t ex is just the Radon transform s + s"
of E I (Rn) into E I (Pn). Since tex is injective the range of a.
is dense. Since the range of tex is closed (as mentioned above)
a standard result on Frechet spaces implies that ex has closed
range; hence the surjectivity E(G(n-l,n))v = E(R?). This argu-
ment now generalizes to the present situation because of Prop. 4.2.

PBOPOSITIOR 4.4. Assume K compact and let N denote the kernel

of s + s" on E I (X) . Assume the subspace E I (X) c E I(!::) is
closed, E 1 (!::) carrying the strong topology. Then

E(!::)v

where {f € E(X): s(f) 0 for s € N}.

PROOF. By general theory, the double annihilator

equals the closure of N in the topology cr(E 1 (X),E(X)). The

dual of E 1 (X) in this topology is E (X) and the dual of E 1 (X)
in the strong topology is the same, E(X) being reflexive. Closed
subspaces, being intersections of hyperplanes, are therefore the
same for both topologies. Since N c E1 (X) is strongly closed
<P + ¥ is a continuous map of
E(!::) into and the transpose maps into E I (X).

By Hahn-Banach, (N~) 1 = E 1 (X)/(N~)~ = E (X)/N so

1 ta. becomes
the map

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 168 S. HELGASON

Since this mapping is injective, a(E(~)) is dense in N~. The

range ta(E'(X)) is strongly closed by assumption so as described
above it is also closed in the topology a(E'(~) ,E(~)). Hence by
standard Frechet space theory, a(E(~)) is closed so E(~)v = N~
as claimed.

REMARK. For the Radon transform on a symmetric space X dis-

cussed in Section 3 the assumption in Prop. 4.4 is satisfied (by
3.11), the kernel N = 0 and N~ ~ E(X).

5. SUPPORT PROPERTIES AND HUYGENS' PRINCIPLE FOR THE WAVE

EQUATION. The following support theorem is a step in the range
characterization (2.11).

Let f 6 S(Rn) and A > 0. Suppose f has surface

integral 0 over every sphere S enclosing the
ball lxl < A. Then f(x) = 0 for lxl > A.

While this result presumably generalizes in some way to

Riemannian manifolds, it has to my knowledge only been extended
to those of constant curvature, namely as follows, [15i].

THEOREM 5.1. Let X= Hn be the hyperbolic space of n

dimensions. Let f 6 C(X) satisfy

sup lf(x)ekd(o,x)l <"' (5.2)

x6X

for each k > 0, d denoting distance and o ~ fixed point in

X. Let B be ~ ball in X and suppose

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 169

for each sphere S enclosing the ball B. Then

f(x) = 0 for X e B.

We shall now point out a consequence of this for the wave

equation on X. Taking X = Hn with curvature -1 we consider
the Cauchy problem for the modified wave equation on Hn x R,

(~+( 1 2 )u
n=_) a u u(x,O)
---
2
u 1 (x). (5.3)
2 at2'

This equation has been studied for example in [12], [15g],

[21b], [26] for n = 3 and for general n in [19] and [15n]
(see also [1]). For n odd, the solution is given by

(5.4)

where en

(5.5)

(Mtf)(x) being the mean value of f on the sphere St(x) in

X. The solution formulas (5.4-5.5) show immediately the validity
of Huygens' principle, namely that the value u(x,t) of the
solution at the point (x,t) depends on the values of the
initial data and only in an arbitrarily thin shell

in Hn. Using Theorem 5.1 we can prove a certain converse of

Huygens' principle.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 170 S. HELGASON

PROPOSITION 5.2. 1!:!. odd) Assume the initial data u0 , u1 in

(5.3) satisfy the decay condition (5.2). Fix s > 0, x 6 X.
Assume the solution to the Cauchy problem (5.3) vanishes in the
two half-cones

{(z,t) d(x,z) < ltl - s.}

Then

u 0 (y) = u1 (y) = 0 for d(x,y) > s.

PROOF. By the homogeneity of Hn we can take x = o. The

transvection of Hn mapping o to z maps the ball Bltl(o)
onto the ball which contains if
ltl > d(o,z) + s.
Put v(x,t) = 1/2(u(x,t) - u(x,-t)). Then v satisfies the
assumption of the theorem with v 0 (x) = 0, v 1 (x) = u 1 (x). Thus,
if t = ch t, t0 = ch t 0 with t 0 = d(o,z) + s,

0 for t > t0 •

Because of condition (5.2) this implies

(Mtu 1 )(z) = 0 if t > d(o,z) + s.

In other words, J u 1 (x)dw(s) = 0 if the sphere S encloses the

s
ball Bs(o). Thus by Theorem 5.1, u 1 (y) = 0 for d(o,y) > s.
Similarly, using the function 1/2(u(x,t) + u(x,-t)) we con-
clude u(y) = 0 for d(o,y) > s so the result is proved.

REMARK. For the wave equation on the flat space R2 m+ 1 the

analogous result was proved by Menzala and Schonbek [22] in a
·similar way on the basis of our (5.1).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 171

6. THE X-RAY TRANSFORM. Let X be a complete Riemannian mani-

fold on which any two points can be joined by a unique geodesic.
The X-ray transform f + f is then defined by

f(Y) J f(x)ds(x), ( 6. 1 )
y

where Y is any geodesic in X, ds the element of arc length,

and f any continuous function on X for which the integral
converges for each Y.
The following result was proved in [15h] on the basis of
Theorem 5.1.

THEOREM 6.1. Let X be~ symmetric space of the noncompact

~. B any ball 1g X. Let f satisfy (5.2) and assume
A

f(Y) = 0 whenever Y n B $.

Then f(x) = 0 for X e B.

REMARK. If rank X> 1 condition (5.2) can be replaced by the

weaker condition f(x) = O(d(o,x)k) for all k.
We shall now prove a similar result relative to horocycles.
Let G be the identity component of the group of isometries of
X and g its Lie algebra. Given Y e g let y be the cor-
responding left invariant vector field on G and define the
differential operator
-
v(Y) on X by

- d
(v(Y)f)(p) = {dt f(exp(-tY)•p)}t=O" (6.2)

Then the mapping v extends to a homomorphism of the algebra

D(G) of left-invariant differential operators onto an algebra
v(D(G)) of differential operators on X. Let a function

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 172 S. HELGASON

f e E(X) be called rapidly decreasin8 on X i f for each

D € v(D(G)) the function Df satisfies (5.2) for each k > 0.
=
Let G = KAN be an Iwasawa decomposition of G and let A+

be the open Weyl chamber corresponding to a- 1N at + e

N i.e. ,
t
if t + + "'· Here at(t > 0) is a half one-parameter group in
A+. A horocrcle in X is an orbit in X of a point x e X
under a group gNg- 1 conjugate to N. All horocycles are
conjugate under G. Let o = {K} denote the origin in X = G/K.
Suppose now X has rank one. The interior of the horocycle
N•o is by definition the set Ut>oNat•o, cf. [3a]. More general-
ly, if t + Y(t) is a geodesic in X, Y(O) = x, we have Y(t) =
gt•x where gt is a one-parameter group of transvections. The
group

NY = {g 6 G (6.3)

is a factor in an Iwasawa decomposition and the set Ut>ONgt•x

is the interior of the horocycle Ny•x.

THEOREM 6.2. Let X be ~ srmmetric space of the noncompact ~

and of rank one. Let f; be~ horocrcle in X and f 6 E(X) .2.

rapidlr decreasing function 1 whose X-rar transform f satisfies

f(Y) = 0 whenever Y n f; 3 0. (6.4)

Then

f(X) = 0 for X e interior of f;,

PROOF. We start with the case when X is the hyperbolic plane

with the Riemannian structure

1rhe proof will show that this assumption can be reduced a bit.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 173

ds2 = dx2+dy2 y > o. (6.5)

y2

The geodesics are the semi-circles

Yu,r : x = u + rcos e, y = r sin e, o <e < n,

and the half-lines x = const., y > 0. Since the horocycles are

permuted transitively by G we can take ~ as the line y- 1.
On Yu,r we have ds = (sin e)- 1 de so (6.1) becomes

Jnf(u + rcos e, rsin e)(sin e)- 1 de. (6.6)

0

Thus our assumption (6.4) amounts to

f(x,y)dw 0, r < 1, (6.7)

y

where dw is the Euclidean arc element. By (6.5) we have

y = ed(o,y) so the rapid decrease of f implies that f(x,y)/y
extends (by reflection) to a smooth function on R2 by
F(x,y) .. f(x, IYI )IIYI· By (6.7) we have

I Sr(x)
F(s)dw(s) = 0,

for X on the x-axis and r < 1• With B (x)

r
as the ball with
boundary Sr(x) we have

I F(u,v)dudv 0, (6.9)
Br(x)
whence
J (a 1 F)(x+u,v)dudv 0, (6.10)
Br(O)

where a1 = a/au. Using the divergence theorem on the vector

field F(x+u,v)a/au we get from (6.10)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 174 S. HELGASON

I F(x+ ,v)udw(u,v) = 0. (6.11)

Sr(O)

Now (6.8) implies trivially

I Sr(O)
F(x+u,v)xdw(u,v) a 0 (6.12)

and by (6.11-6.12)

s = (51 • 52) • (6.13)

Iterating the implication (6.8) => (6.12) we obtain

J F(s)P(s 1 )dw(s) = o,
Sr(x)

where P is an arbitrary polynomial. Since F(s 1 ,s 2 ) = F(s 1 ,-s 2 )

and since the polynomials P(s 1 ) form a separating algebra on
Y
x,r we obtain F = 0 on Sr(x). Thus f:: 0 on the strip
0 < y < 1. This proves the theorem for X= 8 2 .
In the general case we can take ~ as the horocycle N•o
as above and we assume that (6.4) holds. Let 6 be the Cartan
involution relative to K. If Xa is a root vector in the Lie
algebra of N then H = [Xa,6Xa] spans the Lie algebra of A.
If Ga c G is the analytic subgroup with Lie algebra
then Ga •o is a totally geodesic submanifold
of X isometric to H2 (up to a factor). Also the horocycle
exp(tXa)·o is the intersection (Ga·o) n (N•o). By the
first part of the proof, f(a_t•o) = 0 for t > 0. Using this
on the translated function x + f(n•x), n being a fixed element
of N, we get f(na_t•o) for t > 0 and this concludes the
proof.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 175

For any symmetric space X a natural generalization of the

X-ray transform is the flat Radon transform f + f which to a
function f on X associates the integrals

over flat totally geodesic submanifolds E of X of maximal

dimension. The p-plane transform of Section 2 is its infinitesi-
mal counterpart and analogies with the principal results (2.2)
and (2.9) are certainly worth exploring.

BIBLIOGRAPHY

1. T. Branson, "Group representations arising from Lorentz

conformal geometry," (preprint).
2. s. S. Chern, "On integral geometry in Klein spaces,"
Ann. or Math. ·n
(1942), 178-189.
3a. J. Faraut, "Un theorem de Paley-Wiener pour la trans-
formation de Fourier sur un espace Riemannien symetrique de rang
un," J. Funct. Anal. E.2. (1982), 230-268.
3b. B. Fuglede, "An integral formula," Math. Scand. §_ (1958),
207-212.
4. I. M. Gelfand, S. G. Gindikin and M. I. Graev,
"Integral geometry in affine and projective spaces," J. Soviet
Math. Vol.~ (1982), 39-167.
5.
I. M. G~lfand, M. I. Graev, and s. J. Shapiro. a)
"Integ~al geometry on k-dimensional planes;" Functional Anal.
Appl. 1 (1967), 15-31. b) "Differential forms and integral
geometry," Functional Anal. Appl. l (1969), 24-40.
6. I. M. Gelfand and M. I. Graev, "Geometry of homogeneous
spaces, representations of groups in homogeneous spaces and
related problems of integral geometry, I." Amer. Math. Soc.
Transl. 37 (1964), 351-429 (transl. from Transact. Moscow Math.
Soc • .§. · Ci959)) .
1. I. M. Gelfand, M. I. Graev, and N. Vilenkin, Generalized
Functions, Vol. 5, Academic Press, New York (1966).
8. S. G. Gindikin, "Integral geometry as geometry and as
analysis," these proceedings.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 176 S. HELGASON

9. s. G. Gindikin and F. I. Karpelevi~, a) "Plancherel

measure of·Riemannian symmetric spaces of non-positive
curvature," Dokl. Akad. Nauk, SSR 145 (1962), 252-255. b) "On a
problem of integral geometry," Sel-:-Math. Sovietica, 1 (1981),
169-184 (transl. from Chebotarev Memn. Vol., Kazan, 1964).
10. F. Gonzalez, "Radon transforms on Grassmann manifolds,"
Thesis MIT (1984).
11 • E. L. Grinberg, "Euclidean Radon transforms, ranges and
restrictions," these Proceedings.
12. P. Guenther, "Ueber einige specielle Probleme aus der
Theorie der linear der linear partiellen Differentialgleichungen
zweiter Ordnung," Ber. Verh. Saechs. Akad. Wiss. Leipzig 102
(1957), 1-50. -
13. V. Guillemin, "Perspectives in integral geometry," these
Proceedings.
14. Harish-Chandra, "Spherical functions on a semisimple Lie
group I, II," Amer. J. Math . .§_Q_(1958), 241-310; 553-613.
15. S. Helgason, a)"Differential operators on homogeneous
spaces," Acta Math. 102 (1959), 239-299. b) "A duality in inte-
gral geometry; some generalization of the Radon transform. Bull.
Amer. Math. Soc. 70 (1964), 565-601. c) "A duality in integral
geometry on symmetric spaces," Proc; U. s. - Japan Seminar in
Differential Geometry, Kyoto (1965)~ Nippon Hyronsha, Tokyo
(1966). d) "The Radon transform on Euclidean spaces, compact
two-point homogeneous spaces and Grassmann manifolds," Acta Math.
113 (1965), 153-180. e) "Radon-Fourier transforms on symmetric
spaces and related group representations," Bull. Amer. Math. Soc.
71 (1965), 757-763. f) "The surjectivity of invariant differen-
tial operators on symmetric spaces," Ann. of Math. 98 (1973),
451-480. g) "Solvability questions for invariant·differential
operators," Proc. 5th Int. Colloq. on Group Theoretical Methods
.!.!!. Physics, Montreal (1976), Academic Press (1977). h) "The X-
ray transform on a symmetric space," Proc. Conf. Diff. Geom. and
Global Anal. Berlin (1978), Lecture Notes·in Math. No; 838,
Springer-Verlag, New York (1980). i~pport of·Radon trans-
forms," Advan. Math. 38 (1980), 91-100. j) The Radon Transform,
Birkhaeuser Basel andBoston (1980). k) "Ranges""""Ot'the Radon
transforms," AMS Short Course on Computerized Tomography, Jan.
(1982) Proc. Symp. Appl. Math. Amer. Math. Soc. Providence
(1983). l)"The range of the Radon transform on a symmetric
spaces~" in Proc. Conf. Representation Theory of Reductive Lie
Groups, (P. Trombi ed.) Utah (1982), Birkhaeuser, Boston (1983).
m) Groups and Geometric Analysis, Academic Press, New York
(1984). n)li"Wave equations on homogeneous spaces," in Lie lirogp
Representations III, Lecture Notes in Math No. 1077, pp. 25 -2 7,
Springer-Verlag, New York (1984).
16. A. Hertle, a) "Continuity of the Radon transform and its
inverse on Euclidean space," Math. Z. 184 (1983), 165-192. b)
"On the range of the Radon transform and its dual," Math. Ann.
267 (1984), 91-99.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME RESULTS ON RADON TRANSFORMS 177

17. H. Jarchow, Locally Convex Spaces, Stuttgart, Teubner

(1981).
18. F. John, a) "Bestimmung einer Funktion aus ihrer
Integralen·ueber gewisse Mannigfaltigkeiten," Math. Ann. 109
( 1934), 488-520. b) "The ultrahyperbolic differential equation
with 4 independent variables," Duke Math. J. ! (1938), 300-322.
19. I. A. Kiprijanov and L. A. Ivanov, "The Euler-Poisson-
Darboux equation in a Riemannian space," Soviet Math. Dokl. 24
(1981), 331-335. -
20. B. Kostant, "On the existence and irreducibilitty of
certain seties of representations," in Lie Groups and Their
Representations, Halsted, New York, 197~
21. P. D. Lax and R. s. Phillips, a) Scattering Theory for
Automorphic Functions, Ann. of Math. Studies 87, Princeton Univ.
Press~ Princeton, N. J. (1976). b) "Translation representations
for the solution of the non-Euclidean wave equation," Comm. Pure
Appl. Math. 32 (1979), 617-667.
22. G. P. Menzala and T. Schonbek, "Scattering frequencies
for the wave equation with a potential term," J. Funct. Anal. 55
(1984), 297-322. -
23. J. Radon, "Ueber die Bestimmung von Funktionen durch
ihre Integralwerte Laengs gewisser Mannigfaltigkeiten," Ber.
Verh. Saechs. Akad. Wiss. Leipzig §2 (1917), 262-277.
24. H. H. Schaefer, Topological Vector Spaces, Springer-
Verlag, ·New York (1971).
25. D. C. Solmon, "The X-ray transform," J. Math. Anal.
Appl. 2i (1976), 61-83.
26. R. S. Strichartz, "Explicit solutions of Maxwell's
equation on a space of constant curvature," J. Funct. Anal. 46
(1982)' 58-87.
DEPARTMENT OF MATHEMATICS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
CAMBRIDGE, MASSACHUSETTS 02139

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

CLASSICAL INTEGRAL GEOMETRY IN RIEMANNIAN HOMOGENEOUS SPACES

RALPH HOWARD

1. INTRODUCTION. Let G be a Lie group and K a closed sub-

group of G. If M and N are compact submanifolds of the
homogeneous space G/K then a good deal of energy in integral
geometry has gone into computing integrals of the following type,

J I(M n gN) nG(g) ( 1 .1)

G

where nG(g) is the invariant measure on G and where I(M n gN)

is an "integral invariant" of M n gN. For example in the case
G is the group of isometries of Euclidean space M and N
are submanifolds of and I(M n gN) • vol (M n gN) then
evaluation of (1 .1) leads to formulas due to Poincare, Blaschke,
Santalo and others (See [10] for references) or in the same case
if I(M n gN) be one of the integral invariants arising in the
Weyl tube formula then the evaluation of (1.1) gives the Kine-
matic formula of Federer [4] and Chern [2]. In the case G is
the unitary group U(n + 1) acting on complex projective space
C Pn, then letting I(M n gN) = vol(M n gN) in (1 .1) leads to
results of Santalo [9] or letting I(M n gN) be the integral of
a Chern class leads to the recent Kinematic formula of Shifrin
[11]. In this paper we will assume G/K has an invariant
Riemannian metric and discuss a unified method of evaluating of
(1.1) and also a method of transfering integral geometric
® 1987 American Mathematical Society
0271-4132/87 [Link] + $.25 per page

179
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 180 RALPH HOWARD

formulas from one homogeneous space to any other homogeneous

space with the same isotropy subgroup.
In outline the method (which to a large extent is based on
ideas in the papers of Federer [4] and [5] and especially the
paper [1] of Brothers) is as follows. First assume M and N
be submanifolds of the Lie group G which is assumed to have a
left invariant Riemannian metric. Let f : M x N + G be
-1
f(~, n) = ~n . We apply the Coarea formula (which is stated in
Section 2) to f and interpret the result geometrically. The
result of this (the "basic integral formula" of Section 2) is
that for any M and N submanifolds of G and ~ "any" inte-
gral invariant of M n gN, the integral (1 .1) can be evaluated
in a very straightforward manner.
In the general case, that is M and N submanifolds of a
Riemannian homogeneous space G/K, let IT G + G/K be the
natural projection and apply the basic integral formula to the
submanifolds rr- 1M and rr- 1N of G. The result of this can
then be pushed back down to G/K. This leads to the evaluation
(1 .1) in this case. For the most part the proofs are only
sketched (although the energetic reader should be able to fill in
the details in Sections 2 and 3).
One advantage to this method is that it makes it clear that
the form of a integral geometric formula in a homogeneous space
G/K does not depend on the full group of motions G, but only
on the invariant theory of the isotropy subgroup K. This allows
us to transfer integral geometric formulas proven in one homo-
geneous space to any other homogeneous space with the same iso-
tropy subgroup. Several examples of this are given in Sections 4
and 5 to show that the methods here are not only useful for

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 181

proving more general formulas, but also well adapted to doing

concrete calculations.
In section 6 the general Kinematic formula for submani-
folds of a Riemannian homogeneous space is given.
The notation and terminology is standard. By "smooth" we
mean class c=. If M is a smooth manifold then TM is its
tangent bundle and ™x its tangent sapce at x. If f : M+ N
is a smooth map then f*x : ™x + TNf(x) is the derivative of f
at x 6 M. We regard discrete subsets S of a manifold as
submanifolds of dimension zero and in this case the volume of S
is defined to be the number of ~oints in s. If M is a
Riemannian manifold then OM will denote the volume density on
M by OM. Then OM can be thought of as either a measure on M
or as the absolute value of one of the two locally defined volume
forms on M. (Thus integration over M is defined even when M
is not orientiable.)
I would like to thank Ted Shifrin for some conversations on
integral geometry which got me hooked on the idea of trying to
understand Kinematic formulas in the context of Riemannian homo-
geneous spaces. I would also like to thank Ted and Mike Gage for
suggesting the name "transfer principle" for the ideas in Section
4. Finally, it is a pleasure to acknowledge the organizers of
the integral geometry conference at Bowdoin College for giving me
a chance to speak and for making my stay there a very pleasant
week.

2. THE COAREA FORMULA AND A BASIC INTEGRAL GEOMETRIC FORMULA. If

Mm+n and Nn are Riemannian manifolds and f : Mm+n + Nn is a
smooth map then the fiber of f over y 6 Nn is the preimage

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 182 RALPH HOWARD

f- 1 [y] = {x 6 M : f(x) = y}. ( 2. 1 )

(The fiber may be empty). While the structure of any one fiber
can be quite pathological (any closed subset of Mm+n can be
realized as the preimage f- 1 [y] of some smooth map f : Mm+n + N
and some y 6 N) it is a basic result in differential topology
that for "most" y 6 N the fiber f- 1 [y] is well behaved. The
precise statement is: GEOMETRIC SARD'S THEOREM. If f : Mm+n + Nn
is~ smooth map then for almost all y 6 N the fiber f- 1 [y] is
either ~ or an imbedded m dimensional submanifold of Mm+n.
In fact, more is true. A point y 6 N is a regular value
of f if for all x 6 f- 1 [y] the differential : TM +TN y
X
is surjective. (Thus if f- 1 [y] is empty, then y is a regular

value). For a regular value y it is a consequence of the

implicit function theorem that f- 1 [y] is a m dimensional
submanifold of M. Sard's theorem is that almost every y 6 N
is a regular value of f. See [6] Chapter 3 for details.
For each regular value y give f- 1 [y] the Riemannian
metric it inherits as a submanifold of Mm+n and denote the
induced volume density by n 1 • If ~ : Mm+n + R is smooth
f- [y]
and compactly supported then the function defined on N by

y + L,
f [y]
~(x)n
f
_1
[y]
(x) (2.2)

is defined for all regular values y of f and thus for almost

all y 6 N. (When f- 1 [y] is empty, set the integral in (2.2) to
zero.) A basic operation in integral geometry is the computation
of "fiber integrals".

f f_ 1 ~(x)O _1 (2.3)
N f [y] f

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 183

For one method of evaluating this we first define

the Jacobian Jf of f. If x e M and the linear map
f*x :
TMm+n +
X
n
TNf(x) is surjective let e1 ' ... ,en be
.L
an orthonormal basis of Kernel(f*) and set
),
Jf(x) ~

II f *xe1 " • • • A
f*xenll· (2.4)

If f*x is not surjective then define Jf(x) m 0. The main

result on fiber integration is then the

SMOOTH COAREA FORMULA. Let f : Mm+n + Nn be a smooth map and

Mm+n R a Borel measurable function such that
+
- ----- ---- ----
x Jf(x) is integrable M. Then the function on N
+ ~(x) 2Q
-----
defined ~ (2.2) is integrable and

f f_ 1 [y] ~(x)O
N f f
_1
[y]
ON(y) = f ~(x) Jf(x) OM(x). (2.5)

REMARK. There is a much deeper version of the coarea formula due

to Federer (see [4] or [5]) where the map f is only assumed to
satisfy a Lipschitz condition. The version given above is good
enough for most applications to smooth integral geometry and is
reasonably straightforward to prove. (However I know of no
reference for an explicit proof other than [7]).
Before we give our main integral geometric result, some
difinitions are needed. If V and W are subspaces of an inner
product space, v1 • ... ,vp is an orthonormal basis of v, and
w,' ... ,wq is an orthonormal basis of w, then define the
"angle" o(V, W) between v and w by

(2.6)

Here, as usual, the length of a decomposable vector

x 1 " • • • " xn is given by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 184 RALPH HOWARD

(2. 7)

Note that
0 :0 o(V,W) c o(W,V) :0 1. (2.8)

with o(V, W) = 0 if V n W ~ {0} and o(V, W) if V and

W are orthogonal.
Let G be a Lie group with a left invariant Riemannian
metric, < >. That is, if L~ is the left translation
L~(g) = ~g then each L~ is an isometry of (G, < , >). Left
translation allows all the tangent spaces to G to be identified
with the tangent space TGe to G at its identity element e.
Therefore it is possible to define the angle between subspaces of
tangent spaces to G at different points. To be precise, if V
is a subspace of TG~ and W is a subspace of TGn, then define

o(V,W) = o(L~~ 1 *V' Ln- 1 * W) (2.9)

It follows that for all g 6 G
o(V,W) o(Lg* V,W) = o(V,Lg* W). (2.10)

Recall that the group G is unimodular i f and only if the left

invariant density OG is also right invariant. With these
preliminaries, our main integral geometric formula can be stated.

BASIC INTEGRAL FORMULA. Let G be a unimodular Lie ~with ~

left invariant metric < , >. Let M and N be submanifolds of

G with dim(M) + dim(N) ~ dim(G), and let ~ : Mx N ~ R be
any Borel measurable function on M x N such that the function
(~, n) ~ ~(~, n)o(T~M~, T~Nn) is integrable. Then

ff
G MngN
~<~.g- 1 ~) 0 MngN(~)nG(g)
(2.11)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 185

REMARKS. (1) The fo~mula (2.9) is ve~y closely ~elated to the

fo~mula of theorem 5.5 in the paper [1] of B~othe~s. (2) The

assumption the g~oup G is unimodular is not needed, howeve~ in

the gene~al case the formulas become cluttered with facto~s

involving the modula~ function of G.

SKETCH OF THE PROOF. The idea of the proof is to apply the

coarea formula to the function f : M x N ~ G defined by

(2.12)

and interp~et the ~esult geomet~ically.

A di~ect application of (2.5) yields

J f_ 1 $ n _1 OG(g) = J J $(~,n)Jf(~.n)OMxN(~,n). (2.13)

G f [g] f [g] MxN

Set k = dim(M) + dim(N) - dim(G). Then a calculation (involving

some rathe~ unpleasant multilinea~ algebra) yields

(2.14)

Fo~ each g 6 G it is easily ve~ified that the map

Wg ·• M n gN ~ f- 1 [g] given by Wg(~) = (~,g- 1 ~) is a bijection,

and that when g is a regula~ value of f both M n gN and

f- 1 [g] are smooth manifolds of dimension k and wg is a

diffeomorphism. Anothe~ calculation shows, for any vecto~ X

tangent to M n gN that, hg*X II = 12 II X II • Thus when g is

a ~egular value of f a change of va~iable yields

(2.15)

Putting (2.14) into (2.15) and (2.15) into (2.13) completes the

proof.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 186 RALPH HOWARD

3. POINCARE'S FORMULA IN HOMOGENEOUS SPACES. On of the oldest

results in integral geometry is Poincare's formula for the aver-
age number of points of intersection of two plane curves. More
specifically let c1 and c2 be curves in the Euclidean plane
R2 of length L1 and L2 and let G he the group of orienta-
tion preserving isometries of R2 with its usual left invariant
metric. Then

(3.1)

where #(c 1 n gc 2 ) is the number of points in c 1 n gc 2 • We now

give a generalization of this to Riemannian homogeneous spaces.
To fix the notation in this section G will be a Lie group
and K a compact subgroup of G. Let G/K be the homogeneous
space of left cosets ~K of K in G. Then G can (and will)
be viewed as a group of tranformations acting on G/K by letting
g 6 G send ~K to g~K. If IT : G + G/K is the natural
projection and e is the identity element of G then ll(e)
will be called the origin of G/K and denoted by 0. It will be
assumed G has a left invariant metric < , > that is also
right invariant by elements of K. Then there is a unique
Riemannian metric on G/K (also denoted by < , >) which makes
the natural projection n : G + G/K into a Riemannian submer-
sion. Thus for each ~ e G, the restriction of n*~ to
kernel(ll*~)i is a linear isometry of kernel(ll*~)i with
T(G/K)n~· This metric is invariant under the action of G on
G/K. Conversely every metric on G/K which is invariant under
G is induced in this manner from some left invariant metric on
G which is also right invariant by elements of K.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 187

We now need to define the angle between subspaces tangent to

G/K at different points as was done in the case of subspaces
tangent to G. In the latter case the subspaces were left trans-
lated to the identity element and the angle then computed. If V
is a subspace of T(G/K}x and W is a subspace of T(G/K}Y
then we can choose ~. n 6 G so that ~(O) = x, n(O) = y and
try to define the angle between V and W to be the angle
-1 -1
between ~ *V and n* w. . This does not work as it is not
independent of the choice of ~ and n. This problem can be
overcome by averaging over all possible choices of n.

DEFINITION. If x, y 6 G/K and V is ~subspace of T(G/K)x

and W is ~ subspace of T(G/K}y then define

(3. 2)

where n are any elements of G so that x and

~.
--- ~ ( o) =

n(O) y (.££.what.!!!_ the same thing II(~) = x, II(n) ~ y).

Straight forward calculations show this is independent of
the choice of ~. with II(~) = x, II(n) a y and for all g 6 G

(3.3)

Also the inequality 0 ~ aK(V, W) ~ vol(K) holds.

POINCARE'S FORMULA FOR HOMOGENEOUS SPACES. Let M and N be

submanifolds of G/K of finite volume. Assume dim(M) + dim(N)
~ dim(G) and that G is unimodular. Then

f vol(M
G
n gN)"G(g)
"
= ff
MxN
aK([Link] , [Link] )OM N(x,y).
X Y x
(3. 4)

COROLLARY. If G .!!!_transitive ~ the set of tangent spaces to

each of M and N then

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 188 RALPH HOWARD

(3. 5)

where ~ any element of M and is any element of N.

(The right side of (3.5) is independent of the choice of and
y0 Qz the transformation rule (3.3).)

REMARKS. (1) The corollary is in Section 5 of the paper [1] of

Brothers. His proofs are harder as he proves the result in the

case M and N are normal currents; thus much more analysis is
involved. (2) As with the results of the last section it is
possible to do away with the assumption G is unimodular.

SKETCH OF THE PROOF. Let g

preimages of under the natural projection rr : G + G/K. Apply
the basic integral formula to the submanifolds M and N of G
with the function ~ 2 to get

(3. 6)

For those g where " n gN

M " is a smooth submanifold (and
g 6 G) the restriction of II to " n gN
this is almost every M

is a Riemannian submersion with each fiber isometric with K.

thus vol(M n gN) = vol (K) vol (M n gN) • Therefore

IG vol(M "
n gN)OG(g) = vol (K) I vol
G
(M n gN)OG(g). (3.7)

It remains to interpret the right side of (3. 6) in terms of

the geometry of M and N. The map (!;' n) + (II!;,IIn) from

M N X is a Riemannian submersion with fiber n- 1[x] X n-1 [yJ

isometric to K X K. If !;X and ny are elements of G with

IT(!;x) = X and II(ny) = y then the isometry is given by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 189

(a,b) ~ (~xa'nyb). Therefore the right side of (3.6) can be

rewritten as

(3. 8)

where straightforward (but tedious) calculations yield

~(x,y) = f f o(T~M~ a'T~Nn b)OKxK(a,b) (3. 9)

KxK X y
= f f o((~xa)*
KK
-1 ~ -1 -1
T Mx,b* ny*T Ny)OK(b)OK(a)
~

~ ~
= vol(K)oK(T MX ,TN y ).

Putting this into (3.8) and the result of that and (3.7) into
(3.6) completes the proof.

4. THE TRANSFER PRINCIPLE AND EXAMPLES. We now show how it

is possible to transfer integral geometric formulas proven for a
homogeneous space G/K to any other homogeneous space which has
the same isotropy group. To be precise let G, K and the
metrics on G and G/K be as in paragraph 3.1. Let G' be
another Lie group of the same dimension as G and K' a compact
subgroup of G' of the same dimension as K. Assume that G'
has a left invariant [Link] < , >' which is also
right invariant by elements of K'. give G'/K' the Riemannian
metric that makes the natural projection of G' onto G'IK' a
Riemannian submersion. Assume
(A) There is an isomorphism p : K ~ K' and a linear
isometry ~ : T(G/K) 0 ~ T(G'/K') 0 , that intertwines p, that is

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 190 RALPH HOWARD

(B)
vol(K) vol( K').

TRANSFER PRINCIPLE. With notation~ above let x,y 6 G/K,

X' , y' 6 G' /K' and V ~ linear subspace of T(G/K) , V'
X

subspace of T( G' IK') x'' w ~subspace of T(G/K)y' W' ~

subspace of T(G' /K') y'. Assume there are elements F,;,n e G and
!',;' t Tl' e G' so that
- -- !',;(0) = x, n(O) = y, E;;'(O') s x', n'(O') = y'

and

( 4. 1 )
Then
(4. 2)

The proof is nothing more than a change of variable in the

integral defining oK(V, W) and is left to the reader.

EXAMPLES. We now show by example how the constant

.L .L
oK(T M ,T N ) in equation (3.5) can be computed by evlauating
xo Yo
the integral JG vol(M n gN)QG(g) for the proper choice of M
and N. The transfer principle can then be used to transfer the
value of this constant to other homogeneous spaces with the same
isotropy subgroup K. The first two examples are well known
while the others seem to be new. In these examples the values of
all constants will be expressed in terms of the volumes of the
standard spheres sk (the set of unit vectors in Rk+ 1 ). These
volumes have the well known values where r is the gamma
function.

k+l
2(1f;z
r(~)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 191

(a) We start with the case where G/K has constant sectional
curvature. The group of orientation preserving isometries of sn
is the matrix group SO(n + 1) (the group of real orthogonal
matrices with determinant +1) and the isotropy subgroup of sn
at the north pole is the subgroup SO(n) imbedded in SO(n + 1)
in the natural way. Therefore SO(n + 1) /SO ( n) R sn. To define
a Riemannian metric on SO(n + 1) we follow [8] vol. 2,
pg. 264-264. First define an inner product < > on the
'
vector space of (n + 1 ) )( (n + 1) real matrices by
<A,B> • 1/2 tr (AB*) (B* transpose of B) and give SO(n + 1)
the metric it inherits as a submanifold of this inner product
space. This metric is both left and right invariant by
elements of SO(n + 1) and makes the natural projection
n : SO(n + 1) + SO(n + 1)/SO(n) • Sn into a Riemannian
submersion. Thus vol(SO(n + 1)) • vol(SO(n)) vol(Sn) and
so by induction

vol(SO(n+1)) = vol(S 1 )vol(S 2 ) ••• vol(Sn). (4.3)

Because SO(n + 1) is transitive on the set of p planes

tangent to sn and also on the set of q planes tangent to sn
w.L) is the same for all p planes V
the value of oSO(n) ( V.L ,
and q planes W tangent to sn. This value is easily computed
by letting M • sP (imbedded as a totally geodesic submanifold)
and N = Sq in equation (3.5) and noting that sP n gSq is
isometric with Sp+q-n for almost all g 6 SO(n + 1). The
result is

vol(Sp+q~n) volLSO(n~
(4.4)
vol(SP)vol(Sq)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 192 RALPH HOWARD

Now let G/K be any simply connected Riemannian manifold of

constant sectional curvature c, where c can be positive,
negative or zero. Let G be the group of orientation preserving
isometries of GIK. Then the isotropy subgroup is isometric with
SO(n) and we assume that the volume of K is normalized so that
if MP is any p dimensional submanifold of GIK, Nq any q
L L
dimensional submanifold, x 6 M, y 6 N that oK(T Mx,T NY) is
given by the value on the right of (4.4). Thus if M and N
have finite volume equation (3.6) implies

vol(Sp+q-n)vol(SO(n+1)) vol(MP)vol(Nq)
vol(SP)vol(Sq)

In particular this holds when M and N are compact submani-

folds of Euclidean space. See the book [10] of Santal6, para-
graph 15.2, for another derivation when G/K is Euclidean.
(b) This time we consider complex analytic submanifolds of
Kahler manifolds of constant sectional curvature. To begin let
CPn be the complex projective space of n complex (and 2n
real) dimensions. Then the group U(n + 1) (the group of
(n + 1) K (n + 1) complex unitary matrices) acts on CPn in a
natural way. The stabilizer of a point in CPn is then
U(1) K U(n). Therefore CPn can be realized as a homogeneous
space as CPn = U(n + 1)/(U(1) K U(n)). Put a Riemannian metric
on U(n + 1) by first putting a real inner product < , > on
the (n + 1) K (n + 1) complex matrices by

<A,B> = ~ real part of trace (AB*)

(where B* conjugate transpose of B) and giving U(n + 1)

the metric induced on it as a submanifold of this inner product

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 193

space. This me~ric is invariant under both left and right trans-
lations by elements of U(n + 1). Give CPn the metric that
makes the natural projection U(n + 1) + U(n + 1)((U(1) x U(n))
- CPn a Riemannian submersion. (For details of the construe-
tion just outlined see [8] vol. 2, pages 273-278). With this
metric CPn is a Kaehler manifold such that all holomorphic
sectional curvatures are 4 and all the totally real sectional
curvatures are 1. There is a Riemannian submersion of s 2 n+ 1
onto CPn (the Hopf fibration) that fibers with fibre s 1•
Thus

Considering s 2 k+ 1 as the set of unit vectors in ck+ 1 we see

that U(k + 1) acts transitively on s 2 k+ 1 and that the stabi-
lizer in U(k + 1) of a point of s 2 k+ 1 is conjugate to U(k).
Thus s 2 k+ 1 2 U(k + 1)/U(k) and the natural projection induced
from U(k + 1) to s 2 k+ 1 is a Riemannian submersion. Therefore
vol(U(k + 1)) vol(s 2 k+ 1 )vol(U(k)) and whence

vol(U(n + 1)) = vol(S 1 )vol(S3) ••• vol(S 2 n- 1 )vol(S 2 n+ 1 ).

If MP is any complex submanifold of CPn of complex

dimension p and Nq is a complex submanifold of complex
.1. .1.
dimension q then the number oU( 1 )~u(n)(T Mx 0 ,T Ny 0 > with
x 0 S M, Yo S N is independent of x 0 , y 0 , M and N. Therefore
it can be computed from equation (3.4) by letting MP = cpP and
Nq = CPq (imbedded as totally geodesic submanifolds) and noting
that Mp n gNq = CPp+q-n = CPp+q-n for almost all g S U(n + 1).
This yields

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 194 RALPH HOWARD

(T ~ p ~ q ) vol(CPp+q-n)vol(~~n+1))
0 U(1)xU(n) Mx 0 •T NYo =
. (4.5)
vol(CPP)vol(CPq)

Now let E be any simply connected Kaehler manifold of

constant holomorphic sectional curvature c and complex dimen-
sion n. Then E can be realized as a homogeneous space G/K
where K is smoothly isometric with U(1) x U(n) and G acts
on E by Kaehler isometries. Normalize the metric on K so
that vol(K) = vol(U(1) "U(n)). Let MP be any complex subman-
ifold of E = G/K of complex dimension p and Nq any complex
submanifbld of complex dimension q. Then by the transfer
principle the number oK(T~Mx ,T~NY x0 6 M, y0 6 N is given by
the right side of (4.5). Therefore equation (3.5) yields that if
M and N have finite volume

(c) In this example we again let E be the simply connected

Kaehler manifold of constant holomorphic sectional curvature c
and complex dimension n. Let J be the almost complex struc-
ture on E. Recall that a submanifold MP of real dimension p
is totally real if and only if JTMx is orthogonal to ™x for
all x6 M. Suppose MP is totally real and Nq is a complex
submanifold of complex dimension q with p + 2q ~ 2n. If M
and N have finite volume then

JGvol(MP n gNq)OG(g) = JGvol(Nq n gMP)oG(g)

vol(RPp+ 2 q- 2 n)vol(U(n!_ill vol(MP)vol(Nq).

vol(RPP)vol(CPq)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 195

Here RPk is real projective space with its metric of constant

sectional curvature one. It is double covered by sk, thus
(RPk) = 1/2 vol(Sk), In the case that E = GIK = CPn the
formula is proven by letting MP = ftpP imbedded in CPn as a
totally real and totally geodesic submanifold of CPn, N = CPq
imbedded as a totally geodesic submanifold, verifying that M n gN
= ftpP+ 2 q- 2 n for almost all g 6 U(n + 1) and using this in
equation (3.5) to compute The general case
then follows from the transfer principle.
(d) If RPn is imbedded in CPn as a totally real and
totally geodesic submanifold then, for any g 6 U(n + 1) for
which RPn and g RPn intersect transversely, #(RPn n g RP)
= n + (here #(A) = number of points in A). Just as in the
last two examples this implies that if Mn and Nn are totally
real submanifolds of E, a simply connected Kaehler manifold of
constant holomorphic sectional curvature, then

JG#(Mn n gNn)OG(g) = (n+1)vol(U(n+1)) vol(Mn)vol(Nn).

vol(RPn) 2

5. HYPERSURFACES IN TWO POINT HOMOGENEOUS SPACES. We now give a

less trivial application of the transfer principle. Recall that
a Riemannian homogeneous space G/K is a two point homogeneous
space if and only if the action of K on the unit sphere of
T(G/K) 0 is transitive. This easily implies that G is transi-
tive on the set of tangent spaces to any hypersurface in G/K.
The two point homogeneous spaces have been classified ([7] page
295) and in all cases the group G is unimodular.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 196 RALPH HOWARD

THEOREM. Let GIK be a two point homogeneous space of

dimension n. Let Mp be a p dimensional submanifold of GIK
and let Nn-1 a hyper-surface of GIK. If Mp and Nn-1 have
finite volume then

JGvol(MP n gNn- 1 )oG(g)

( 5 .1)
vol(K)vol(SP- 1 )vol(Sn) vol(MP)vol(Nn-1)
vol ( sP) vol ( ;n- 1-)-

REMARK. This result is somewhat surprising: In most cases G

will not be transitive on the set of tangent spaces to MP.

PROOF. Identify Rn with the tangent space T(G/K) 0 of GIK

at 0. Let K ~ Rn be K x Rn with the product Riemannian
metric and view it as a group of transformations on Rn by the
rule (a,v)X = a*X + v. The group K ~ Rn then acts on Rn by
isometries. Let V be any p dimensional subspace of Rn at
0 and let BP be the unit ball in v. Then the translations of
Rn, and thus also K ~ Rn, is transitive on the set of tangent
spaces to BP. Let sn- 1 be the unit sphere in Rn = T(G/K) 0 •
Because G/K is a two-point homogeneous space, the group K ~ Rn
is transitive on the set of tangent spaces to sn- 1 . Note that
with the obvous notation SO(n) ~ Rn is the group of orientation
preserving isometries of Rn and thus the results of example
(a) of section 4 apply to this group. By corollary 3.7, example
(a), obvious symmetry properties of the sphere Sn-1 , and that
vol(SO(n + 1)) = vol(Sn)vol(SO(n)) for any y 0 6 sn- 1 ,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 197

oK(v"\r\sn- 1 ) )vol(BP)vol(Sn- 1 )
Yo
s JRn JK vol(BP n (a*Sn- 1 + v))GK(a)GRn (v)

vol(K)
vol(SO(n))
JRn JSO(n)vol(B p n (b*S
n-1
+v))GK(b)GRn (v)

= vol(K)
T.~
vol(Sp- 1 )vol(SO(n+1)) vol(B p) vol ( s n-1 )
vol(SO\nJJ vol(SP)vol(Sn-1)

vol(K)vol(SP- 1 )vol(Sn)vol(BP)
vol(SP)

so that

vol(K)vol(SP- 1 )vol(Sn)
(5.2)
vol(SP)vol(Sn- 1 )

Now let MP, Nn- 1 be as in the theorem, and let x 6 MP,y 6 Nn- 1 .

Because V was an arbitrary p dimensional subspace of Rn =T(G/K) 0

it follows from the transfer principle that oK(TiM~,TiN~-l) is
given by the right side of (5.2). The theorem now follows from
Poincare's formula (3.4). This completes the proof.

6. THE KINEMATIC FORMULA FOR SUBMANIFOLDS OF HOMOGENEOUS SPACES.

In this section G, K, G/K and the Riemannian metrics on these
spaces will be as in Section 3. We now give a class of integral
invariants of compact submanifolds of G/K. Loosely these will
be integrals (over the submanifold) of polynomials in the compo-
nents of the second fundamental form of the submanifold where,
for this to be defined, the polynomial must be invariant under
the isotropy subgroup K in an appropriate sense. In making
this definition it is useful to distinguish the case where G

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 198 RALPH HOWARD

is transitive on the set of tangent spaces to the submanifold

from the general case.

DEFINITON. Let v0 be a p dimensional subspace of T(G/K) 0 .

Then ~ p dimensional submanifold MP of G/K is of ~ v0
if and only if for all x 6 M there ~ ~ E; 6 G with

E;*VO = ™x·
Recall that if M is a submanifold of G/K then the second
fundamental form of M in GIK at X 6 M is a symmetric
bilinear map from ™x x ™x to T~Mx. (See [8], vol. 2). For
each linear subspace of T(G/K} 0 let

II(V 0 ) = vector space of all symmetric

( 6. 1 )
bilinear forms from v0 x v0 to v0 •

The elements of II(V 0 ) should be thought of as the elements of

the second fundamental forms of submanifolds of G/K which pass
through 0 and have v0 as tangent space at 0. Let K(V 0 ) K
be the subgroup of elements that stabilize v0 , that is

( 6. 2)

The group K(V 0 ) acts on II(V 0 ) in the natural way, that is if

-1 -1
and h 6 II(V 0 ) then (ah)(u,v) a a*h(a * u, a * v).
Since II(V 0 ) is a vector space it makes sense to speak of
polynomials on II(V 0 ). Then P is invariant under K(V 0 ) if
and only if P(a~h) P(h) for all a 6 K(V 0 ).

DEFINITION. Let v0 be ~ subspace of T(G/K) 0 and P ~ ~

nomial on II(V 0 ) invariant under K(V 0 ). Then for

which~
- ---
each compact submanifold M of GIK (possibly with boundary) of
~ v0 define, for each x 6 M.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 199

(6.3)

where ~ is any element of G with ~*VO • ™x (this is

independent of the choice of ~ ~ the invariance of P) and
define

IP(M) = J P(h~)OM(x). (6.4)

M

The set of integral invariants IP just defined has the

basic invariance property IP(gM) = IP(M), that is IP(M) is
independent of the position of M in G/K up to G motions.
To define integral invariants for submanifolds M of G/K
when G is not transitive on the set of tangent spaces to M we
extend the second funcamental form of M at x to a bilinear
map on T(G/K)x x T(G/K)x with values in T(G/K)x.

DEFINITION. If M ~ ~ submanifold of G/K then

- -,- - EXTENDED
the
SECOND FUNDAMENTAL form of M in G/K at X

metric bilinear map from T(G/K)x x T(G/K)x to T(G/K)x given

H~(u,v) • h~(Pu,Pv) (6.5)

where P ~the orthogonal projection of T(G/K)x onto ™x·

The extension of the definitions is now easy. Let

EII(T(G/K) 0 ) = vector space of symmetric bilinear maps from

T(G/K) 0 x T(G/K) 0 to T(G/K) 0 •

Then K acts on EII(T(G/K 0 ) in the same way K(V 0 ) acted on

II(V 0 ). If M is a submanifold of GIK, x eM, ~ e G so that

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 200 RALPH HOWARD

~(0) = x and if P is a polynomial on EII(T{G/K) 0 ) invariant

under K then

~-1
P(H~) = P(H 0 M)

is defined independent of the choice of ~ with ~(0) = x.

Therefore IP(M) can be defined by

IP(M) = I P(H~)nM(x).
M

with this definition we still have that IP(gM) IP (M) for all
g e G.

THE KINEMATIC FORMULA. Let G, K, G/K be as in Section 3 and

assume G is unimodular. Let v0 and w0 be linear subspaces
of T(G/K) 0 with dim(V 0 ) + dim(W 0 ) ~ dim(G/K) and let P be a
polynomial on EII(T(G/K) 0 ) such that
(a) P is homogeneous of degree 1 and invariant under K.

(b)

Then there is a finite set of pairs (Qa' Ra) such that

(1) each Qa is a homogeneous polynomial on II(V 0 ) invariant
under K(V 0 ).
(2) each Ra is a homogeneous polynomial on II (W0 ) invariant
under K(W 0 ),
( 3) degree (Qa) + degree (Ra) = l!. for each a, and
(~) for all compact submanifolds M of G/K of type vo and
N of type w0 (they may have boundaries) the Kinematic formula

I Ip(M n gN)OG(g) = l: IQa(M)IRa(N) (6.5)

a

holds.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 201

OUTLINE OF THE PROOF. As in Section 3 let A = 'If-1M and

" = 1r -1 N. I f cp is defined on MX N by
N
-1
cp(t;,n) = P(HMnt;n
'Iff;
N) (6.6)

Then

( 6. 7)
A A

The restriction of 1r to M n gN is a Riemannian submersion

onto M n gN with fibre isometric with K. Also on each fibre
11"- 1 [x] cp(t;, g- 1 t;) has the constant value P(H~ngN). Therefore

fA Acp(f;,g- 1 t;)OM(f;) (6.8)

MngN
vol(K) J P(HMngN)n (x)
MngN x M
= vol(K) Ip (MngN).

Using this in the basic integral formula (2.11) yields

(6.9)

For each (x, y) e M x N choose tx' ny e G with c;x*VO

• ™x•ny*WO • TNY. Then, as in the proof of Poincar~'s formula,

ff
A A
MxN
(6.10)
-fI
where

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 202 RALPH HOWARD

p
vol(K)IK(x,y) (6.12)

with

p
IK(x,y)

I P(H !; x- 1Mn b - 1 ( ny- 1N) )o((!; 1 J. -1 -1 J.

) T Mx,b* (ny)* T Ny)OK(b)
K 0 X (6.13)

Combining (6.9) - (6.13) gives, at least on the formal level,

(6.14)

This reduces the proof of (6.5) to the concrete problem of

evaluating the integrals in some usable form. This
can be done in several ways and one of them leads to (6.5).

REMARKS. (1) It is possible to drop the assumptions that G is

transitive on the tangent spaces to M and N and the assump-
tion G is unimodular. The resulting formulas are too messy to
be of much interest. (2) In the case where G/K is a simply
connected manifold of constant sectional curvature and G is
either the group of all isometries or all orientation preserving
isometries of G/K then the condition (b) of the Kinematic
formula can be replaced by the inequality t ~ dim(M) + dim(N) -
dim(G/K) + 1. (3) A transfer principle holds for the Kinematic
formula just as it did for Poincar~'s formula. Rather than give
its exact statement we give an example. Let G be the group of
all orientation preserving isometries of the three dimensional

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CLASSICAL INTEGRAL GEOMETRY 203

Euclidean space R3 (with the same normalizations as in example

(a) of Section 4. Then in his paper [3], C. S. Chen proved
that if M and N are compact surfaces in

JJ K 2 ds nG(g)
G MngN
21r 3Area(M)J (2H 2 + llh11 2 )nN 21T3Area(N>f (2H 2 + llhll 2 )nM
N M

where K is the curvature of the curve M n gN,

H2 = ((A.l + >.. 2 );2/ is the square of the mean curvature, and
>.2 + >.2 is the square of the length of the second
II hll 2 1
~

2
fundamental form (here >. 1 and >. 2 are the principal
curvatures). By the 'transfer ·principle, this formula holds
exactly as written for all compact surfaces in any three dimen-
sional simple connected space of constant sectional curvature.

BIBLIOGRAPHY

1. J. E. Brothers, "Integral geometry in homogeneous

spaces," Trans. Am. Math. Soc. 124 (1966), 480-517.
2. s. s. Chern, "On the Kinematic formula in integral
geometry,"·J.·Math. Mech . .1£ (1966), 101-118.
3. C. s. Chen, "On the Kinematic formula of square of mean
curvature," Indiana Univ. Math. J. 22 (1972-3).
4. H. Federer, "Curvature measures," Trans. Amer. Math.
Soc. §1 (1959), 418-419.
5. H. Federer, Geometric Measure Theory, Springer, Berlin
(1960).
6. M. W. Hirsh, Differential Topology, Springer-Verlag
(1976).
7. R. Howard, "The Kinematic formula in Riemannian
geometry," to appear as a Memoir of the A.M.S.
8. S. Kobayashi and K. Nomizu, Foundations of Differential
Geometry I, II, Wiley (Interscience), New York, (1963 and 1969).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 204 RALPH HOWARD

9. L. A. Santalo, "Integral geometry in Hermitian spaces,"

Amer. J. Math, 74 (1952), 423-434.
10. L. A. Santalo, Integral Geometry and geometric
Probability, Addison-Wesley, Reading, Mass-rf976).
11. T. Shifrin, "The Kinematic formula in complex integral
geometry,"·Trans. Amer. Math Soc., vol. 264, no. 2 (1981), 255-
293.
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF SOUTH CAROLINA
COLUMBIA, SOUTH CAROLINA 29208

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]

Contemporary Mathematics
Volume 63, 1987

DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS

KENNETH D. JOHNSON

1. INTRODUCTION AND NOTATION. In 1968, E. M. Stein and G. Weiss

([6]) introduced a generalization of the classical Cauchy~Riemann

equation to real n-space. They based this generalization on the

representations of the group SO(n). They also developed a
function theory for solutions of these Cauchy-Riemann equations,
extending some of the classical results of holomorphic functions.
They showed that solutions of these systems are harmonic, and
they developed an HP-theory for solution spaces. Although vari-
ants of this theme have been studied to obtain representation
theoretic results (e.g. Schmid [5], Knapp and Wallach [4]), this
work remained neglected for some ten years; however, recent work
of J. Gilbert, R. Kunze, R. Stanton, and P. Tomas ([ 1 ]) focusing
on extending the analytic results of [6], has rekindled interest
in this paper.
The purpose of this paper is to extend the notion of Cauchy-
Riemann equations to Cartan motion groups, and show that, for
Cartan motion groups of classical type, solutions of the Cauchy-
Riemann equations are also solutions of a large system of second
order equations which include the Laplace equation.
Let K be a compact, connected Lie group with T a maximal
toroidal subgroup and E+{T) a positive root system, and
-- •busing notation ~- set D(K) equal to the set of highest
© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

205
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 206 KENNETH D. JOHNSON

weights, A, of complex irreducible representations of K posi-

tive with respect to our choice of E+(T). If K = K1 • K2 , the
local direct product of K1 and K 2 , we assume as we may --
that our choice of toroidal subgroups of K1 and K2 and posi-
tive root systems are compatible so that

For A 6 D(K) let nA : K + GL(VA) be the corresponding repre-

sentation on the complex vector space VA. For A, A' 6 D(K)
let VA* VA' be the unique K-invariant and irreducible sub-
space of VA Q VA'' of highest weight A+ A'. (In what fol-
lows, we shall have no further need to refer to T, E(T) or

To define the notion of a Cartan motion group, let S be

the category whose objects consists of all pairs (G,K) where G
is a noncompact, connected, semisimple Lie group and K is a
maximal compact subgroup. A morphism from (G 1 ,K 1 ) to (G 2 ,K 2 )
will be a homomorphism from G1 to G2 which takes K1 into

K2.
For (G,K) 6 S let g(resp. k) be the Lie algebra of
G(resp. K) Now g has a Cartan decomposition, g = k + p,
where p is the orthogonal complement to k with respect to the
killing form. Since p is AdK-invariant, we may define the
semidirect product of K and p, K • p, by setting

(G,K) 0 • K • p

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CAIITAN MOTION GROUPS 207

we obtain a functo~

0 S + G~

whe~e G~ is the catego~y of g~oups. Let s0 be the image of

S unde~ this functo~.

DEFINITION 1.1. A Ca~tan motion g~oup is an object in s0 • A

g~oup (G,K) 0 • K • p 6 s0 is said to be classical if G is a
classical type, and is called i~~educible if the ~eal ~ep~esenta-

tion Ad : K + GL(p) is !~~educible.

Suppose H = K • p is an !~~educible Ca~tan motion group.

Fix A 6 D(K) and suppose f is a em-function from p to VA.
then f may be identified with f : H +VA where

f(k,X) = ~A(k)f(X).

In order to define the Cauchy-Riemann equations, it is necessary

to consider two types of irreducible Ca~tan motion g~oups.

DEFINITION 1.2. (i) A Cartan motion g~oup K • p is said to be

of type (a), if the representation

Ad : K + GL(pC)

is irreducible.
(11) A Cartan motion group K • p is said to be of type (a)
if it is !~~educible but the rep~esentation

Ad K + GL(pC)

is not irreducible.
If H = K • p is a Cartan motion g~oup of type (a), f~om

the theory of symmetric spaces there is a K-inva~iant complex

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 208 KENNETH D. JOHNSON.

structure J on p. Hence, if p+(resp. p_) is the

+i(resp. -i)-eigenspace of J, Pc = p+ + p_, and the repre-
sentations of K on p+ and p_ are irreducible and inequiv-
alent. We now fix for each group K • p of type (8) a K-
invariant complex structure J on p.
Suppose F is a C~-function on p to a complex vector
space W.
If u e p set

d
Cl(u) F(x) dtF(x+tu)!t 0

If {e 1 , ••• ,er} is an orthonormal basis of p define

VF p + W 6il Pc

by setting

r
VF(x) L a(ej) F(x) 6il ej.
j=1

DEFINITION 1.3. (i) Suppose H = K • p is an irreducible Cartan

motion group and let A e D(K) - {0} = D'(K). If H is of type
(a), a c~-function f p +VA is said to satisfy the Cauchy-
Riemann equations if

(ii) If H is of type (8), a C~-function f P +VA is said to

satisfy the Cauchy-Riemann equations if

Vf : p + VA *p +

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 209

Finally, let V(p) be the algebra of constant coefficient

differential operators on p of finite order, and let J = V(p)K
be the ring of K-invariant operators in V(p). We shall refer
to a function F on p as strongly K-harmonic if

DF = 0

for all D 6 J+ where

{D e ; 01 O}.

In Section 2, we prove Theorem 1, a result for groups of

type (B), which although quite simple provides a prototype for
our main result. In Section 3, we develop some algebra concern-
ing classical groups and describe the classical Cartan motion
groups of type (a). In Section 4 we state and prove our main
result.

2. GROUPS OF TYPE (B). Fix H = K • p, a Cartan motion group of

type (B), and suppose A 6 D'(K). Let f : p +VA be a solution
of the Cauchy-Riemann equations. As

Vf : p + VA*p,

a fortiori

Vf : p +VI\ 6il p.

Suppose J is the K-invariant complex structure on p which is

ion P+, and let {e 1 ,Je 1 , ... , en,Jen} bean orthonormal
basis on p. For S a S n, set

fa = ea + i Je a , f a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 210 KENNETH D. JOHNSON

and

A simple calculation now yields

Vf

and so

af
Cl.
= 0 (1 $ Cl. $ n).

These are the standard Cauchy-Riemann equations.

THEOREM 1 • For f as above let S be the linear span of all

aBaa (1 $ a, B $ n) . That is ,

Then S is a K-invariant subspace of V(p), for D6 S

Df = 0,

and the ideal of operators generated by S, I(S), contains J+.

PROOF. The fact that S is K-invariant is clear. So it remains

only to prove that J+ c I(S). The proof of this fact may be
found in [3] Section 5.

ALGEBRAIC REMARKS. In this section we study some basic proper-

ties of the fundamental representations of K where K is
either SU(n), Spin(n), or Sp(n). Recall that

SU(n) = {A 6 GL(n,C) : AA* = I, det A - 1},

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 211

Sp(n) {A 6 GL(2n,e)

and Spin(n) is the universal (2-fold) cover of

SO(n) ={A 6 GL(n,R) : AAt I, det A 1} •

If K = SU(n) or Spin(n) set N = n, while if K = Sp(n) set

N = 2n. If z e en let E(Z) be the exterior multiplication on
vectors. Observe that, for any of the above choices of K, the
linear map

given by setting

E(w Q z) = E(z)w,

commutes with the action of K.

For K = Spin(n) (resp. Sp(n)), let ( , ) be the nonde-
generate K-invariant symmetric (resp. skew-symmetric) bilinear
form on eN X eN. Then the linear map

given be setting

I(w Q z) = t(z)w

is K-invariant, where for u 1 , ..• ,ur 6 eN

If K = SU(n) let ( , denote the K-invariant nondegenerate

bilinear pairing on eN X eN* Then we may define t(z)(z 6 eN*)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 212 KENNETH D. JOHNSON

analogous to the cases for- Spin(n) and Sp(n), and obtain that
the cor-r-esponding linear- map

given by setting

I(w Q z) = t(z)w,

is K-invariant. For w 6 AreN, y 6 eN, and z 6 eN for K=

Spin(n) or Sp(n), Whl. le z 6 eN* f or- K = SU( n ) , we obtain

(t(Z)E(y) + E(y)t(z))w (z,y)w. ( 3. 1 )

The fundamental repr-esentation spaces of SU(n) ar-e the

spaces Ar-en for- :;; r' ::> n-1; those of Sp(n) ar-e spaces u
r'
Ar-e2n for- 1 :;; r :;; n; those of Spin(2n) ar-e the spaces Ar-e2n

for :;; r' ::> n-1 and the r-epr-esentation spaces w+ and w_, the
two half spin r-epr-esentations; and, those of Spin(2n+1) are the
spaces Ar-c 2 n+ 1 for- 1 :;; r :;; n -1 and the r-epresentation space,
W, the spin r-epr-esentation.
If K = Spin(2n) the linear- map

given by setting

M(w Q z) =z • w,

is K-invar-iant. Observe that for- w 6 W±, z, y 6 c 2 n

(z • y + y • z) • w = -(z,y)w (3. 2)

If we r-eplace Spin(2n) by Spin(2n+1) and W± by W we

obtain the same for-mula.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 213

As our main result deals with classical Cartan motion groups

K • p of type (a), it is useful to have a list of these groups.
Since these groups are derived from irreducible symmetric spaces,
a complete list is obtained from Helgason [2] p. 341. In this
list, we shall find it useful to list certain matrices of differ-
ential operators. We do this using the following convention.
If Z represents an arbitrary p x p hermitian or p x q

complex or real matrix, az will denote the matrix with (i,j)-

entry a If Z is an arbitrary p x p real or complex
azij ·
symmetric or skew-symmetric matris, az will denote the matrix
with (i,j)-entry

1. K = SU(n), p = {Z:Z Z* is an n x n complex matrix trZ O}.

Set D = az.
2. K = SO(n), p {X:X xt is an n X n real matrix trX 0} •

Set D = ax.
3. K = SO(n), p {X:X -xt is an n X n real matrix}.
Set D = ax.
~~· z 1 , z 2 are n x n complex matrices}
4. K = Sp ( n) , p =
{
1-
-z_z-12 z1 . z * = z
1 1, z2
t
= -z 2 , tr z 1 = o

Set D = (-:~1z :~z2 ]

1 2

[~; z~ ]· z 1 , z 2 are n " n complex matrices }

l
5. K Sp(n), p - { -z · t
1 z1 zr. z2 = z2

Set D [:~1z2
= :~~ z2
6. K = SO(p) x SO( q), · p {X X is a p " q real matrix}

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 214 KENNETH D. JOHNSON

7. K

Set

Up to finite coverings of K, the above 7 cases give a complete

classification of all K and p for the classical type (a)
Cartan motion groups. The action of K on p in all of the
above cases is clear.

OUR MAIN RESULT. Keeping the notation in the classification of

the 7 cases of Section 3, we now state our main theorem.

THEOREM 2. Let H = K• p be a classical Cartan motion group of

A 6 D'(K). a C -function f p +VA
CD
type (a) and let Suppose :

is a solution of the Cauchy-Riemann equations.

(2) Then, i f K is simple (cases, 1 ,2,3,4,5 of Section 3) and D
is the matrix of these examples, D2 f = 0.
(ii) In case 6, let K1 = Spin(p) and K2 = Spin(q). Then write
A = A1 + A2 where A1 6 D(K 1 ) and A2 6 D(K 2 ). If A1 f. 0'
DtDf "' o, and i f A2 = 0, DDtf = 0.
(ii) In case 7 let K1 = Sp(p) and K2 = Sp(q), and write
A = A1 + A2 where A1 6 D(K 1 ) and A2 6 D(K 2 ). If A1 f. 0,
DtDf = 0, and if A2 f. 0, DDtf = o.

PROOF. A complete proof ~s given by an ad hoc verification of

each case. However, as there is a similarity of procedure in
every case, we shall give the proof for H= K • p in case 2
and 7.
Suppose K = Spin(n) and p • {X : X • xt is an n x n
symmetric matrix, tr X • 0}. Identifying p with symmetric

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 215

2-tensors and letting {e 1 , ••• ,en} be the standard orthonormal

basis of Rn we have, using Einstein summation convention

Vf

Suppose first that VA is a subrepresentation of

VA' •" ArCn fo"'• some "'• ...~ n 2- 1 • Th en wr1· t 1ng,
· ·
us1ng th e summa t 1on
·
convention, fa. : p +VA' and
we have

afa.
Vf = lc1
2
+ oij ) ----
axij
Q w~
...
Q e 1. ® eJ.•

Using the fact that the maps I and E of Section 3 commute

with the action of Spin(n), we have

and

Applying V to each of these equations and contracting with E

in the first and I in the second yields

and

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 216 KENNETH D. JOHNSON

Summing and using equation (3.2) we have

and this yields D2 f = 0.

If VA is not contained in VA' Q ArCn- 1 , VA is contained
in the tensor product of spin or half-spin representation. So,
if n is even VA c VA' Q W±, while i f n is odd VA
VA' Q W. Replacing E and by Clifford multiplication our
result follows from Equation 3.2.
We now prove (iii) for the case A1 I 0. Writing an
arbitrary matrix of p as

(1 ::; I ::0 2p, 1 ::0 J ::0 2q)

and identifying p with a subspace of tensors, we have

Using the fact that any fundamental representation of Sp(p) is

contained in Arc2P ( 1 ::; r ::; p) we have for some A' that

VA c vA I
Q Arc2P.

Writing f = f a Q wa where fa : p _.VA' and

wa e Arc 2 P we have from Section 3

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 217

and

o.

As in the preceding case, we have

a2 f
a~IJa~KL g (ei,eK)wa g eJ g eL

a2 f
= (ei,eK) a~I~a~KL g eJ g eL = o.

Rewriting (~IJ) in terms of the zi 's we obtain

This concludes our proof in this case. The remaining cases
follow along the same lines.

THEOREM 3. Let H = K .p and f be as in Theorem 2. If H

is not one of the following two groups, f is strongly K-
harmonic.
(1) H is as in Case 3 and n even.
(ii) H is as in Case 6 and p = q.

PROOF: If K is simple and not as in (i), the ring of K-

invariants of V(p), is generated by the operators

(k ~ 2)

and hence we are done if K is simple.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 218 KENNETH D. JOHNSON

If H is as in Case 6 and p ~ q then J is generated by

the operators

(k ~ 1)

and hence we are done in this case.

In Case 7, J is generated by the operators

( k ;:: 1 )

and hence we have our result for every case.

CONCLUDING REMARKS. 1. As the. proof of Theorem 2 consists of ad

hoc methods it does not generalize easily to groups of exception-
al type. It would be useful to have an intrinsic proof which
would generalize to exceptional (non classical) groups.
2. It is unlikely that the exceptions to Theorem 3 actually
occur, especially if one applies the work of Knapp and Wallach
[4] to the group S0(2n,2n). One might in fact try to describe
all D e V(p) which annihilate solutions of the Cauchy-Riemann
equations.
3. Although we shall not examine similar Cauchy-Riemann equations
for semisimple groups here as in [1], one might ask whether there
are second order systems similar to those of Theorem 2 which
annihilate solutions of the Cauchy-Riemann equations. We remark
that for the group SL(n,R) such systems do exist.

BIBLIOGRAPHY

1. J. E. Gilbert, R. A. Kunze, R. J. Stanton, and P. A.

Tomas, "Calderon-Zygmund higher gradients and representation
theory," Proc. Aygmund Conf. Univ. of Chicago, (1983).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 DIFFERENTIAL OPERATORS AND CARTAN MOTION GROUPS 219

2. s. Helgason, Differential Geometry and Symmetric Spaces,

Academic Press, New York (1962) .
.
3. K. Johnson and A. Kor,nyi, "The Hua operators on bounded
symmetric domains of tube type," Ann. of Math. ill (1980), 589-
608.
4. A. W. Knapp and N. R. Wallach, "Szego kernels associated
with discrete series," Inv. Math; ~ (1976), 489-578.
5. W. Schmid, "On the realization of the discrete series of
a semisimple Lie group," Rice Univ. Studies .2§. (1970), 99-108.
6. E. M. Stein and G. Weiss, "Generalizations of the Cauchy-
Riemann equations and representations of the rotation group,"
Amer. Jour. of Math. 90 (1968), 163-196.
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF GEORGIA
ATHENS, GEORGIA 30602

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

AN L2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM

FOR THE OSCILLATOR REPRESENTATION

Lisa A. Mantini

1. INTRODUCTION. This lecture concerns an integral transform

which is a direct-image mapping from the realization of the
oscillator representation in b 2 -cohomology into the space of
holomorphic sections of a vector bundle over complexified
Minkowski space. The transform intertwines the natural actions
of the conformal group, SU(2,2), on L2 -cohomology and on sec-
tions, and it is injective. In addition, the sections in the
image of the transform satisfy the massless field equations.
This transform is thus analogous to the Penrose transform, except
that it is defined on square-integrable Dolbeault cohomology in-
stead of classical Dolbeault cohomology. We use L2 -cohomology
because it is a Hilbert space in which SU(2,2) acts unitarily, so
that the transform provides a geometric construction which uni-
tarizes the "ladder" representations of SU(2,2). The transform
defined here also exists in higher dimensions, so we will con-
sider representations of U(p,q) with arbitrary p and q.

2. MINKOWSKI SPACE AND THE PENROSE CORRESPONDENCE. In this

section we recall the construction of solutions of the massless
field equations over complexified compactified Minkowski space
which is provided by the Penrose correspondence, so that the

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

221
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 222 LISA A. MANTINI

analogy with our result will be more apparent. The results

described here are due to Penrose [9],. [10], Woodhouse [13], and
Eastwood, Penrose, and Wells [3]. The summary we present is
based on the survey article of Wells [12].
Recall that by Minkowski space H we mean equipped
with the Lorentz metric, a symmetric bilinear form b of
signature (1 ,3). If w & (t,x,y,z) 6 M, then
b(w,w) = t 2 - x 2 - y 2 - z 2 . The Lorentz group 0(1,3) is the
group of all linear isometries of Minkowski space, and the con-
formal group C(1,3) is the group of all conformal transforma-
tions of (compactified) Minkowski space. The space H can be
identified with the space H(2) of hermitian 2 x 2 matrices,
via the transformation

t+x y+izl
w = (t,x,y,z) ~ [ . = w
y-lZ t-x

which satisfies b(w,w) = det w. We remark that SL(2,C) doubly

covers 0(1 ,3) 0 , the identity component of the Lorentz group,
since any A e SL(2, C) determines a [Link] transformation
w ~ *AwA of H(2) which preserves the determinant and which is
the identity i f A = -1 2 . We will show later that SU(2,2)
provides a 4-1 cover of C(1 ,3) 0 .
Our goal is to study differential equations on M, includ-
ing their symmetry properties under the above-mentioned
transformation groups of M. First, we express these differen-
tial equations on a related complex manifold. Let h denote a
hermitian form of signature (2,2) on c 4. If V is a subspace
of c4 we say that V is positive (resp. negative) if hiV is
positive definite (resp. negative definite), written hiV >> 0

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 223

(reap. hjV « 0).Similarly, we say that a subspace v of c4

is null if
+
hjV "' 0. We define subsets M , M
- and Mo of the
'
Grassmannian G(2,C 4 ) of 2-planes in c4 to be the sets of
positive, negative, and null 2-planes in c 4, respectively.
Note that M0 is the Shilov boundary of M+.
We use coordinates on c4 for which the matrix of h is
given in 2 x 2 block form as

(2.1)

·For any 2 X 2 complex matrix v, if we have fixed a choice of

coordinates on c4, we let V( v) denote the 2-plane spanned by
the columns of

[;J
LEMMA 2.1: The injection M + Mo given by

w ..... V(w)

is conformal, with dense image, and therefore M0 is the pre-

viously-mentioned conformal compactification of M.

PROOF: Using coordinates in which h1 is the matrix of h, we

easily see that V(w) is a null plane and the set of all such is
dense in M0 • 0

REMARK: The spaces M+ and M0 are orbits of the action of

SU(2,2) on G(2,C 4 ), and this action is by conformal transfor-
mations. The 4-element center of SU(2,2), generated by u 4,
clearly acts trivially, thus we see why SU(2,2) gives a 4-1
cover of C(1 ,3) 0 •

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 224 LISA A. MANTINI

Henceforth, we refer to M0 as compactified Minkowski

space and G(2,C 4 ) as complexified, compactified Minkowski space
(complexified Minkowski space, for short). As in (2.1), we use
coordinates on c4 for which the matrix of h is given by h1•
We then see that V e M+ if and only if V = V(~) for some
~ 9 M2 x 2 (C) satisfying -i(~ - *~) >> 0, so that ~ has posi-
tive definite hermitian imaginary part. Thus, we may think of
M+ as an upper half space in complexified compactified Minkowski
space.
We wish to exhibit the complexified forms of the wave equa-
tion and Maxwell's equations on M+. The simpler of the two is
the wave equation, whose complexified form on M+ is given by

(2.2)

where $ is a complex-valued function on M+. Note that as the

hermitian imaginary part of ~ approaches zero, ~ approaches
its hermitian real part (~ + *~)/2
(t+~ y+izl =
y-1z t-x J for some
w=
w = (t,x,y,z) 9 M. Therefore, as we pass to the Shilov boundary
M0 of M+, the operator det(a/a~) "approaches" the wave
operator

Thus, i f $ is a solution of (2. 2) which extends suitably to

M0 , f = $IMO determines a complex-valued solution of the classi-
cal homogeneous wave equation on M.
The classical form of the homogeneous Maxwell's equations on
H is given by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN 1 2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 225

as
at + v X E 0, v . B 0,

(2.3)
aE - v
at X B 0, v .E 0,

where V denotes the vector of differential operators

and the vector fields E = (E 1 ,E 2 ,E 3 ) and B = (B 1 ,B 2 ,B 3 ) on H

denote the electric and magnetic fields, respectively. We may

express Maxwell's equations more succinctly in the exterior
algebra of M. We use the Hodge *-operator on differential
forms which is compatible with the Minkowski metric
dt 2 - dx 2 - dy 2 - dz 2 on AT*(H). Specifically, * is given on
2-forms by

where the set {xi'xj,xk,xR.} = {t,x,y,z}. The sign is determined

by the Minkowski norm of dxiAdxj multiplied by the signature of
the permutation taking (xi ,x j'xk,xR.) into (t,x,y,z).
Maxwell's equations may now be expressed in the form

da = o, d*a = o, (2. 4)

for ' 6 A2 T*(H), where the coefficients of a determine the

electric and magnetic fields, as follows:

Note that, on 2-forms, * 2 = -1. Thus, if we consider complex-

valued 2-forms, will decompose under the action of *

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 226 LISA A. MANTINI

into the direct sum of +i and -i eigenspaces

A2 T*(H)
c
A2
+
(il A2 . (2. 5)

Bases {w 1 ,w 2 ,w 3 } of A2 and {w4,w5,w6} of A2 are given

+
by

w1 = (dy-idz)A(dt+dx),

w2 = (dy-idz)A(dy+idz) - (dt+dx)A(dt-dx),
w3 (dt-dx)A(dy+idz),
w4 (dt+dx)A(dy+idz),
w5 (dy-idz)A(dy+idz) + (dt+dx)A(dt-dx),
w6 = (dy-idz)A(dt-dx).

Any 2-form B may therefore be written as a sum B

where 8+ 6 A2 + is given by

B+ =% (E 2 +iB 2 +i(E 3 +iB 3 ))w 1 -% (E 1 +iB 1 )w 2

-% (E 2 +iB 2 )-i(E 3 +iB 3 ))w 3

and B e A2 is given by

B lf2 ( - ( E2- i 8 2 ) + i ( E3 - i 8 3 )) w4 + lf2 ( E1 - i 8 1 ) w5

+% (E 2 -iB 2 )+i(E 3 -iB 3 ))w 6 .

We refer to elements of as self-dual, and elements of

as anti-self-dual. Maxwell's equations now appear in the form

dB = 0. (2.6)

The decomposition (2. 5) is invariant under the Lorent·z group

SL(2,C). The two 3-dimensional pieces A2 are each isomorphic,
±
as SL(2,C)-modules, to the natural representations of SL(2,C)
on the space P(2,C 2 ) of homogeneous polynomials of degree 2

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN 1 2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 227

on being the holomorphic action, and A2 being the

antiholomorphic action. We will use this simpler representation-
theoretic model for the functions we wish to construct on
complexified Minkowski space. Let ~ = (~ 0 .~ 1 .~ 2 ) and
' = (~ 0 •• 1 .~ 2 ) be vector-valued functions on M+ such that
~(~). •<~) 6 P(2,C 2 ) for any ~. The functions ~. ~ will
determine the electric and magnetic fields, as follows:

~0 ~ [E 2 +iB 2 +i(E 3 +iB 3 )J, ~0 Y2 [-(E 2 -iB 2 +i(E 3 -iB 3 )J,

w, -Y2 (E 1 +iB 1 ), ~, = 112( E1 - i B1 ) ,

1P2 - -1/2 [E 2 +iB 2 -i(E 3 +iB 3 )~, ~2 ~ [E 2 -iB 2 +i(E 3 -iB 3 )J.

The complexified form of Maxwell's equations on M+ is then

given by

0. i 1 • 2. j 0. 1 ; (2. 7a)

and

a a
-a- ' j - -a- ~j+1 0, j 0,1. k 1 • 2; (2. 7b)
~,k ~2k

+
As we pass to the Shilov boundary M0 of M, ala~,, "ap-
preaches" a/at + a lax, etc. Thus, if solutions 'j and ~k

of (2.7) extend suitably to their restrictions to

determine solutions of (2. 3) on M.

The equations (2.7) are referred to as the massless field

equations of helicity ±1, with (2. 7a) providing the positive
helicity, or self-dual, solutions and (2.7b) providing the nega-
tive helicity, or anti-self-dual, solutions. We may also
consider the massless field equations of arbitrary helicity ±m/2
on complexified Minkowski space. Using the holomorphic and anti-

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 228 LISA A. MANTINI

holomorphic representations of SL(2,C) on the space P(m,C 2 ) of

homogeneous polynomials of degree m on we would similarly
require functions ljl, cp on such that
for all ]J.

The construction of solutions of the massless field equa-

tions over complexified Minkowski space which is provided by the
Penrose correspondence is based on the following double
fibration.

Here P; is the subset of complex projective 3-space consisting

of all positive lines in c 4, the subset F; 2 of the flag
manifold F 12 is the set of all positive lines contained in
positive 2-planes in c 4, and p1 and p2 are the natural
projections. The set-valued mapping T is defined by
T = p2 o p1 -l so that if L e P3, T(L) = the set of all positive

2-planes containing L, and if V e M+, T-l (V) = (the set of all

lines in V) = CP 1 . Let O(m) denote the sheaf of holomorph1c
sections of the mth power of the hyperplane section bundle on
cP 1 . The Dolbeault cohomology H0 • 0 (CP 1 , O(m)) is given by

P(m,C 2 ), m ~ 0,

{ 0} , m < 0.

By Serre duality,

m ~ 0.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 229

Solutions of the massless field equations of a given heli-

city m/2 on complexified Minkow~ki space will be constructed as
holomorphic sections of certain "spinor" bundles over M•. For
m ~ 0 let Em denote the vector bundle over M+ whose fiber
over any V 6 M+ is given by H0 • 1 (T- 1 (V),0(-m-2)),. P(m,C 2 )•.
By pulling back the sheaf 0(-m-2) and then "integrating over
the fiber", the Penrose transform induces a map

where, for w 6 H0 • 1 (P;,O(-m-a)) and V e M+, the section '¥

w
of Em is defined by

THEOREM 2..1 (Penrose [9], [10], Woodhouse [13], Eastwood-

Penrose-Wells [3]): Let w e H0 • 1 (P;,O(-m-2)), V eM+, and let
'¥ be defined as above. Then '¥
w
describes a holomorphic sec-
tion of Em over M+. Furthermore, if m ~ 0, '¥ is injective
and 'i'w determines a solution of the massless field equations of
helicity m/2 on complexified Minkowski space.

3. L2 -COHOMOLOGY. In this section we summarize the relevant

facts concerning L2 -cohomology which will be required below. The
references for this section are Blattner and Rawnsley [1] and
Carmona [ 2] •

Let W denote en endowed with a fixed hermitian form h

of signature (p,q), p + q = n. Let cr,s(W) denote the space

of (r,s) forms on W and the (r,s)-forms with

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 230 LISA A. MANTINI

compact support. We will describe differential forms on W

using the standard multi-index notation:

• • • A ctz:-
Js

and so on. A notion of square-integrable cohomology requires an

inner product on differential forms. To this end, we choose a
positive definite hermitian form g on W. We extend g to the
tangent spaces of W, and then to the cotangent spaces, via the
usual identifications. We may then define, for w,T e C~'s(W),

<w,T>g = J g(w(z),,(z))exp(~ h(z,z))dm(z), (3.1)

w
where the measure dm(z) is defined so that

We may now define L~'s(W;g) to be the completion of

with respect to this inner product. Only its Hilbert space
structure depends on g; when we wish to refer to the underlying
t opo 1 og1ca
· 1 vee t or space, we wr it e r,s(W).
L2

Our coboundary operator in cohomology is a. Following

Blattner and Rawnsley [1], we consider the complex

ar 1 0 ar 0 +1
• L 0 'r- 1 (W) ----> L 'r(W) --> L 2 • r (W) • • • .
2 2

We extend a to operate weakly, in the sense of distributions,

so that a becomes a closed, densely-defined operator with the
image of ar-1 contained in the kernel of ar (which is
closed). We may now define

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 231

It is possible to find a space of representatives for

which is given by differential forms. We denote by
the adjoint of a with respect to <·. ·> g • As in the Hodge
approach to cohomology, define the space of g-harmonic
differential forms as

We now have a direct sum decomposition

.If we fix a set of coordinates on W, we can describe H0 •r(W;g)

explicitly. Since g is positive definite, h and g can be
simultaneously diagonalized. Fix a basis of W so that in the
corresponding coordinates the matrix for h is given by

h0 = I = d i ag ( -1 , ,-1,+1, •+1 ) • (3.2)

p,q

the diagonal matrix with q -1's followed by p +1's on

the main diagonal, and the matrix for g is given by
s0 ~ diag(a 1 , ••. where each is real and
positive. Using the standard multi-index notation, if
S ~ (1,2, ••• ,q) 6 Nq and T • (q+1, ..• ,n) 6 NP, then
2 2
so that h(z,z) = -lz 8 1 + lztl ·

THEOREM 3.1 (Carmona [2]): H0 •r(W;g) = {0} unless r = p, and

in that case, H0•P(W;g) = { w e L~·P(W;g) 1 w(z) = <P(z)dzT where
<P satisfies:
i) <P(z) is holomorphic in zs·

ii) <P(z)exp(~ lztl 2 ) is holomorphic in zT,

iii) f I<P<z>l 2 exp( 1/ 2 h(z,z))dm(z) <CD}.
en

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 232 LISA A. MANTINI

Theorem 3.1 shows that ~(z) where

f is holomorphic. The set of such differential forms with
f a polynomial is dense in H0 •P(W;g).

The action of U(p,q) by left translation on L~'P(W;g)

commutes with a, therefore U(p,q) acts naturally on H0 •P(W).
However, the hermitian form h preserved by U(p,q) is not
positive definite. Blattner and Rawnsley [1] show that there is
a canonical choice of inner product on H0 •P(w) such that the
action of U(p,q) is unitary, as follows. Define an "inner
product" <·, ·>h as in (3 .1) , so that for

<w,T>h = J h(w(z),T(z))exp(~ h(z,z))dm(z).

w
Let denote the adjoint of a with respect to and
define a space of "pseudo-harmonic" differential forms as

(3.3)

Biattner and Rawnsley show that <•,•>h is positive semi-defi-

nite on (3.3) and, modulo elements of zero norm, the resulting
Hilbert space is isomorphic to H0 •P(w).
Since U(p,q) does not preserve the positive definite
hermitian form g, left translation by elements of U(p,q) does
not preserve H 0 •P(W;g). If we denote by Pg the orthogonal
projection

P
g
: ker ap + H0 •P(W;g),

then the action of G € U(p,q) on is given by

a(G) = Pg 0 i(G).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN 1 2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 233

LEMMA 3.1: If w 8 ker ap is given by

w(z) = ~(z)dzT + I' ~J(z)dzJ,

JfT
then the orthogonal projection Pg(w) = Pg(~d~ is given by
Pg(w)(z) = dzT f ~(w)K(z,w)dm(w),
en

COROLLARY: Let w 8 H0 •P(W;g) be given by w(z) ~(z)dzT, and

let G- 1 = (~ ~), for G 8 U(p,q). Then

(o(G)w)(z) = det IT dzT J ~(G

-1
w)K(z,w)dm(w).
en

The group U(p,q) does not act irreducibly on H0 •P(W) or

H0 •P(W;g). If we define

(3 .4)

then H0 •P(W;g) decomposes into the direct sum of irreducible

representations

lil H~'P(W;g).
m=-<»
Similarly,
CD

m=-co

The irreducible representations and ar-e the

so-called "ladder-" r-epr-esentations of U(p,q).

4. INTEGRAL TRANSFORM. In this section we describe the construe-

tion of our integral transform and the differential equations
which result. We refer to [5] for the proofs of the assertions
contained here.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 234 LISA A. MANTINI

In the Grassmannian G(p,Cn) of p-planes in W, we let M+

denote the set of all p-planes on which the restriction of our
hermitian form h is positive definite. As defined in the
previous section, we consider the L2 -cohomology spaces
H~'P(W;g) and H~'P(V;hiV), for V 6 M+. Theorem 3.1 and
( 3 .4) imply that w 6 H~'P(V;h IV) if and only if
w(u) = f(u)exp(-lf2 (hiV)(u,u))du where f is a homogeneous
polynomial of degree m. In particular,

0 {P(m,cP), m ~ 0,
H ,p(V;hiV) =
m { 0}' m < 0.

There exists a holomorphic vector bundle Em over M+, for

m ~ 0, which has fiber H~'P(V;hiV) over V 6 M+, and which is
a homogeneous vector bundle for the action of U(p,q). The space
of holomorphic sections of Em, denoted 0 (M+ ,Em), will contain
the image of the integral transform we construct below.

Our integral transform, like the Penrose transform, is a

direct-image mapping from cohomology to sections. The following
lemma allows us to define such a map in this L2 -cohomology
setting.

LEMMA 4.1: For all w 6 H0 • P(W;g) and for all V 6 M+, the
restriction wiV satisfies

Since V is p-dimensional, any (O,p)-form on V is a-closed,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN 1 2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 235

therefore every square-integrable (O,p)-form on V determines

an L2 -cohomology class in u0 •P(V;hiV). Let PV denote the
orthogonal projection

Lemma 4.1 guarantees that PV(wiV) is well-defined for any w 8

u0 •P(W;g). In addition, Pv satisfies the following properties:

LEMMA 4.2: For all and for all V e M+ , Pv<wiV)

. 11 y on
d epen d s h o 1 omorp h 1ca V 8 M+. Fur th ermore, PV preserves
homogeneity degree, so that if. w 8 H~'P(W;g), PV(wiV) 8
u~·P(V;hiV). Finally if m < o, Pv<wiV) .. o.
We may now define our transform

for m ~ 0 by defining

for w e u~·P(W;g), v e M+. Note that ~ "' 0 on H~'P(W;g) if

m < 0, by Lemma 4.2. We next show that ~ is independent of
the choice of g and so is well-defined on u 0 •P(W).
m

PROPOSITION 4.1: For any two choices g and g' of positive

definite hermitian forms on W, the following diagram commutes:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 236 LISA A. MANTINI

Here • denotes the natural injection. We conclude that the map

t is independent of g and intertwines the natural actions of
U(p,q) on H~'P(W;g) and on O(M+,Em).

THEOREM 4.1: There exists a well-defined mapping

for m ~ 0 which is given as follows:

t;;;(V) = the L 2 -cohomology class of wiV in H~'p(V;h\V),

where V 6 M+, w6 H~'P(W), and w 6 H~'P(W;g) is a differen-

tial form which represents the cohomology class w. Furthermore,
t is injective whenever m ~ 0, and t intertwines the natural
actions of U(p,q) on and Thus, t pro-
vides a geometric realization of the ladder representations of·
U(p,q) as sections of the vector bundle E
m
over
The sections in the image of t satisfy certain differen-
tial equations on M+, as in the case of the Penrose correspon-
dence. If p ~ 2 and q ~ 2, then there is a family of second-
order differential operators which annihilate each component of
t , for any w e ~ HmO,p(W;g). The wave operator is of this
w m~O
form, for p = q = 2. If p ~ 2, q ~ 1, and m ~ 1, then there
exist systems of linear differential equations satisfied by t w'
for any w e u 0 • P(W;g).
m
If p z q 2, these are the field
equations of massless particles of helicity m/2 on complexified
Minkowski space. For example, the Dirac-Weyl equations of a
neutrino and Maxwell's equations are of trris form, for m a 1
and m = 2, respectively.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 237

In order to describe the differential equations in more

detail, we require coordinates on M+. Recall that we have fixed
a choice of coordinates on W for which the matrix of h is
given by p,q • It follows that
h0 = I V 6 M+ if and only if V
is spanned by the columns of

A
'l
Ip

where A 6 Mqxp(C) satisfies I (A) = Ip - *U » 0. Thus the

restriction of any function f from w to V = V(A) e M+ is
given by (f 1V)(u) = f(Au,u), and the restriction hiV is given
by (hiV)(u,u) = *ui(A)u. As in Lemma 3.1, the orthogonal pro-
jection PV : L~'P(V;hiV) + H0 •P(V;hiV) is given for
w = <j>du e L~·P(V;hjV) by

(PVw)(u) = du f <j>(v)exp(-lf2 *ui(A)(u-v))det I(A.)dm(v).

cP

Therefore, if w 6 H~'P(W;g), tw(V)(u) = Pv(w!V)(u) is given by

tw(V)(u)- du J <P(A.v,v)exp(-~ *ul(A.)(u-v))det I(A.)dm(v). (4.1)

cP

We will express the differential equations in coordinates

for a trivialization of Em. Recall that Em is a holomorphic
vector bundle which is homogeneous for U(p,q). A trivialization
J : M+ x P(m,cP) + Em which realizes the action of U(p,q) as
being by holomorphic transformations is given by

J(A,f)(u) = deti(A)f(I(I)U)exp(-lf2 *ui(A)u)du. (4. 2)

If we define the polynomial ~A 6 P(M,CP) by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 238 LISA A. MANTINI

~A(s) ~ f $(Av,v)exp(Y~vs)dm(v), (4. 3)

cP
it follows from (4.1) and (4.2) that

~
w
(V(A))

so that the section ~w is given in the trivialization of Em

by the map A~ ~A.

Let alaA denote the q x p ma~rix of differential

operators

a
.
~

If p ~ 2 and q ~ 2, consider the 2 x 2 minor of alaA

formed by the intersection of the ith row, jth column, kth row,
and lth column. We denote by Dijkl its determinant,

which is a nontrivial differential operator whenever i I k and

j I 1.

THEOREM 4.2: Let p ~ 2, q ~ 2, m ~ 0, let w e H~'P(W;g), and

let V = V(A) e M+. Let ~w(V(A)) • J(A,~A), with ~A given as
in (4.3), so that the section ~w is given in the trivialization
of Em by the map A ,.. ~A. Then,

0 ijkl ~A '"' O, (4. 4)

whenever 1 ~ i, k ~ q and 1 ~ j, 1 ~ p.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 239

In order to describe the systems of equations which occur

here, we will use the following notation. If a e Nr and s e cr ,
a a.
• s ar lal = a 1 + •
then we define s = s, 1 r '
a! = a 1 ! • • • ar!, and if y e NP·, we define

We express the vector-valued function wA in components as

for B e NP, where the components w~ are defined so that

SB
(erl·

THEOREM 4.3: Let p a: 2, q a: 1, and m ~ 1. Let w t:: H~'p(W;g),

and let wA be given as in (4.3) so that tw(A) = J(A,WA). If

a,B e NP are such that jaj = IBI = m, then

0, :::; i :::; q, (4.5)

whenever y, ~ e NP satisfy a + Y= B + ~.

Let's specialize these results to the case p • q • 2, so

that

In this case we have only one second-order operator o1122 =

det(alaA), and Theorem 4.2 states that if w e H~•P(W;g) with

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 240 LISA A. MANTINI

m ~ 0 and i f ~
Ill
(V(A )) J(A,~A) with ~A given as in (4.3),
then

(4.6)

Compare this equation with (2.2). If m ~ 1, we similarly wish

to restate Theorem 4.3. Note that ~A e P(m,C 2 ) is given by

Theorem 4.3 then states that if w e n~•P(W;g) with m ~ and

~A is given as in (4.3) so that ~w(A) = J(A,~A), then

i 1 • 2.

where a 1 + a 2 = m- 1. If m = 2, these equations become

a ~ j,2-j a ~ j+1,1-j 0, i ~ 1,2, j =0,1. (4.7)

aA i 1 A aAi 2 A

Compare with (2.7a). Note that although the coordinate systems

and 1..t on are different, the differential equations are
the same. The change of variables which takes into
(c 4 , h 0 )
induces a change of coordinates on M+ under which ~A
. -m For
is mapped into cp1..t = det(]..t+ll 2 ) •m(]..t+il2)~(i]..t+l2 H1..t+il2 )-1.
example, -r 0 (1..t + i i 2) = 1, and

(]..t,,+i)2 2
- 2 ]..t21(]..t11+i) ]..121
-]..t,2(]..t11+i) (]..t22+i)(]..t,,+i) + ]..112]..121 -]..121 (]..t22+i)
2 2
]..112 - 21..t12(]..t22+i) (]..t22+i)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2-COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 241

The differential equations remain unchanged under this change of

coordinates. Note also that this construction produces only the
self-dual solutions of the massless field equations.

5. CONCLUDING REMARKS. The integral transform described above,

although similar in form to the Penrose transform, is not derived
from a double fibration. The author believes that there is a
double fibration somewhere in this framework, yet to be dis-
covered, although the non-sheaf-theoretic nature of the L2 -coho-
mology theory poses some difficulties. It should at least be
possible to find a more invariant formulation of the transform
which does not rely so heavily on the choice of coordinates. We
remark that L2 -cohomology was used by Rawnsley, Schmid, and Wolf
[11] in a much more general setting to unitarize a broad class of
similar representations. Moreover, a transform very similar to
the above, constructed by Patton and Rossi [8], does fit into the
framework of a double fibration. Their work is based on a dif-
ferent model of the oscillator representation which does not use
L2 -cohomology, so the geometric significance of their result is
not the same. However, they obtain a better description of the
differential equations which result.
It should be noted that we obtain above only the positive
helicity, or self-dual, solutions of Maxwell's and the other
massless field equations over complexified Minkowski space.

Note added in proof: A similar construction of the negative

helicity solutions, using L2 -cohomology of nefative q-planes in
W, has recently been completed by the author, see [14].

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 242 LISA A. MANTINI

In the future the author would like to investigate possible

applications of this type of construction to non-linear or non-
homogeneous differential equations of mathematical physics, as
well as other aspects of the problem of finding geometric con-
stuctions of unitary representations. The author would like to
thank R. Bryant for the invitation to present this lecture, and
H. Rossi, M. Vergne, and R. 0. Wells, Jr. for helpful conversa-
tions. She would also like to thank D. J. Wright and J. Haass
for the use of the Sun Workstation at M.I.T., on which this
paper was written.

BIBLIOGRAPHY
1. R. M. Blattner and J. H. Rawnsley, Quantization of the
action of U(k,l) on R2(k+l), J. Funct. Anal. 50 (1983), i1ffi-2fli.
2. J. Carmona, Repr~sentations du groupe de Heisenberg dans
les ~ace~ ..i.e (O,q)-formes.• Math. Ann. 205 (1973), 89-112.
3. M. G. Eastwood, R. Penrose, and R. 0. Wells, Jr., Coho-
mology· and massless fields, Comm. Math. Phys. ~ (1981), 305-351.
4. L. A. Mantini, "An analog of the Penrose correspondence
for representations of U(p,q) on L2-cohomology," Ph.D. disserta-
tion, Harvard University, May 1983.
5. L. A. Mantini, An integral transform in L2-cohomolo~y
for the ladder representations of U(p,g), J. Funct. Anal., 6
(1985)' 211-242.
6. c. Patton, On representations in cohomology ~ pseudo-
hermitian ~metric spaces, Bull. Amer. Math. Soc.. 5 (1981 ),
63-66. -
1. c. Patton, Zero-rest-mass fields and the Bargmann com-
plex structure, in "Complex Manifold Techniques in Theoretical
Physics," Research Notes in Mathematics 32, Pitman, London, 1979,
126-134.
8. c. Patton and H. Rossi, Unitar~ structures on coho-
mology, Trans. Amer. Math. Soc. 261 (19 5), 235-258.--

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 AN L2 -COHOMOLOGY ANALOGUE OF THE PENROSE TRANSFORM 243

9. R. Penrose, Twister algebra, J. Math. Phys. ~ (1967),

345-366.
10. R. Penrose, Twister functions and sheaf cohomolgy, in
"Advances in Twister Theory," Research Notes in Mathematics .TI,
Pitman, London, 1979, 25-36.
11. J. Rawnsley, W. Schmid, and J. A. Wolf, Singular unitary
representations and indefinite harmonic theory, J. Funct. Anal.
2.1 (1983), 1-114.
12. R. 0. Wells, Jr., Complex manifolds and mathematical
physics, Bull. Amer. Math. Soc. 1 new series-rl979), 296-336.
13. N. Woodhouse, Twister cohomology without sheaves, in
"Advances in Twister Theory," Research Notes in Mathematics 37,
Pitman, London, 1979, 37-45. -
14. L. Mantini, An L2 -cohomology construction of negative
helicity massless fLelo equat~ons for U(p,qJ, in preparatLon.
DEPARTMENT OF MATHEMATICS
OKLAHOMA STATE UNIVERSITY
STILLWATER, OKLAHOMA 74078-0613
Current Address:
Mathematisches Institut
Bunsenstrasse 3-5
3400 Gottingen, West Germany

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS ON

COMPLEX HYPERPLANES IN en

ERIC TODD QUINTal

1. INTRODUCTION. Many exciting results have been discovered

about Radon transforms since [Radon, 1917]. The classical trans-
forms have a rich theory (for example [Grinberg], [Guillemin],
[Helgason, 1965], [Hertle]) as well as intriguing applications to
tomography, partial differential equations and group representa-
tions. Gelfand et al. [1966] proved that the classical Radon
transform integrating over complex hyperplanes in en is invert-
ible. This result was cleverly applied to a problem on boundary
values of holomorphic functions by [Stout].
Generalized Radon transforms ~~ transforms integrating over
(perhaps) non-canonical sets in non-canonical rr..easures -- provide
challenging problems. For the most part, the classical theory
does not go over. The incidence relation [Helgason, 1965] and
the double fibration [Gelfand et al., 1969] did much to unify the
theory. Guillemin also has done a lot, by, for example, proving

~esearch partially supported by NSF grant 82-01627.

© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

245
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 246 ERIC TODD QUINTO

that many transforms are elliptic Fourier integral operators

[Guillemin and Sternberg]. Dependence on the measures has been
researched [Quinto, 1980], and other invertibility results have
been obtained (such as [Berenstein, Zalcman], [Cormack, Quinto]
and [Finch, Hertle]) under various assumptions on the sets and
measures. Invertibility can sometimes be proven under invariance
assumptions; in [Quinto, 1983] non-zero smooth O(n)~invariant

transforms on real hyperplanes in Rn were proven to be injec-

tive on domain E'(Rn).
This article proves injectivity for U(n)-invariant trans-
forms integrating over complex hyperplanes in en on domains
L~(Cn) and f'(Cn). This generalizes the classical result,
[Gelfand et al., 1966]. A support theorem is proven for this
transform that is analogous to the theorems for the classical
[Helgason, 1965] and O(n)-invariant [Quinto, 1983] transforms on
Rn. Besides being rotation invariant, the measures on the hyper-
planes are only required to be non-zero at the closest point on
each hyperplane to the origin and to be sufficiently smooth.
Using a new Funk-Hecke theorem (Theorem 4.4) for spherical har-
monies, the proof reduces to solving a type of Volterra integral
equation (Theorem A of Section 3). For distributions, an addi-
tional result [Quinto, 1978] about the Radon transform as a
Fourier integral operator is needed.
Section 2 gives the definitions of the transform (Definition
2.1) and rotation invariance (2.2) as well as the restriction on
the measures imposed by rotation invariance (Proposition 2.2).
Section 3 gives the main injectivity and support theorems

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 247

(Theorems 3.1 and 3.2). Remark 3.4 applies these theorems to prov-
ing a support restriction for projective hyperplanes on CPn.
Finally, in Section 4, the new Funk-Hecke theorem (Theorem 4.4) is
proven. Recent work of Boman and Boman and Quinto show that, with-
out some invariance assumptions, very strong smoothness (analytic-
ity) is needed for injectivity and support theorems to hold.

2. DEFINITIONS AND BASIC RESULTS. Let < , > denote the stan-
dard complex inner product on en, I I the induced norm. Let
x 6 en, and let dx be Lebesgue measure on en. Now let s 2 n- 1
be the unit sphere in en, w 6 s 2n- 1 , and let dw be the
standard measure on the sphere. Let p 6 C with Lebesgue
measure dp on c. The measures used to define function spaces
on these sets will be the canonical ones given above. For
(w,p) 6 8 2n.-1 X c. the set H(w,p) = {x 6 Cnl<x,w> p} is the
complex hyperplane orthogonal to w and containing pw. Note
H(w,p) • H(cw,cp) for each complex number c of norm 1. Let
dxH denote the measure on H(w,p) induced from Lebesgue measure
on en.

DEFINITION 2.1. Let ~(x,w,p) 6 C(Cn x s 2 n- 1 x C), n > 1,

satisfy ~(x,w,p) • ~(x,cw,cp) for each (x,w,p) and
complex number c of norm one. The Radon transform
R : C (Cn) --> C (S 2 n- 1 x C) is defined by
~ c c

R~f(w,p) • J f(x)~(x,w,p)dxH. ( 2. 1 )
x6H(w,p)

In Section 3 the domain of R is extended.

~

The classical Radon transform on complex hyperplanes

[Gelfand et al., 1966] is R~ for ~ • 1. In general is

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 248 ERIC TODD QUINTO

determined by values of ~(x,w,p) for x 6 H(w,p), and, by the

parity assumption on ~ in Definition 2.1, R~f can be viewed
as a function on hyperplanes. This transform is viewed as an
example of the integral transform defined by [Gelfand et al.,
1969].
The transform R~ is rotation invariant iff, for each rota-
tion u € U(n), and each f e Cc(Cn),

R~(fou)(w,p) = R~f(uw,p). (2.2)

Simply put, the integral of fou over H(w,p) in its measure is

the same as the integral of t over the rotated hyperplane,
uH(w,p) in its measure.

PROPOSITION 2.2. Under the assumptions of Definition 2.1, R

~

is rotation invariant if and only if there is a function

U 6 C(R 2 ) such that U(r,s) = U(-r,s) = U(r,-s) and for all
(w,p) and all x 6 H(w,p), ~(x,w,p) = U(lx-pwl ,lpl). If ~ is
ck then u is ck.

PROOF. Clearly, if ~ is given in terms of U as in the

proposition, then R is rotation invariant.
~

Assume R is rotation invariant. Rotation invariance of

~

the measure dxH and (2.2) imply

R~f(uw,p) - J f(ux)~(ux,uw,p)dxH - J f(ux)~(x,w,p)dxH

H(w,p) H(w,p)

for all f 6 Cc(Cn) and all (w,p). Thus ~(x,w,p) • ~(ux,uw,p)

for all X 6 H(w,p). Evaluating this identity for all U that
fix w proves there is a function u1 (s,w,p) for s e R such

that ~(x,w,p) = u1 ( lx-pwl,w,p) for all (w,p) and x e H(w,p).

A similar argument, now using all u e U(n) as well as the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 249

parity assumption in Definition 2.1 proves the existence of a

continuous function U(s,t) as in the proposition. Finally, by
the definition of u, if ~ e ck then ue ck.

3. THE MAIN THEOREMS. Our main theorems on the invertibility of

RP are now presented. Let L2 (Cn) be the set of L2 functions
c
on en of compact support.

THEOREM 3.1. Let the Radon transform R

p
satisfy Definition 2.1
and let M > 0.

( i) R
p
Assume in addition that RP is rotation invariant,
p e c 2 n~ 2 and for each (w,p), p(pw,w,p) ~ 0.

(ii) If
c
f e
L2 (cn) and K > o such that Rpf(w,p) - 0 on
{(w,p)l IPI > K} then f(x) - 0 on { x e en 1 1xI > K}.

Recall that a distribution v is zero on an open set S if

and only if v(f) = 0 for all C~ functions f supported in S.

THEOREM 3.2. Let the Radon transform Rp satisfy Definition 2.1

and let p e CCD.

E '(Cn) -> E' (s 2 n- 1 x C) is continuous.

Assume in addition that RP is rotation invariant and for
each (w , p) , p ( p.u ,w , p) ~ 0.

(11) If v e E'(Cn) and K ~ 0 such that Rp v is zero on

{ ( , p) I I pI > K} then v is zero on {x 6 Cnl lxl > K}.

E'(Cn) -> E'(S 2 n- 1 x C) is injective.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 250 ERIC TODD QUINTO

Thus the rotation invariant transform R is invertible under

~

the rather mild assumptions that ~(x,w,p) is sufficiently smooth

and non-zero at x pw, the closest point on H(w,p) to the
origin. Moreover, a support theorem (Theorems 3.1 and 3.2 (ii))
is true that generalizes the support theorem for the classical
[Helgason, 1965] and rotation invariant [Quinto, 1983] Radontrans-
forms on real n-1 dimensional hyperplanes in Rn. An indepen-
dent proof for the classical transform on en is outlined in
REMARK 3.3. It is straight forward to show, as in [Quinto,
1982], that the support theorem does not hold, even for the
classical transform, on domain L1 (en).
Before proving the theorems, we present some of the theory
of spherical harmonics on S2n-1 • A spherical harmonic on 3 2n-1
is the restriction to s 2 n- 1 of a harmonic polynomial on
R2 n = en. The space of harmonics that are restrictions of poly-
nomials homogeneous of degree m is denoted Hm(s 2 n- 1 ). A
spherical harmonic Y(w) is homogeneous of bi-degree (k,1) if
it is the restriction to s 2 n- 1 of a harmonic polynomial Y(z)
homogeneous of degree k + 1 and such that

Y(cz) = ck- 1Y(z) for all z 6 en and c 6 e, lei = 1. (3.1)

Let H (S2n-1) be the vector space of spherical harmonics

k,1
homogeneous of bi-degree (k,1). Then L2( 3 2n-1) is the orth-
agonal direct sum of the Hk,1 (S2n.,..1). The proof is immediate
since H (S2n.,.1) is the orthogonal direct sum of H (S2n-1)
m k,m-k
for k = 0. 1 • ... ,m, (3.1), and since L2( 3 2n-1) is the ortho-
gonal direct sum of H (S2n-1) for m 0,1,2, •.. [Seeley].
m . a

Important polynomials, Pk, 1 (z) for z 6 e, are given in

2n-1
Proposition 4.2 (4.1) such that, forfixed e 6 S , Pk,i(<w,e>),

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 251

is the unique spherical function (Definition 4.1) in

Hk,JI.(s2n.,.1).
The final·additional tool is a theorem about Volterra
integral equations.

THEOREM A. Let K e R and E(r,p) e Cn~1 ([K,ao)2)' E(r,r) ~ o.

for all r '= K. Let He cn- 1 ([K, .. )) satisfy
H(K) "' H' (K) = ... = H(n-1)(K) = o. Then the generalized
Volterra equation of the first kind

H(r) = J
rr E(r,p)(r-p)n- 2 G(p)dp
K

has a unique solution G 6 C([K, .. )) and G(K) • 0. If H 6 c""

and D 6 c"" then so is G. If, in addition, H is zero to
infinite order at K, then so is G.
The proof of Theorem A follows immediately from standard
theorems about Volterra integral equations (e.g., [Yosida, p. 154,
pp. 145-146]) and is outlined in [Quinto, 1983].

PROOF OF THEOREM 3.1. The continuity result, Theorem 3.2 (i), is

proven using Holder's inequality just as for the classical Radon
transform on Rn (e.g., [Smith, Solmon and Wagner]). For
g e L2loc (S 2 n"" 1 X C) define the dual Radon transform by

R~g(x) "' Jr g(w,<x,w>) ~(x,w,<x,w>) d (3. 2)

wes2n-1

Holder's inequality proves that

is continuous.
Assume R is rotation invariant, ~ 6 c 2 n- 2
~

and ~(pw,w,p) ~ 0 for all (w,p). Proposition 2.2 proves

~(x,w,p) - U( lx-pwl' IPI) for a function u 6 c 2n- 2 (R 2 ).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 252 ERIC TODD QUINTO

Now choose K > 0, integers k,t ~ 0 and

The support theorem, (ii), is proved by showing
surjective from functions of the form

g(w,p) = gk, 1 (p)Yk,t(w) where gk, 1 6 C(C)

k~1
and for c 6 C, lei .. 1, gk, 1 (cp) c gk, 1 (p) (3.3)
and gk, 1 (p) • 0 for IPI :ll K.

to functions

h(x) • hk,t<lxi)Yk, 1 (x/lxl) where

(3.4)
hk, 1 (t) = 0 for t :ll K and hk, 1 e cn~ 1 ([0,~)).

For g of the form (3.3), x 6 en, r = lxl > 0 and

e = x/lxl

R*g(x)
ll
I gk, 1 ( r<e ,w >) Yk, 1 ( w)
wes2n-l (3. 5a)
2 1/2
• U(r(1-l<e,w>l ) ,rl<e,w>l)d

since R is rotation invariant. Theorem 4.4 can now be used to

ll
show R*g is equal to Yk, 1 {e)Vol(S 2 n- 3 ) times an integral on
. u . Pk,t•
ll
the unit ball in c of gk,1 Using the complex
homogeneity assumption on gk, 1 and the explicit expression for
Pk, 1 ' (4.1), as well as a change of variables, one proves

R~g(x) = Yk, 1 (e)Vol(S 2 n~3) 2n

• Jr Pk-1+1 r-[2n~2+k-1](r+p)n-2Q (k~1,n-2,(p/r)2) (3.5b)

K 1
• U((r2-p2)1/2,p)(r-p)n-2gk-1(p)dp

for k ~ 1. If k < 1, k and 1 are switched in the kernel of

the integral equation in (3.5b). The integral can be taken from
K to r as g and h are zero for IPI :ll K.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTNITY OF ROTATION INVARIANT RADON TRANSFORMS 253

Let h be of form (3.4). h(j)(K) = 0 for Then

k,R.
j - 0, ... • n..-1 • By the smoothness assumption on U, since
Q1 (a,b,1) - 1• and since u(pw,w,p) - U(O,IPI) ,. 0, Theorem A
applied to equation (3 .5b) provides a function g(w,p) satisfy-
ing (3.3) for the same harmonic Yk,R. such that

(3. 6)

(gk,R.(p) is extended continuously from domain [K,m) to domain

C using the parity condition in (3.3) and the fact gk,R.(K) s 0).
This proves surjectivity.
Let f 6 L~(Cn) such that R~f(w,p) = 0 for IPI > K and
let h(x) satisfy (3.4). Then using g from (3.6), duality, and
(3.3), onesees Jf(x)h(x)dx = 0. Since Yk,R. was arbitrary, f
is orthogonal to all finite sums of functions of the form (3.4).
As such sums are dense in Li 0 c({x 6 Cnl lxl ~ K}), f(x) = 0 for
IX I > K. his proves (ii) for K > 0. The case (iii), K = O,
is now immediate.

PROOF OF THEOREM 3.2. Assume R satisfies Definition 2.1 and

~

u 6 em. Using the definitions of R* (3. 2) and the topologies

l.l
[Rudin], it can be shown that R* : Cm(s2n-1 X C) -> Cm(Cn) is
~

continuous. Therefore, (i) is immediate.

The proof of (ii) is similar to the proof of Theorem 3.1
(ii) and will only be outlined. Let K > 0. Using Theorem A for
smooth h satisfying (3. 4) supported in lx I > K, one can solve
(3. 6) for smooth g satisfying (3.3) supported in IPI > K. Let
v e E' (en) and assume R
l.l
v is zero for IP I > K. Then, as in
the argument be 1 ow (3 • 6 ) , v is orthogonal to smooth finite sums
of functions (3.4). Since such sums are dense in the set of em
functions supported in {x 6 Cnl lxl > K}, v =0 for lxl > K.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 254 ERIC TODD QUINTO

This proves (ii) for K > 0. The proof for K 0 is now

immediate.
The theory of pseudodifferential operators [Treves] is now
used to prove (iii). As shown in [Quinto, 1978], R*R
]J ]J is a
pseudodifferential operator with symbol

2
o(R*R )(x,y) = l..l (x,y/ly4,<x,y/lyl>)
l..l l..l (21r) IYI2n.,..2

for X e en, and y e cn .. {O} • Since ]J(O,w,O) ~ 0 for all

we s 2 n- 1 , and l..l is C~, this symbol is a non-zero multiple of
1/lyl 2 n- 2 in a set {(x,y)l lxl <a, y € cn-{0}} for some a> o.
Thus R*R
]J ]J is an elliptic pseudodifferential operator in a
neighborhood of zero [Treves, p. 40]. Therefore, the null space
of R*R
]J ]J when restricted to distributions supported in {x € Cnl

lxl < a} consists entirely of smooth functions [Treves, p. 42].

Assume v e EI and R]J v = o. Then (ii) proves v is
supported at the origin. Since the above argument shows v is
smooth, v = 0. This finishes the proof of the theorem.

REMARK 3.3. The support theorem for the classical complex Radon
transform follows from the support theorem for the classical real
transform. The key is the observation that the real 2n-1 di.,..
mensional hyperplane perpendicular to w and p 6 R units from
the origin is the union of the H(w,p + ir) for r e R. By a
nice result of [Hertle], the classical real dual Radon transform
is surjective from C~(Sn- 1 x R) to C~(Rn). The proofs of
Theorems 3.1 and 3.2 (ii) .suggest R~ might also be surjective.
This is an open problem in general. However Hertle's result
implies surjectivity for the classical complex dual Radon trans-
form on range C~(Cn) (even from smooth g satisfying

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 255

g(w,p) = g( cw, cp)). The key to this reduction is that the

classical real dual Radon transform evaluated on g(w,s) for
(w,s) 6 3 2n.-1 R is the complex transform of g(w,re(p)) for
X

(w,p) 6 3 2w·1 c. An averaging trick finishes the proof.

X

REMARK 3.4. These theorems imply invertibility and support theo-

rems for Radon transforms on other spaces. One example occurs on
CPn. For w 6 CPn let H[w] be the projective hyperplane
orthogonal to w. Each hyperplane inherits a measure d~H from
the standard measure on CPn. Now, for v 6 C(CPn,cpn), the
Radon transform
v is defined
R

Rvf[w] • J f(~)v(~,w)d~H
H[w]

Let e e CPn and let U be the isotropy group of e in

U(n+1). Rv is U-invariant iff for each u 9 U, w S CPn and
f 6 C(CPn)

The classical Radon transform (v- 1) is U-invariant for any e

and was inverted in [Helgason, 1965]. A corollary to Theorem 3.2
is

PROPOSITION 3.5. Let e 6 CPn and let U be the isotropy

subgroup of e in U(n+1). Let p e c"'(cPn,cPn) be nonzero
and such that Rp is U·dnvariant. Let f e V'(CPn) be support-
ed away from H[e] and assume b > o. If Rp f[w] "' 0 for
d(w,e) < b, then f is zero on· Ud(w,e)<bH[w]. Here d is the
metric on CPn.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 256 ERIC TODD QUINTO

Simply put, under these hypotheses, if R).I f is zero on

projective hyperplanes "within b of H[e]", then f is zero
there.

PROOF. The proof consists of projecting the transform to a

smooth rotation invariant transform on affine hyperplanes in en.
For X S en+l_{O} let [x] be the complex line through x.
By abuse of notation, H[x] will denote the projective hyper-
plane orthogonal to [x]. Give en+l coordinates so
e • [(0,0, ••. ,1)]. Let s : en ->CPn-H[e] be the diffeo-
morphism sex,, ,xn) = [(x 1 , ••• ,xn,1)]. If u 6 U(n) then
u o s =sou with the obvious identification of U(n) and the
isotropy subgroup of (0, ••• , 1 ). If ( w,p) e s 2n-l " c c cn+1
and Ha(w,p) is the affine hyperplane then s(Ha(w ,p))
= H[(w,-p)]. This implies that the map

eccen) 9 f -> RJ.I(f o s-1 )[(w,-p)] is a rotation invariant

Radon transform in en with non-zero smooth measure. For each

b > 0 there is a K(b) > 0 such that complex hyperplanes
Ha(w,p) for IPI > K (b) correspond bijectively to projective
hyperplanes H[t] for 0 < d(t,e) < b. Theorem 3.2 (ii) can now
be used to finish this proof.
An analogous projection argument using [Quinto, 1983] can be
done to show non~zero smooth rotation invariant transforms on
real n - 1 dimensional totally geodesic submanifolds in hyper-
bolic space are invertible and satisfy a support theorem on
domain E'.

-· A FUNK~HECKE THEOREM FOR SPHERICAL HARMONICS AND U(n). The

goal of this section is to prove a Funk~Hecke theorem for

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 257

spherical harmonics in H (S2n-1) (see (3.1)).

k,i.
Let e e s2n..-1 , let U(n) be the unitary group acting on
8 2n.,..1 and let U( n..-1 ) be the isotropy subgroup of e.

DEFINITION 4.1. A function P 6 C(S 2 n.,.. 1 ) is a spherical func-

k,i. (S n- ) , P is
tion iff P is a spherical harmonic in some H 2 1

U(n.,..1) invariant and P(e) • 1.

This definition [Helgason, 1962] depends on the choice of e.
Define {Qi.(a,b,t)li. = 0,1,2, ... } to be the complete set
of polynomials in t (Qi. has degree 2. ) orthogonal on [0, 1]
in weight ta(1..-t)bdt and satisfying Q2.(a,b,1)=1, a> ..-1, b > -1.

PROPOSITION 4.2. [Johnson and Wallach] Let k ~ 0 and 2. ~ 0.

then contains a unique spherical function given by

Pk, 1 (<w,e>) where

if k ~ 2. then Pk, 1 (re

19
) = (rei 6 )k- 1 Q2.(k-1,n..-2,r 2 ) (4.1a)
if 1 > k then Pk,1=PJI,,k (4.1b)
for all k,i. and z 6 C, Pk,1(z) = fik,1<z) (4.1c)

The set {Pk, 1 } is a complete orthogonal set of functions in

L2 (B) where B is the unit ball in C.
This is essentially [Johnson and Wallach Theorem 3.1 (3)].
Spherical functions are given there on en in terms of hypergeo-
metric functions, however, they are easily expressable in terms
of orthogonal polynomials as in (4.1) (using their general form
and orthogonality relations or [(16), p. 170, Erdelyi] and
Johnson and Wallach's result). The spherical functions
Pk, 1 (<w,e>) in Proposition 4.2 are given on s 2n- 1 and would be
extended to en homogeneously of degree k + 1. The existence
of a unique spherical function in each Hk,i. is guaranteed since

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 258 ERIC TODD QUINTO

U(n) is multiplicity free (see [Guillemin] and [Grinberg] for

nice applications).

PROPOSITION 4.3. Let Yk,~ 6 Hk,~(S

2n-1
) and let e e g2n-1

with isotropy subgroup U(n-1). Then

JYk,~(uw)du = Yk,~(e)Px.~(<w,e>). (4.2)

U(n-1)

PROOF. The function Q(w) = JYk,JI,(uw)du is clearly a U(n-1)

uEU(n-1)
2n~1
invariant function in Hk,~(S ). By Proposition 4.2, Q(w)
cPk,~(<w,e>) for some c. Furthermore Q(e) = Yk,~(e) ~ c.
This proves (4.2)
Finally we prove

THEOREM 4.4, THE FUNK~HECKE THEOREM.

Let g(z) e L 2 (B,(1~Izl 2 )n- 2 dz) where B is the unit ball in

C. Let Yk,~ 6 Hk,~(S 2 n~ 1 ) and e 6 s 2n- 1 • Then

J g(<e,w>)Yk,~(w)dw
3 2n-1
(4.3)
= Yk ~(e)Vol(S 2 n-3) J g(z)Pk ~(z)(1-lzl 2 )n- 2 dz.
' lzl < 1 '

This theorem allows one to simplify many integrals (such as

(3.5a) !) involving._spherical harmonics. The integral over s 2 n.,.. 1
of g(<w,e>)Yk,~(w) can be calculated directly from (4.3) using
(4.1c).

PROOF. Let u € U(n-1). First use group invariance of dW to

replace w by uw in the top integral in (4.3). Integrate over
U(n-1) and switch integrals. Now Proposition 4.2 proves the top
integral is equal to

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INJECTIVITY OF ROTATION INVARIANT RADON TRANSFORMS 259

(4.4)

an integral depending only on <e,w>. Write (4.4) as an integral

over <e,w> = z 6 B. Equation (4.1c) finishes the proof.

ACKNOWLEDGEMENTS

I would like to thank Eric Grinberg for several enlightening

discussions about his thesis as well as this research. I also
thank Ken Johnson for referring me to [Johnson and Wallach] as
well as Carlos Berenstein and Larry Zalcman for inspiring discus-
sions. Finally I thank the Mathematical Sciences Department of
Memphis State University for providing a nice home for me as I
finished this article.

BIBLIOGRAPHY

1. C. A. Berenstein, and L. Zalcman, "Pompeiu's problem on

symmetric spaces," Comment. Math. Helvetici 55 (1980), 593.,..621.
2. A. M. Cormack and E. T. Quinto, "A Radon transform on spheres
through the origin in Rn and applications to the Darboux
equation," Trans. Amer. Math. Soc. 260 (1980), 575-581.
3. A. Erdelyi, w. Magnus, R. Oberhettinger, and F. Tricomi,
Higher Transcendental Functions, Vol. 2, Krieger, Malabar, FL
(1981).
4. D. V. Finch and A. Hertle, "The exponential Radon transform,"
these proceedings.
5. I. M. Gelfand, M. I. Graev, and Z. Ya Shapiro, "Differential
forms and integral geom~try," Functional Anal. Appl. l (1969),
101.-114.
6. I. M. Gelfand, M. I. Graev, and N. Ya. Vilenkin, Generalized
Functions, Vol. 5, Academic Press, New York (1966).
7. E. L. Grinberg, "Spherical harmonics and integral geometry on
projective spaces," Trans. Amer. Math. Soc. 279 (1983), 187.-203.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 260 ERIC TODD QUINTO

8. V. Guillemin, "The Radon transform on Zoll surfaces," Advan.

Math. 22 (1976), 85~119.
9. V. Guillemin and S. Sternberg, Geometric Asymptotics, Math.
Surveys No. 14, Amer. Math. Soc., Providence, R. I. (1977).
10. s. Helgason, Differential Geometry and Symmetric Spaces,
Academic Press, New York (1962).
11. s. Helgason, "The Radon transform on Euclidean spaces,
compact two~point homogeneous spaces, and Grassman manifolds,"
Acta Math. 111 (1965), 153-180.
12. A. Hertle, "On the range of the Radon transform and its
dual," Math. Ann. 267 (1984), 91.,..99.
13. K. D. Johnson and N. R. Wallach, "Composition series and
intertwining operators for- the spherical principal series.
I," Trans. Amer. Math. Soc. 229 (1977), 137-173.
14. E. T. Quinto, On the Locality and Invertibility of Radon
Transforms, PhD Thesis, M. I. T. Cambridge, MA (1978).
15. E. T. Quinto, "The dependence of the generalized Radon
transform on defining measuresi" Trans. Amer. Math. Soc. 257
(1980), 331.,..346. -
16. E. T. Quinto, "Null spaces and ranges for the classical and
spherical Radon transforms," J. Math. Anal. Appl. 90 (1982),
408-420.
17. E. T. Quinto, "The invertibility of rotation invariant Radon
transforms," J. Math. Anal. Appl. 91 (1983), 510-522. Erratum,
ibid. 2i (1983), 602~603. --
18. J. Radon, "Ueber die Bestimmung von Funktionen durch ihre
Integr-alwerte 1aengs gewisser Mannigfalligkei ten," Ber. Verh.
Saechs. Akad. ~ (1917), 262-277.
19. W. Rudin, Functional Analysis, McGraw Hill, New York (1973).
20. R. Seeley, "Spherical harmonics," Amer. Math. Monthly 73
(1966), 115-121.
21. K. T. Smith, D. c. Solman, and s. L. Wagner, "Practical and
mathematical aspects of the problem of resonstructing objects
from radiographs," Bull. Amer. Math. Soc. 83 (1977), 1227-1270.
22. E. L. Stout, "The boundary values of holomorphic functions of
several complex variables," Duke Math. J. ~ (1977), 105-108.
23. F. Treves, Introduction to Pseudodifferential and Fourier
Integral Operators, Vol. 1, Plenum Press, New York-c19BO).
24. K. Yosida, Lectures on Differential and Integral Equations,
Interscience, New York, 1960.
DEPARTMENT OF MATHEMATICS
TUFTS UNIVERSITY
MEDFORD, MASSACHUSETTS 02155

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

ON OVERDETERMINED SYSTEMS ASSOCIATED WITH INTEGRAL GEOMETRY

TRANSFORMS IN THE REAL PROJECTIVE SPACE

Radu ROfilU

Starting with [1], [8], in a sequence of papers was con-

sidered the problem of describing the range of various transforms
from Integral Geometry by differential or pseudodifferential
operators, see for instance, [3], [4], [7].
In [3] was studied the transform which associated with
functions on RPn their integrals on p-dimensional planes of
RPn. The range of this transform is characterized by the over-
determined system

a2 <P{u) a2 cp(u) ;$ i, i' ;$ n+1

= 0 (1)
i i' i' i k, k' p+1
auk auk, auk auk, ;$ ;$

Rn+1.
where , up+ 1) is a (p+1)-frame for
( u1 ,
Since dim Gp+1 (Rn+1) > dim RPn for ;$ P < n the func-
tions from the range are uniquely determined by certain n-dimen-
sional submanifolds of G which in [6] were named "permissible
p+ 1
complexes."
In this note we give a theorem about the existence and
unicity for the solution of the system (1) with the condition

(2)

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

261
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 262 RADU RO!SU

for K-permissible complexes. The theorem is a direct consequence

of inversion formulas obtained in [5] and [6].
The author expresses his deep gratitude to his teacher I. M.
Gelfand and to M. I. Graev.
Let GP(V) be the manifold of the p-dimensional subspaces
of the vector space V, p < dim V • n. We denote by G (V)
p
the
line bundle over GP(V) of the pair b = (b,~) where b e GP(V)
and ~ is a volume element in b, and introduce the spaces
F~(V) ((p = 1, ... ,n-1), >.. e C) of smooth functions on GP(V)
which satisfy the homogeneity condition

f(b,t~) = t>..f(b,~) for any t > 0 . (3)

The space can be regarded as the space of smooth

functions on the manifod Ep,n of the p-frames in V, which
satisfy the condition

f(gx) = I det gl->..f(x) for any g e GL(p,R) (4)

or as the space of smooth functions on the manifold p,n-p of M

px(n-p) matrices, which satisfy natural conditions of decreasing

at infinity.
Also we denote by F~(K), for K a submanifold of GP,
1r- 1 (K)
the space of smooth functions on c E
p,n which satisfy
the homogeneity condition (4), 1r Ep,n ~ GP being the canoni-
cal projection.
Let p, 1 ~ p ~ n-1, be fixed from now on. In [5] and [6]
was constructed the transform

1
We give the formula for I in coordinates, when and. Fp+ 1

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 ON OVERDETERMINED SYSTEMS 263

are interpreted as function spaces on the manifold on n-vectors

and on the manifold of (p + 1) x (n - p) matrices respectively.
We write the vectors u as u = (u1 ,u 2 ) with a p-vector
and the matrices v as v = (:~) where v is a px(n-p) matrix.
Then

(5)

From now on we consider that there are given w and w'

volume elements respect! vely on Rn+ 1 and on (Rn+ 1 )', such
that w • w' = 1.

For functions we define a partial Fourier trans-

form as follows: Let be linearly independent
vectors in Rn+ 1 , 1 be the subspace generated by them and a be
the volume element in 1 such that a(x 1 , ••• ,xp) = 1. Then,
for let

;(1,n)- (2~)~ <n-p+ 1 ) f e -i<n,x>.c

"' x 1 , . . . , xp , x ) v·c-)
x
Rn+l/JL
n+l
where n e ann 1 = (R /1)', x is an arbitrary vector from the
preimage in Rn+l of x e Rn+l/1 and v(x) • wla is the volume
n+l
element in R /1 induced by w and a.
The integral is regarded as a Fourier transform of
homogeneous distributions, see [1].
In [5] was proved the:

LEMMA. Let be f e F p+ 1 and <P = If. Then for any pair (JL,n)
1

where 1 e G (Rn+ 1 ) and n e ann 1\{0}

p

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 264 RADURO~U

where
-f(n) denote the Fourier transform.
This lemma makes sense of the following:

1
DEFINITION. The function f 6 Fp+ 1 (K), for K a submanifold in
Gp+1 (Rn+ 1 ) satisfies the Cavallieri condition if

-
<P(R. 1 ,n) = <f>(R. 2 ,n)

for a. e. n 6 ( Rn +1 ) , \ { 0 } and R.. 6 Gp ( Rn+1) ' R. i c ann n such

-<f>(R.i,n)
l

that is defined, i 1 • 2.
Let now G (Rn+1) be a p-dimensional submanifold.
c c By
p
the Crofton symbol of C we shall mean as in [5] and [6] the
function Crc(h) on (RPn)' which assigns to h the number of
subspaces c 6 C which are orthogonal to h. C is called non-
singular if supp Crc = (RPn)' and harmonic if Crc(h) = const ~ 0
(we denote this constant by Cr(C)).
We associate with each harmonic submanifold C c G (Rn+1)
p
the "harmonic complex" G (Rn+1) consisting of the
c
p+1
subspaces a 6 Gp+ 1 which contain at least one subspace c 6 C.
Obviously dim Kc = n.
Now we can give

THEOREM 1. There exists a unique solution of the

problem (1) and (2) for any which s~tisfies the
Cavallieri condition, being an harmonic complex.
SKETCH OF THE PROOF. 1
Existence. Let <Po 6 Fp+ 1 <Kc) with

-<Po(R.1 ,n) -
• <Po(R.2,n)

for a. e. n 6 (Rn+ 1 )'\{0} and R.i 6 C, i = 1,2. Then the

function <Pc(n) = <Po(R.,n) is well defined and

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 ON OVERDETERMINED SYSTEMS 265

Therefore the inverse Fourier transform for ~

( 21T ) _1/2 ( n+ 1) J e i <n , x >; c ( n ) w ' ( n )

( Rn+ 1) ,

is in the space F p+1 By direct calculation we deduce that the

1 •
function ~a If 0 satisfies the system (1) and from the lemma it
is clear that ~/K = ~0 .
c
2. Unicity. Let be a solution for (1) and (2).
From the Paley-Wiener type theorem proved in [3] we obtain that
there exists f 6 Ff+ 1 such that ~ = If. In [6] the inversion
formulas are using the value of ~ on Kc only. Namely let E
be the bundle over with fiber in
-
b 6 G1' 1T-1(b) Gb
G1 p
consisting of those subs paces c 6 Gp such that
C('lb ~
{ 0} •

Then, if C is a harmonic submanifold such that

Cc1T- 1 (b)

f(b) Cx~)(·,b)
-
where the operator X is defined on with value p-density
on E. Therefore the condition (2) uniquely determines the
solution of (1) and (2).
Theorem 1 can be naturally extended for the "permissible
complexes" of Gp+ 1 whose definition from [6] we recall

DEFINITION. The submanifold K c Gp+1 will be called a per-

missible complex if for a. e. b 6 Gp+1 there is a harmonic
submanifold cb c Gb
such that if cplk 0 then
p
2

Cx~H·,b)l
1
for any cp 6 Fp+
- 0 1.
cb
With an analogous proof we obtain:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 266 RADURO~U

THEOREM 1'. There exist a unique solution of the

problems (1) and (2) for any which satisfies the
Cavallieri condition, K being a permissible complex.
We point out that in [6] a large class of permissible com-
plexes was constructed and that using the formulas from [5] we
can obtain explicit formulas for the solution of (1) and (2).

BIBLIOGRAPHY
1. I. M. Gelfand, M. I. Graev, and M. Ia. Vilenkin, Generalized
functions, vol. ~. Academic Press, 1968.
2. I. M. Gelfand, and G. E. Sh-ilov, Generalized functions, vol.
!• Academic Press, 1968.
3. I. M. Gelfand, s. G. Gindikin, M. I. Graev, "Actual problems
in mathematics" VINTI 16 (Russian) 1980, 53-226.
4. I. M. Gelfand, M. I. Graev, Z. Ia. Sapiro, Funktional Anal.
i. Prilozen ~ 1967, 15-31.
5. I. M. Gelfand, M. I. Graev, and R. Ro~u, Operator algebras
and~ representation, Vol. !• Pitman, 1983, 192-207.
6. I. M. Gelfand, M. I. Graev, and R. Ro~u, J. Operator theory
12 1984, 359-383.
7. V. Guillemin, S. Sternberg., Amer. J. Math. 1979, 915-955.
8. F. John, Duke Math. J. i 1938, 300-322.
INSTITUTUL DE MATEMATICA
STR. ACADEMIEI 14
BUCURESTI 1, ROMANIA

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

THE POMPEIU PROBLEM IN EXTERIOR DOMAINS

IN SYMMETRIC SPACES

M. Shahshahani and Alladi Sitaram

1. INTRODUCTION. Let G be a non-compact Lie group with finite

center, K c G a fixed maximal compact subgroup and X • G/K
the corresponding symmetric space of the non-compact type. We
endow X with the G-invariant Riemannian structure induced from
the Killing form and let d(•,•) be the corresponding distance
function. Let dx denote the Riemannian volume element on
X and Br the "geodesic" ball of radius r centered at
0 eK e X i.e. Br {x S X, d(x,O) =:; r}. It was shown in

[9] that i f 0 c X is a set of positive finite measure and

f e L1 (X), then

J f(x)dx - 0 for all g e G

gQ

implies f = 0 a. e.. (On the other hand, it is known (see e.g.

[1], [2], or [9]) that without the integrability assumption on
f, the above conclusion is not justified.) In -this paper we
consider the "support problem" for integrable functions. More
precisely, we show:

THEOREM 1. Assume rank (X)= 1 and f e L1 (X) be such that:

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

267
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 268 M. SHAHSHAHANI and ALLADI SITARAM

(1)

where Br(y) is the ball of radius r and center y. Then

Supp f c BR+r"

We say f 6 Ll 0 c(X) is "very rapidly decreasing" if

ess sup ekd(o,x) lf(x) I < "'• Vk ~ 0.

x6X

Using standard facts about Haar measurements (see [5]) one can
show that such an f is in LP(x),Vp ~ 0.
Then we have

THEOREM 2. Let X be any symmetric space of the noncompact type

and f a "very rapidly decreasing" function on X. If (1)
holds, then Supp f c- BR +r •
Slight modification of our proof shows that Theorem 1 holds
for f 6 LP(x), 1 ~ p ~ 2. However we show in 4. that it fails
for all p > 2.
Our results are analogues of the support theorems of
Helgason [6] for the X-ray and Radon transforms. It is interest-
ing to note that in Theorem 1 it is not necessary to require any
'
further decay conditions at infinity on f, while in [6] strong
decay conditions are indeed necessary. For X = Rn the analogue
of Theorem 1 is proved in [10].
The class of problems considered in this paper have come to
be known in the literature as Pompeiu's problem - see for e.g.
[11]. Finally for any unexplained notation or terminology in
this paper, we refer the reader to [1] or [5].

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE POMPEIU PROBLEM IN EXTERIOR DOMAINS 269

2. THE MAIN LEMMA. Consider the Iwasawa decomposition G ~ KAN

and let M be the centralizer of A in K. We denote the Lie
algebra of A by a and let a*(a*)
c denote its dual
(complexified dual). For A e a~ we let ~A denote the repre-
sentation of G on given by:

(~A (x)v)(b)

where we have used the Iwasawa decomposition

y = k(y)•exp H(y)•n(y) e KAN, y e G

and p denotes (as usual) half of the sum of the positive re-
stricted roots. For the purpose of integration it is convenient
to identify functions on X with right K-invariant functions on
G. It is not difficult to show that for f e L1 (X) and A e a*,
the "Fourier-transform"

is the bounded linear operator on L2 (KIM) mapping v e L2 (KIM)

to the function

b + c( J v(b )db )•J e(iA-p)H(g-lb)f(g)dg

KIM 1 1 G

where c is a constant depending only on the normalization of

measures (see e.g. [1]). Therefore we may identify ~A(f) with
the function f on a* x KIM given by:

Note that if f is K-biinvariant then f(A,b) is independent of

b e KIM and will be simply denoted by
-f(A) (- in this case f

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 270 M. SHAHSHAHANI and ALLADI SITARAM

coincides with the so called spherical Fourier transform of f).

Clearly if f is also compactly supported then f(A,b) is
defined for all A e a~ and is entire as a function of A for
b 6 K/M.

If rank (X)= 1, we have dim a* and so we can

identify a* with R and a~ with C With this identification
we have:

THE MAIN LEMMA. Assume rank (X) = 1. Let 1 denote the

r
indicator function of Br. Then 1r has real and only real
zeroes.

REMARK. The proof of the main lemma is essentially contained in

the proofs of Proposition 2 and 3 in [1], although it is not
explicitly stated there. For this reason we only sketch the
proof. It is tempting to try and prove this fact by case by case
.evaluation of the Fourier transform of 1r using special func-
tions. We have been unable to do so, and furthermore the proof
sketched below is conceptually more satisfactory.

SKETCH OF THE PROOF. (i) Suppose 1r(s) = 0, then 1r(-s) 0.

Set a = p2 + s2 and consider the equation

(6. + a)g (2)

where t:. denotes the Laplace-Beltrami operator of the Riemannian

metric specified in 1. Since the symbol of t:. +a is -A 2 + s2
by standard arguments (2) admits a compactly supported solution
g and since t:. is elliptic, g is real analytic in the exteri-
or of Br and the connectedness of the exterior of Br implies

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE POMPEIU PROBLEM IN EXTERIOR DOMAINS 271

g • 0 in the exterior of Br.

(ii) Let u • g- 1/a. Then for any open set U ~ Br we have
f<u~u- u~u) ~ o.
(iii) Since ~u + au a 1
r
- 1• we have in view of (ii)

(3)

By a simple calqulation the left hand side of (3) is:

Since g s 0 in the exterior of Br, the third term above

vanishes. Furthermore, vol{U/Br) can be made arbitrarily
small. Hence a is real.
(iv) It is clear that if then
-f(A,b) does
not vanish for any >. purely imaginary unless f • 0 a.e.
Therefore reality of a z p2 + ~ 2 implies that of ~. Finally
the existence of a zero of 1r(A) follows from the fact that it
is an entire function of exponential type whose restrictrion to
a* vanishes at infinity - see [9].
This completes the sketch of the proof of the main lemma.

3. THE MAIN RESULTS. Before starting on the proofs of Theorems

and 2 stated in 1, we recall some facts which wlll be used.
DEFINITIONS. (a) An entire function f on em is said to be
of exponential type-R, (where R > 0), if for every e: > 0, 3
constant ce: such that lf(z)l s Ce: e(R+e:) llzll • Vz 6 em.
(b) A function f on Rm is said to be of slow growth if
3 positive constants k and N such that
lf(x)l ;S k ( 1 + llxlj)N, Vx e Rm.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 272 M. SHAHSHAHANI and ALLADI SITARAM

Observe that the (Euclidean) Fourier transform of a compact-

ly supported distribution on Rn is an entire function of
exponential type whose restriction to Rn is of slow growth and
conversely and that under the Fourier transform convolution goes
over to multiplication. If we combine these observations with
the Malgrange division theorem for functions of exponential type
(j8j) and the Titchmarsh-Lions convolution theorem (j3j), we
have the following:

THEOREM A. Let f, g be entire functions on em of exponential

type R1 , R2 respectively suctr that their restrictions to Rm
are of slow growth. If f/g is entire and of slow growth on
then f/g is of exponential type R1 + R2 .
Next we recall the Paley-Wiener theorem of Helgason-Eguchi-
Hashizume-Okamoto ([7] and [4]) for distributions of compact
support on G/K. In [7] the following theorem is established:
THEOREM. The Fourier transform is a bijection from the set of
C~ compactly supported functions on X to C~ functions ~ on
a*c x (KIM) satisfying
(a) ~(X,b) is holomorphic as a function of X
(b) There is a constant R ~ 0 such that for every integer
N ~ 0

sup
X6a~,b€K/M

(c) For every a in the Weyl group and g 6 G we have

I e-(ioX+p)H(g
-1
k)~(oX,k)dk
= J
.
e-(lX+p)H(g
-1
k)~(X,k)dk.

K/M K/M

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE POMPEIU PROBLEM IN EXTERIOR DOMAINS 273

The proof given in [7] extends to the case of distributions

with compact support. We will state this extension here in a
slightly non-standard form which however will be convenient for
our purposes. Before that we introduce some notation. Fix an
orthonormal basis ••. of such that each
transforms according to some irreducible finite dimensional rep-
resentation of K and v0 is the constant function on KIM.
(Note that for any A 6 a*, nA(k)v 0 = v 0 , Vk 6 K and this is the
essentially unique unit vector in L 2 (K/M) with this property.)
If f 6 L1 (X), then nA(f) is completely determined by its
action on v0 • For 6 6 K, let

where x6 is the (normalized) character corresponding to 6.

Using the Killing form restricted to a* as the inner product on

a* we can identify a* with Rm, where m = rank of X and
a~ with em. Hence with this identification it makes sense to
talk about a function on a* being of slow growth or a function
on a~ being of exponential type. We now state the Paley-Wiener
theorem mentioned above in the following convenient form:

THEOREM B. Let f 6 L1 ( x0 \X). Then for some bounded continuous

functions a 1 ,a 2 , ••• ,an on a* and i 1 ,i 2 , ••• ,in 6 z+ u {0}

we have

+ • • • +

Further:
(a) If Supp f ; BR, then each ai(A) extends to an entire
function on a*c of exponential type-R.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 274 M. SHAHSHAHANI and ALLADI SITARAM

(b) Conversely if each of a 1 ,a 2 , ••. ,an extends to an

entire function of exponential type-R on a~, then Supp f ~ BR.
We now come to the proofs of Theorems 1 and 2.

PROOF OF THEOREM 1. For a function g on G, let g denote

v -1
the function given by g(x) = g(x ). Then the hypothesis of the
theorem implies that f * 1r ~ h where h 6 L 1 (G/K) and
Supp h ~ BR(O). (Here * denotes convolution on the group G.)
v
Since is K-biinvariant 1r = 1r and so we have f * 1r = h,

h as above
Now f = LA x6 * f (in the sense of distributions) and each
o6K

x0 * f 6 L1 (x 0 /X). Then by Theorem B we have

+ • • • +

where each ai(l) extends to an entire function of exponential

type-R whose restriction to a* is of slow growth (- indeed
bounded).
Let

+ • • • + b (l)v . •
n 1n

Then each b 1 (l) is a bounded continuous function on a*.

We have:

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE POMPEIU PROBLEM IN EXTERIOR DOMAINS 275

But

LHS

and so we have

We already know bk(A) is a continuous function on a* and

since by the main lemma 1r has only real zeros, bk(A) extends
to an entire function. By part (a) of Theorem B 1r is an
entire function of exponential type r and since ~k(A) is of
exponential type R, it follows from Theorem A that bk(A) is
of exponential type R + r. We can finally conclude from (b) of
Theorem B that Supp f c BR+r"

PROOF OF THEOREM 2. Retaining the notation in the preceeding

paragraph we note that the hypothesis of the theorem implies each
bi(A) is entire. Hence proceeding exactly as before and using
ak(A)
Theorem A, we get bk(A) a---- is of exponential type R + r
1 r (A)
and again using (b) of Theorem B we have Supp f c 8 R+r"

-· CONCLUDING REMARKS. In view of Theorems 1 and 2 and the

results in [9] a natural question to ask is: If ~ e c0 (X) and
I ~(x)dx = 0, Vg e G, then is ~ s 0? (Here Co(X) stands for
gBr
continuous functions vanishing at m.) It turns out that our
main lemma implies that this is not the case. More precisely we
have

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 276 M. SHAHSHAHANI and ALLADI SITARAM

PROPOSITION. Let rank (X)= 1. Given r > 0, there exists a

non-trivial function ~ 6 c0 (X) n L2 +e(X) (for every e > 0)
such that J Hx)dx 0, Vg 6 G.
gBr

PROOF. By the main lemma ~Ao 6 a* such that 1r(A 0 ) ~ o.

Exactly as in [2] or [9] it can then be argued that

J ~A (x)dx = 0, Vg 6 G where ~A is the elementary spherical

gBr o o

function corresponding to Ao 6 a*. However it is well known

that if A0 6 a*, ~A 6 c0 (X) n L2+£ (for any £ > 0) and the
0
proof of our proposition is complete.
For a similar result in the case X Rn, see [10].

BIBLIOGRAPHY
1. C. A. Berenstein and M. Shahshahani, "Harmonic analysis and
the Pompeiu problem," Amer. J. Math. 105 (1983), 1217-1229.
2. C. A. Berenstein and L. Zalcman, "Pompeiu's problem on sym-
metric spaces," Comment. Math. Helvetici 55 (1980), 593-621.
3. W. F. Donoghue, Distributions and Fourier transforms,
Academic·Press, 1969. ---
4. M. Eguchi, M. Hashizume, and K. Okamoto, "The Paley-Wiener
theorem for dist~ibutions on symmetric spaces," Hiroshima Math.
J. l (1973), 109-120.
5. S. Helgason, Differential Geometry and Symmetric Spaces,
Academic Press, 1962.
6. S. Helgason, Groups and Geometric Analysis, Academic Press,
1984.
7. S. Helgason, "The surjectivity of invariant differential
operators on symmetric spaces," Ann. of Math. 98 (1973), 451-480.
8. B. Malgrange, "Existence et approximation des solutions des
equations auz derivees partielles et des equations de convolu-
tion," Ann. Inst. Fourier (Grenoble) ! (1955), 271-355.
9. A. Sitaram, "Some remarks on measures on non-compact semi-
simple Lie groups," Pacific J. Math.~ (1984), 429-434.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 THE POMPEIU PROBLEM IN EXTERIOR DOMAINS 277

10. A. Sitaram, "Fourier analysis and determining sets for Radon

measures on Hn,n Ill. J. Math. ~ (1984), 339-347.
11. L. Zalcman, "Offbeat integral geometry," Amer. Math. Monthly
87 (1980), 161-175.

JET PROPULSION LABORATORY

CALIFORNIA INSTITUTE OF TECHNOLOGY
PASADENA, CALIFORNIA 91109
AND
DEPARTMENT OF MATHEMATICS
UNIVERSITY OF WASHINGTON
SEATTLE, WASHINGTON 98195

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

CURVATURE INTEGRALS AND CHERN CLASSES OF SINGULAR VARIETIES

THEODORE SHIFRIN

1. INTRODUCTION. Classical integral geometry deals with curva-

ture integrals on Riemannian manifolds and relations among them;
as typical examples, one has isoperimetric inequalities, and
Blaschke-Poincare kinematic formulas (of. [13]). Chern's defini-
tion of characteristic classes by means of curvature forms ([1])
forged a link between differential geometry and topology with
what is perhaps the ultimate integro-geometric formula - the
Gauss-Bonnet theorem (in its various obstruction-theoretic
guises). Moreover, integral geometry proves to be a powerful
weapon in dealing with questions in noncompact algebraic geometry
(e.g., value distribution theory) and noncompact Morse Theory.
The prototype here is a local Gauss-Bonnet theorem, say for
complex submanifolds of eN:

PROPOSITION. Let X c eN be an n-dimensional complex submani-

fold (with compact closure) and let QX be the induced hermitian
curvature form. Let H e PN- 1 * be a hyperplane in eN and let
n(X, H) be the number of points x e X such that TxX c H.
Then

(1.1)

(where cn(QX) is the top Chern form of X).

@) 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page
279
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 280 THEODORE SHIFRIN

Turning to the case of singular algebraic varieties, it is

easy to see that even in the presence of singularities, volume
and curvature integrals are still finite (although the curvature
forms themselves tend to blow up - cf. Section 3 for a particular
example). Nevertheless, one ought to be able to detect the
singularity with curvature integrals ([8]). For example, a
great deal of work has been done on the case of a hypersurface
v 0 = {f(z 0 , ..• ,zn) 2 0} c CN+ 1 with an isolated singularity at
the origin, by considering v0 as embedded in a family of smooth
hypersurfaces vt = {f(zo· ,zn) = t}. From the (more global)
topological point of view, one is interested in the number
of vanishing cycles, and this is the top Milnor number
~(n+ 1 )(V 0) in this situation. As a typical example, one
has f(x, y) = y 2 x 2 (x + 1), where the local picture is

and the global picture is

The main point here is to calculate the "defect"

where Be:= {!!zll s e:} c CN+ 1 • Indeed, using (1.1), we can see

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 281

that local Morse theoretic data should enter here. One goal
of such work ([5], [6], [11]) was to give a formula for the
top Milnor number in terms of limiting curvature integrals

Iv n B c j C0 v )" 00
n-j
'
where w =
I=T
21r a-a log 11 z 11 2 • In any event,
t e: t

such results may be viewed as an extrinsic study of the

singularity by curvature integrals.
Another natural approach is the intrinsic and this is the
thrust of this note. Given a singular subvariety xn c PN and,
say, a stratification S of X by complex submanifolds S,
each S inherits a hermitian structure from PN and therefore
is endowed with a canonical curvature form n8 . As a first step
one might consider only the smooth locus X0 c X and the associ-
ated curvature integrals

I~(X)
J
= IXocn-j(n Xo> ,. wj, ( 1 • 2)

where w is the Kaehler form on pN (the reason for the odd

notation will become clearer in Section 2). Indeed, one can do
more, for it is not hard to see that the Chern forms
define closed, integral currents of bidegree (i, i) on ~. and
these currents enjoy many of the properties of the classical
Chern classes of a smooth variety. For example, a general sort
of adjunction formula relates the Chern classes of a generic
linear section of a smooth variety to those of the variety
itself. In integral geometry one has a simple kinematic formula
on a local level ([14]), which, when phrased in terms of the
integrals (1.2), says that for j = 0, ..• ,n- k, and L a
generic linear space of codimension k,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 282 THEODORE SHIFRIN

( 1 • 3)

We shall refer to (1.3) as the reproductive property of the

curvature integrals (1.2). Indeed, in the course of proving
(1.3) we observe that there is an adjunction formula for the
currents themselves.
As a reasonable next step, one can consider such curvature
currents and integrals for all the strata S of S, and ask how
one can piece them together to retrieve intrinsic information
about X. To do this, one need only identify the currents
ci(n 8 ) as the Chern-Mather classes of S and then use work of
MacPherson, Le-Teissier, and Dubson (among others) to combine
them appropriately to give the Chern-MacPherson classes of X.
It will then follow that they have the reproductive property as
well.
These matters form the heart of this paper and occupy Sec-
tion 2. In Section 3 we turn to the example of the Whitney
umbrella in p3
• and use it to illustrate all the concepts and
results of the previous section. Finally, we conclude in Section
4 with a few questions which this work elicits.
I would like to thank Gary Kennedy, Clint McCrory and Robert
Varley for the stimulating conversations which grew out of our
seminar on Chern classes of singular varieties and which led to
this work. I would also like to thank Robert Bryant for his
invitation to speak at the Bowdoin Conference.

2. CHERN FORMS ON SINGULAR VARIETIES. Let X be a connected

n-dimensional subvariety of PN, X0 c X the smooth locus of X,
and n the curvature form on x0 induced from the Fubini-Study

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 283

metric on PN (in fact, embedding X in any hermitian manifold

will do just as well). Then the Chern forms

1- 1 . a 1 ••• aj
(---) J - .I ~ e: . Q -
21T J. B1"'Bj a181

define intrinsic closed forms of type (j, j) on X0 ; unfortu-

nately, as such, they represent neither cohomology nor homology
classes on all of X. However, they can be viewed as currents on
the singular variety X in a natural way (cf. [7] for the appro-
priate definitions).
We begin with the Gauss map Y: Xo • X x G(n, TP N). Here
G(n, TPN) is the Grassmann bundle whose fiber over z 6 PN
consists of the n-dimensional subspaces of the tangent space
Tz PN • While it is more intuitive to think of the classical
projective Gauss map to G(n, N), the Grassmannian of Pn's in
PN, then the projective twist enters as usual with the Euler
sequence on G(n, TPN)

0 + 0( -1 ) + E + E Q 0( -1 ) + 0, ( 2. 1)

where 0(-1) is the tautological line bundle on PN, E is the

tautological bundle on G(n, TP N ), and E is the tautological
bundle on G(n, N), pulled back appropriately. We put the
obvious hermitian structure on E, and then it follows by
naturality that Q = Y*(QE).
To consider the behavior of n near the singularities
of X, it behooves us to look at the closure X of
Y(X) c X x G(n,TP N ). This variety, called the Nash blow-up of
!• has the virtue that it is the minimal resolution of X which
carries a tangent bundle, namely the restriction to X of the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 284 THEODORE SHIFRIN

tautological bundle E. Let v : X + X be the projection. Then

cj<nx) = v*(cj(nE)) is a bidegree (j, j) integral current on

N
X; moreover, since c/nE) is a closed form on G(n, TP ), it
follows that cj(nX) is a closed current on X. In particular,
the Chern forms cj(n) on X0 , being the restrictions of
cj(nx) to x0 , are integrable. 1 Geometrically, the current
cn-j<nx) represents the homology class v*(P.D. cn-j(E)), where
by P.D. we mean "cap with the fundamental homology class of
X." The latter is by definition the £h Chern-Mather class
c~(X) e H2 j(X, Z). For example, c~(X) e H2 n(X, Z) is the
fundamental class of X, and c~(X) = J 0 cn(n)
= Jxcn(nE) e H0 (X, Z) s z. If x is sm;oth, of course, then by
the Gauss-Bonnet theorem, c~(X) x(X), but in general there is
no obvious formula here. If X is a curve with an ordinary
d-fold point, then X is the normalization of X and
M A

c 0 (x) = x(X).
One of the standard geometric techniques - both classical
and modern - is that of slicing. In algebraic geometry one has
Lefschetz theory and various adjunction formulas; in integral
geometry, one has kinematic formulas. To juxtapose the two, let
w be the Kaehler form on X c PN, and put

(2.2)

1 For this reason, we will no longer distinguish notationally

between the two, since from the context it will be clear with
which we are working.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 285

this is the degree of the Chern-Mather class In [14] it

is proved that this functional on smooth varieties enjoys the so-
called reproductive property:

THEOREM. Let X c PN be an n-dimensional complex submanifold

with compact closure. Normalize the invariant measure dL on
G(N - k, N) to have volume 1.

(a) For j • 0, ••• ,n-k,

J I~(XnL)dL ~I. (- 1 )i-j-k (i-j-1)

1 M
Il.(X). (2.3)
G(N-k,N) J imj+k k-

(b) Inverting this, we recover the integro-geometric in-

variants of X from those of its average linear section: for
i "' k, ••• ,n,

M min(i,n-k) k J M
I .(X) -
J
L
jzi-k
(k+ ._.) G(N-k,N) IJ.(xnL)dL.
J l
0 (2.4)

It is worth remarking that while compactness and Poincare duality

are fundamental in the proof of this theorem, it is not necessary
that X be compact and boundaryless. Now, by applying the
theorem to the smooth locus X0 of X we obtain the following
result on the degrees of Chern-Mather classes.

THEOREM. Let X c PN be an n-dimensional algebraic variety, and

let L 6 G(N-k,N) be generic. Then for j = 0, ••• ,n-k,

(2.5)

and for i = k, ,n,

2we remark that the degree of a cycle on a singular projective

variety ~s perfectly meaningful, for one uses the smooth ambient
space P to compute the requisite intersection number.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 286 THEODORE SHIFRIN

min(i,n-k) k M
?
. j=l-k
(k+J.-i) IJ.(XnL). (2.6)

PROOF. First of all, as we have already indicated, we obtain an

integrated version of (2.5) by applying the kinematic formula

(2.3) to X0 • However, in this algebraic setting, we would like

to remove the integration on the left-hand-side. That is, we
need to know that is constant for generic L.
Now, if L is transverse to X0 (and of course this is a
generic property), we have the exact sequence of hermitian vector
bundles on (X n L) 0 = x 0 n L:

0 ~ T(XnL) ~ TX ~ 0(1)@k ~ 0 (2.7)

(here 0(1) is the hyperplane bundle, the dual of the tautologi-

cal bundle 0(-1)). From (2.7) it follows that on (X n L) 0

j-i (2.8)
" w •.

Integrating cn-k-j(OXnL) " wj over (XnL) 0 gives

I~{XnL), and so

n-~-j _ n-k-j-i(n-j-i-1) J n-k-i

.L ( 1) n-k-j-i oci(Ox) "w
1=0 (XnL)

Therefore we need only show that the typical integral

w
n-k-i (2.9)

is independent of (generic) L. Recall, however, the current-

theoretic interpretation: since c 1 <ox>• and hence c 1 <ox)"

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 287

w
n-k-i is a closed current on X, and since X n L and
X n L' are homologous analytic cycles, it follows that the
integral (2.9) is the same for L and L'.
Lastly, the latter formula (2.6) follows by a combinatorial
inversion. o

REMARK. The proof shows that something stronger holds: for L

generic,

(2.10)

is an equation of currents on X n L (here H is the hyperplane

class); we may view this as an adjunction formula for singular
varieties. To extablish this directly (purely topologically)
seems to require a closer examination of the Nash blow-up and,
further, some use of Whitney conditions.
The Chern-Mather classes, while of geometric interest them-
selves, prove to be the necessary tool for defining MacPherson's
generalized Chern classes [12], for which we shall use the (non-
standard) notation c*MP (X). These classes have a great virtue,
functoriality in the category of stratified spaces (and so it is
not surprising that it was Deligne and Grothendieck who first
conjectured their existence). From this functoriality, in par-
ticular, follows the Gauss-Bonnet property

(2.11)

The Chern-MacPherson classes c~P(X) e H*(X, Z) are expressible

in terms of a Whitney stratification S of X; this leads us to
the natural curvature-theoretic interpretation of these classes
in terms of curvature forms of all the strata, and not just the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 288 THEODORE SHIFRIN

top-dimensional one as we've done so far. The main point for us

here (cf. [2], [10] for details) is that there are integers
Ss(X) for the smooth strata S of a Whitney stratification of
X which satisfy the property

Ss(X) = eSnL(XnL) for L a generic linear space, (2.12)

and in terms of which one has the equation

(Here we use the natural inclusions is S c-+ X tq push forward

homology classes and currents.)
Since the coefficients Ss(X) are stable under generic
linear section, it is immediate from our earlier theorem that
Chern-MacPherson classes also enjoy the reproductive property.
Specifically, let

THEOREM. Let X c PN be an n-dimensional algebraic variety with

Whitney stratification S, and let L 6 G(N-k,N) be transverse
to all strata. Then for j s 0, ... ,n-k

(2.14)

and for i k, ... ,n,

min(i,n-k) k MP
rt:1P(X) . ~ (k+j-i) Ij (XnL). (2.15)
1
J=l-k

PROOF. If L is transverse to all strata, then we apply (2.5)

to the closure s of a typical stratum S to obtain

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 289

. M
I. (
J
snL) ,.
dim S
L ( -1 ) - r
i=j+k
i
e-k-1J-
. k . . k M
) Ii ( s) .

Thet'efot'e

I~p(XnL)
J
L eSnL(XnL) I~(SnL)
dim(SnL)~j J
L e3 (x) I~(snL) by (2.12)
dim S~j+k J

dim S . . k i j 1 M
L e (X) L (-1) 1 _J_ ( - - ) I (S)
dim S~j+k S i=j+k k-1 i

~L (- 1 )i-j-k(i-J_·-1) '\'
L
M-
e 8 (S. )Ii(S)
i=j+k k 1 dim S~i

REMARK. As befot'e, one has the stt'onget' adjunction-type fot'mula.

We at'e not awat'e of its statement in the t'ecent litany of litet'a-
tut'e on Chet'n-MacPhet'son classes. That is, fot' j = 0, ..• ,n-k,
and L S G(N - k, N) genet'ic,

Before concluding, a few comments on the natut'e of the

coefficients e 3 (X) at'e in ordet'. Indeed, we wish to make the
point here that they too are computable in terms of curvature
integrals - this time, local ones. Let X be an algebraic
variety equipped with a Whitney stratification, and let p e X.
Then MacPherson [12] defined a topological obstruction Eup(X)
which measures the singularity of x at p. As work of Gonzalez-
Sprinberg ([4], [10]) shows, it reflects the simultaneous limit-
ing behaviot' of tangent planes and chords at p. Specifically,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 290 THEODORE SHIFRIN

working with a germ of X at 0 in eN, consider the rational

map
X .,. X X
PN-1 X G(n,N)
X ... (x,["Ci1J, TXX)

Let Y be the closure of the image, and Z the fiber over 0.

It is not hard to see that Z c Y is a divisor and Gonzalez-
Sprinberg proves that

(2.17)

where E and 0(-1) are the restrictions to Z of the

tautological bundles on G(n, N) and PN- 1 respectively, and
E- 0(-1) is the formal difference bundle. Having arrived at
this point, one expects a formula for Eu 0 (X) as a residue-type
curvature integral, and from standard complex differential geo-
metric techniques it follows that (cf. [11])

(2.18)

where s£ = {II z II = £}::. eN, n = a log II z 11 2 • and q, is the

hermitian curvature form of the bundle of rank n - 1 whose
fiber at X 6 X0 is the orthogonal complement in TXX of the
position vector 0}. (Note, by the way, that since the Whitney
conditions hold, any limiting tangent plane must contain the
corresponding limiting chord, and so for £ small, Ox cannot
be orthogonal to TxX.)

EXAMPLE. If X is a curve, then Eu 0 (X) is the multiplicity of

X at 0; if X is a surface, then the formula above takes the
simple form

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 291

(2.19)

The case of a surface with isolated singularity was treated in

[15].

It is a fact (of. [2], [10]) that Eup(X) is a constant as

p varies on a stratum of a Whitney stratification S of X. It
follows that we can now define the coefficients Bs(X)
recursively by the equation

1' all X 6 X, (2.20)

where it is understood that Eu (S)

X
0 if x e s, and

EXAMPLE. Let X be a variety with one singular point p. Then

we take the stratification so = xo and s1 = { p}. Choosing
X 6 so, Eux(X) = 1 and so Bxo (X} = 1. Choosing X = p, we
have the equation

whence ep(X) = 1 - Eup(X). For example, if X is a curve and

p is a point of multiplicity d, the ep(X) 1 - d.

3. THE WHITNEY UMBRELLA. Let (w, x, y, z) be homogeneous

coordinates on p3 and let be the
'
Whitney umbrella. The singular locus L of X consists of
they-axis, with two distinguished points (1, 0, 0, O) and
(0, 0, 1, 0), called pinch points. The projective Gauss mapping
is given by the rational map

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 292 THEODORE SHIFRIN

Y: X ---> p3*
(3.1)
-
Y(w,x,y,z) = (x 2 ,2xw,-z 2 ,-2yz)

and one checks easily that the image Y(X) satisfies the same
equation; i.e., the umbrella is self-dual.
To understand the geometry of X and its various Chern
classes, we begin by finding the normalization X of X, and
from that the Nash blow-up X. First of all, we observe that the
obvious "parametrization"

(s,t) --> (1 ,st,t 2 ,s) (3. 2)

of the affine portion of X extends to P2 as a linear system

of conics with base point (0, 1 ' 0). Blowing up the point
(0, 1 ' 0) in p2 gives us a smooth surface X which is
birational to X and, moreover, the mapping we have given is a
biholomorphism of X - { t1 = 0} with X - L One checks in
particular that the exceptional divisor in X maps biholo-
morphically to a distinguished line {w = y 0} at infinity
in X.
To determine the Nash blow-up X ¥ X, it is convenient to
lift the Gauss map (3.1) to the normalization X. Since at each
pinch point there is a P1 of limiting tangent planes, it is
not surprising that we must blow up the points (1, 0, 0) and
( 0' 0, 1 ) in p2 c X in order for the lift of y to be
globally defined. Having done so, we let X be p2 with each
of the vertices ( 1 ' 0, 0)' ( 0' 1 ' 0) and (0, 0, 1 ) blown up.
Then one can check that the rational map

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 293

on gives rise to a well-defined map Y: X+ p3*, and so

there is clearly a well-defined Nash tangent bundle Y*E on x.
To fix notation, let a, a', a'' 6 H2 (X) be the exceptional
divisors corresponding to (0, 1, 0), (1, 0, 0), (0, 0, 1) re-
spectively. Then one checks that Y maps the cycle a doubly to
the line {w* ~ y* = 0} in p3* and the cycles a' and a''
to the lines {w* = z* = 0} and {x* = y* = 0} respectively.
(Geometrically, for example, the limiting tangent planes as we
approach the pinch point (1, 0, 0, 0) form the pencil of planes
containing the z-axis x = y = 0.) On the other hand, a general
l(ne 1 c P2 c X evidently maps to a cubic curve in p3*. As a
check, we may view the map Y as arising from a linear system of
cubics in P2 passing doubly through (0, 1, 0) and singly
through (1, 0, 0) and (0, 0, 1). The associated linear system
on X is therefore 131 - 2a - a' - a"l, and this system has
degree (3H- 2a- a' - a'') 2 9 - 4 - - 1 = 3, echoing the
fact that the dual of the umbrella is a cubic surface in p3*.
It is now straightforward to compute the Chern-Mather
classes of X. From an application of the Mayer-Vietoris se-
quence we infer that H2 (X) a Z ~ Z. It will be convenient
to use as a basis the cycles A v* (a). = {w = y = 0} and
B = {x = z, w = y} = v*(1- a). Clearly v*(a')
The hyperplane section of X is given by H = A + 28; for
reference, we note that

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 294 THEODORE SHIFRIN

the singular locus r satisfies the homological equation

2r =A+ B, since v- 1 (r) is a line in P2 c X and maps doubly
to L

Now, to calculate cM
1 (X) , we begin by computing c 1 (Y*(E)),
where E is the tautological J-plane bundle on P3*. This is of
course in homol08:£ the cycle -~- 1 (H*), where H* c p3* is a
hyperplane, and from our earlier discussions of the map Y it
follows that

c 1 (Y*E) • 2a+a'+a''-31.

We next infer from the Euler sequence (2.1) that c 1 (E)

= c 1 (E) + 3H, where H is the hyperplane class on X c P3.
Therefore

c~(X) = v*(c 1 (Y*E)) = v*(2a+a'+a''-31) + 3H

2A - 3(A+B) + 3(A+2B)
= 2A + 38 (or, equivalently, B+4r).

So far as the other interesting Chern-Mather class is concerned,

we note that the degree of c~(X) 6 H0 (X) a Z is given by
c 2 (Y*E) 6 H4 (X) • Z, which we again compute via E and (2.1).
In particular, c 2 (E) 3H • H + 2c 1 (E) • H + c 2 (E). The missing
datum is c 2 (Y*E) = #{x 6 X : Y(X) 6 1 0 } for a generic line
10 c p3*, and this is of course the degree of ~(X) c p3*,
i.e., three. Assembling everything, we deduce

c~(X) = 3 degree X+ 2(-A-3B)•(A+2B) + 3

3 • 3 + (-2)(4) + 3 = 4

(here the intersection calculation is actually done on the smooth

surface X).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 295

From the viewpoint of curvature integrals, we use the

parametrization (3.2) to calculate the forms explicitly. Pulling
back the Kaehler form of p3* by the Gauss map and then
accounting for the projective twist with (2.1), we find (gasp!)
that

-4 2 2 4
----4----------------------{ < 1t 1 <1 + 1t 1 ><1 + 1t 1 >>ds " ds +
[(1+1tl >lsl2+41t12(1+1t12)]2 .
(41t14 + lsl2(1+1t14 +1st12 + 41t12)) dt "dt +
<lt1 4 - 2lt1 2 - 1)(st ds" dt + st dt" ds)} +

______ 3_______ { ( ( 1 +I t 12 )( 1 + I t 14 )) ds " ds +

[1+lst12+1t14+1si2J2
<lsl2(1+lsl2 + 41t12 + 1t14) + 41t12)dt" dt +
(1-2lt1 2 1t1 4 ><st ds" dt + st dt" ds)}.

Note that at s = t = 0, the coefficients are O((lsl 2 + 1t1 2 >- 1 >

and hence 0 is locally integrable. It is worth remarking that
o2 is also O((lsl 2 + 1t1 2 >- 1 ), so that the Chern forms do
indeed define integral currents on X. Moreover, one can (and
did!) explicitly integrate to find that

J c1(0) a 5, J c1(0) = 2,
{s-O} {t=1}

and this checks with our earlier homological results (although

one must be careful).
We should also comment that the reproductive property (2.6)
is easily verified. The generic plane section X n H of X
misses the pinch points and is transverse to the singular locus
I:, and is therefore a nodal cubic, whence

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 296 THEODORE SHIFRIN

• degree (X n H) = 3 and I~(X) = degree c~(X)

degree (2A + 3B) 5, since A and B are both lines in P3.
Passing to the Chern-MacPherson story, we must first compute
the coefficients e8 (X). Working with the Whitney stratification

where p1 ' p2 are the pinch points, we find that for X 6 I:,
Eux(I:) 1' since X is a point of the smooth variety I: ; p 1.
Now, if X 6 s1' Eux(X) = 2 since X (locally) has two smooth
sheets at x; but i f X = pi' Eux(X) must be computed by either
(2.17) or (2.18). To apply the Gonzalez-Sprinberg formula
( 2.17), we need to know the simultaneous blow-up Yc X " P2 " P 2 *
(working locally in a neighborhood of (0, 0, 0) 6 c3). By
explicit computation of the relevant equations, one finds that
the locus of limiting tangent planes is the P1 * c P2 * of planes
containing the z-axis (with an embedded component corresponding
to the special yz-plane!), and that the locus of limiting chords
is the P1 c P2 of lines through the origin in the yz-plane,
counted with multiplicity two. Therefore Z is the non-reduced
subscheme of P2 " P 2 * consisting of a double line in the first
factor and a line in the second (having a common point). Since
both tautological bundles have first Chern cl~ss -1 on a re-
duced P1 , we find that

Eup. (X) = c 1 (E-O( -1) )[Z] -1-2(-1) +1 •

1

One now applies (2.20) to compute that

ep_<x> 1, e8 (X) -1, e 0 (X) 1.

1 1 X

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 CURVATURE INTEGRALS AND CHERN CLASSES 297

The Chern-MacPherson classes are now easily computed by

(2.13).

as (X) c~(E) + c~(X)

1

(-1) c~(E) + c~(X) = -r + B + 4r B + 3L

And

a (X) + a (X) + as (X) c~P(E) + c~(X)

P1 P2 1

1 + 1 + ( -1 ) ( 2) + 4 = 4 '

which checks the Gauss-Bonnet rheorem c~P(X) x(X), since we

know the Betti numbers of X.
So far as the reproductive property (2.15) is concerned, we
observe that I~p(X) = degree c~P(X) = 4, since B and E are
both lines. Now I~P(X n H) = degree c~P(X n H) x(X n H) = 1,

since X n H is a nodal cubic. And IMP(X

1
n H) = 3, since
degree (X n H) = 3, and so (2.15) checks.

4. A FEW QUESTIONS. 1. MacPherson's Chern classes are uniquely

determined ([12]) by a functor'ial property (which we've not made

explicit) and their values for smooth varieties. In particular,
MP
from functoriality it follows that c 0 (X) = x(X). Is there some
sense in which the latter condition and the rep!:'oductive p!:'operty
(2.16) should cha!:'acte!:'ize c~P?
2. We stipulated that the function Eup(X) is constructible,
i.e., is constant on strata of a Whitney stratification of X.
Can one see this di!:'ectly fr'om Loeser's integral formula (2.18)?
3. (Pit-man Wong) Miyaoka's famous result that for a smooth
surface X of general type J c 1 (o) 2 - 3c 2 (o) ~ 0 may be viewed
X

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 298 THEODORE SHIFRIN

as an isopermetric-type inequality in integal geometry. How is

this inequality affected by singularities?

BIBLIOGRAPHY

1. s. s. Chern, "Characteristic classes of hermitian mani-

folds," Annals of Math ..!!.1. (1946), 85-121.
2. A. Dobson, Calcul des invariants numeriques des
singularites et applications, SFB Theoretische Mathematik
Universitaet, Bonn, (1981).
3. W. Fulton, Intersection Theory, Springer Verlag (1984).
4. G. Gonzalez-Sprinberg, "L'obstruction locale d'Euler et
le theor~me de MacPherson," Asterisque, 82-83.
5. P. Griffiths, "Complex differential and integral
geometry and curvature integrals associated to singularities of
complex analytic varieties," Duke Math J • ..!!..2_ (1978), 427-512.
6. G. Kennedy, "Griffiths' integral formula for the Milnor
number," Duke Math J. ~ (1981), 159-165.
7. J. King, "The currents defined by analytic varities,"
Acta Math. 127 (1972), 185-220.
8. R. Langevin, "Courbure et singularites complexes,"
Comment. Math Helv. ~ (1979), 6-16.
9. R. Langevin and T. Shifrin, "Polar varieties and
integral geometry," Am. J. Math. 104 (1982), 553-605.
10. Le D.T. and B. Teissier, "Varietes polaires locales et
classes de Chern des varietes singulieres," Annals of Math~
( 1981). 457-491.
11. F. Loeser, "Formules integrales pour certains invariants
locaux des espaces analytiques complexes," Comment. Math. Helv.,
to appear.
12. R. MacPherson, "Chern classes for singular algebraic
varieties," Annals of Math. 100 (1974), 423-432.
13. L. A. Santalo, Integral Geometry and Geometric
Probability, Addison-Wesley (1976).
14. T. Shifrin, "The kinematic formula in complex integral
geometry," Trans. Am. Math. Soc. 264 (1981), 255-293.
15. A. N. Varchenko, "The integrality of the limit of the
curvature integral along the boundary of an isolated singularity
of a surface in c3," Russian Math Surveys 33: 6 (1978), 263-264.

DEPARTMENT OF MATHEMATICS
UNIVERSITY OF GEORGIA
ATHENS, GEORGIA 30602

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

HYPOTHESIS TESTING IN INTEGRAL GEOMETRY

GUESSING THE SHAPE OF A PLANE DOMAIN

PETER WAKSMAN

INTRODUCTION. I will discuss certain probability distributions

associated to domains S c R2 . For convex domains these are the
distributions of chord lengths, which can be generalized to non-
convex domains. Also I wish to talk about taking these distribu-
tions as measuring an object and about the fundamental difference
between direct measurement and "hypothesis testing" (where we
compare the measurements of an unknown object to the measurements
of known objects). In Section 3 the second method is used to
solve the problem of guessing the shape of a domain given obser-
vations along a finite number of random lines. (For a more
detailed exposition of these ideas see [5]).
The present probabilistic approach to shape recognition is
natural for the recognition of moving objects. One hopes it will
be useful in other contexts where deterministic approaches are
inappropriate.

1. THE DISTRIBUTIONS: Let S s R2 be compact, connected, with

piecewise smooth boundary, and assume that closure (§) = S. We
let dt = the rotation/translation invariant measure for oriented
lines in the plane (see Santalo [2]), and let L(S) = lines

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

299
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 300 PETER WAKSMAN

s 2LC, LC is the perim-

0
meeting (so measure (L(S)) = where
eter of the convex hull of s).

A. FOR s CONVEX: Let 0 : L(S) _. R be the function assigning

e L(S) s
0

to each line R. the length of the intersection i. n

( 0 is called the "Radon transform"), and let B8 (t) = measure

{[Link](i.) ~ t}. Then B8 (t)/2L =the cumulative distribution of o
(L = the perimeter of S). For example: for S an m x n

rectangle, Bs = 4Bmn where Bmn(t) = t1[0,m](t) + ml[m,n](t) +

/t2 m2 /t2 n2
+ (m + n -t)1{n,d](t) + n (--r=--)1[m,d](t) + m( t- )1[n,d](t)

where and 1 = {1 if a ~ t ::; b Thus

[a,b](t) 0 otherwise
Bmn(t)/(m + n) = the probability that a random chord is of length
~ t. Bmn and dBmn/dt are curious looking distributions (see
Figure 1.)

B. FOR S NON-CONVEX: More than one generalization is possible.

Generalizing o to be the sum of the lengths of the components
s, o is the classic Radon transform.
0
of i. n It is difficult to
calculate the distribution of o for any non-convex examples,
although it is sometimes easy to measure (e.g. in C.A.T. scan-
ning). Thus we consider a different generalization. Let

1 if a ::1 t
Ha(t) = {
0 otherwise

and notice that in the convex case we can write

Bg(t) = fL(S)Ho(R.)(t)di.

For non-convex s, we generalize Ho(i.) as follows

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPOTHESIS TESTING IN INTEGRAL GEOMETRY 301

10
1
10

1 1

!I
I
I

~
•
1\ I

:I ' -=--=i\~.
10 '\ I
101
J=
1
I
1 1

1 1

Figure 1. Showing s, B3 K 4Bmn and for S

a rectangle

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 302 PETER WAKSMAN

a)

~ 1
10
10

1 1

b)

T
~ IOkc= 1
±

c)
T
l
~
1
:1

1 !Ole:: 1
10

:r
±
I

1
,===t-

d
Figure 2. Showing s, Bs = L(±)Bmn and Bs dt B3 for some
examples of non-convex S

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPOTHESIS TESTING IN INTEGRAL GEOMETRY 303

DEFINE: T(x,y) for x,y 6 a(t n §) by

T(ai,bj) T(bi,aj) + 1

T(ai,aj) = l(bi,bj) { -1 i f i f. j
0 if· i B j

where (ai, bi) is a component of t n S and ai < bi < ai+1

on t.

DEFINE: Htns(t) = ~ 1(x, y), the sum being over all

d(x,y):>t
x,y 6 a(t n S) with x ;:; y on t. Call Htns(t) the t-
crossing number for the line t, relative to S. Also, we define

Note that s 8 (t)/2Lc the expected value of the t-crossing

number. (For fixed S let Xt(t) = HtnS(t). Then Xt(t) is a
non-negative integer for each t, so {Xt}t>O is a parametrized
family of non-negative integer valued random variables on L(S).
This is an example of a stochastic process. B8 /2Lc is the
expectation of the process, and can still be measured
statistically.)

THEOREM: For S a union of coordinate rectangles (write

S = u (±)mxn), B3 = ~(±)Bmn' the summ being over all ordered
pairs of corners. For a given pair of corners, m and n are
the horizontal and vertical separations between the corners and
the (±) sign is given by

+ if the corners are in the same quadrant (mod. 2)

( ±) {
- if the corners are in different quadrants (mod. 2),

if both corners are black or both are white; the reverse if one
is black and the other is white. (Figure 2 shows B8 for some

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 304 PETER WAKSMAN

non-convex domains. Using the theorem one calculates that Bs =

16B 11 - 8B 12 + 2B 22 for example 2a); and Bs = 8B 11 + 2B 22 for

example 2b). Thus the Bmn are like the sines and cosines of
the trigonometric polynomials.

2. DIRECT MEASUREMENT VERSUS HYPOTHESIS TESTING. BS measures

S; for S U(±)mxn we have a "dense" collection of shapes
and so for an arbitrary domain T, BT can be approximated by
such Bs. Thus we ask: Given BT, what is the best approxima-
tion by a function of the form Bs
hypothesis that T = S" for these special S's).
Blaschke [1] asked (in the context of a convex T): 1. Does
BT determine T? 2. Given an f, when does there exist a T
such that BT f? From a practical point of view these are the
wrong questions because they suggest that, if the answer to 1. is
negative, we should abandon this approach to shape recognition.
However,non-uniqueness is rare: (one can show [3, 4] that a
generic polygon is uniquely determined by its B-function. More
simply 1. is not relevant to the question of calculating the
approximation and to the existence of S. Also, 2. is wrong
since every function can be approximated (although not necessari-
ly to within an arbitrary degree of accuracy).

3. STATISTICAL QUESTIONS. Let

1 k
H(t) k LHt T(t)
i = 1 in

For a random sample { .1!. 1 , .1!. 2 , . . . , .l!.k} of lines. The problem of

guessing the shape of T given these measurements is interpreted

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 HYPOTHESIS TESTING IN INTEGRAL GEOMETRY 305

as follows: We need to choose a manageable norm • II , and

to find an S B U(±}mxn so that is mini-
mized. (The study of different norms is thus part of integral
geometry). A strategy is to. restrict the possible S's further
by letting vN,q be an N X N square divided into smaller
squares and constructing S as a union of these smaller squares.
The best choice in vN,q could guide the choice in vN,q+ 1
or vN, 2 q. Thus we interpret the problem of guessing shape in
terms of the theory of approximation by the special functions

REFERENCES

1. W. Blaschke, Vorlesungen uber Integralgeometrie, Chelsea

Pub. co. (1949), p. 51.
2. L. A. Santalo, Integral Geometry and Geometric
Probability, Encyclopedia of Mathematics, vol. 1, Addison-Wesley
Pub. Co, Reading, Mass (1976).
3. P. Waksman, "The associated function of a plane polygon,"
Ph.D. dissertation, University of Minnesota (1983).
4. P. Waksman, "Plane polygons and a conjecture of
Blaschke's," Journal of Applied Probability (Dec. 1985).
5. P. Waksman, "Hypothesis testing in integral geometry,"
Transactions of the A.M.S. (Sept. 1~86).

DEPARTMENT OF MATHEMATICS
UNIVERSITY OF SOUTHERN CALIFORNIA
LOS ANGELES, CALIFORNIA 90089-1113

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

NON-LINEAR INTEGRAL TRANSFORMS

R. S. WARD

1. NON-LINEAR INTEGRALS. This lecture deals with the use of non-

linear integral transforms to solve non-linear partial differ-
ential equations. Little of the material is new (indeed, much of
it is ancient history), but the approach described here is per-
haps a novel one.
The following two examples should illustrate what I mean by
"non-linear integrals."
EXAMPLE A. Consider the system of linear ordinary differential
equations f'(t) = A(t)f(t), where f(t) is an n-vector and
A(t) an n x n matix. The solution of these equations has the
form f(t) • UA(t)f(O). Now if n = 1, or if (say) A and B
are diagonal matrices, then UAUB = UBUA = UA+B and there is a
linear structure. Indeed, UA is (in such a case) the expo-
nential of an integral of A. But in general, of course, one
does not have this sort of linearity: UA is a "non-linear
integral" of A. Note, incidentally, that this example has a
geometric interpretation: namely that A comes from a connec-
tion on a vector bundle, and the differential equations represent
parallel transport of a vector f.
EXAMPLE B. This concerns Wiener-Hopf matrix factorization, also
known as the Riemann-Hilbert problem, or as the probiem of trivi-
alizing a holomorphic vector bundle over the Riemann sphere
@ 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page
307
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 308 R.S. WARD

Given a non-singular n x n matrix F(~) of functions

of a complex variable ~. which is holomorphic in a neighbour-
hood of the circle 1~1 = 1, the problem is to find non-singular
matrices H(~) and K(~) such that F = HK, K is holomorphic
for 1~1 s 1, and H is holomorphic for I~ I ~ including
~ = ... If, say, F is diagonal, then the problem becomes
linear: If F1 = H1 K1 and F 2 = H2 K2 , then
F1F 2 = (H 1H2 )(K 1 K2 ). More explicitly, consider the case n = 1.
If the "winding number" of F vanishes, then log F is holo-
morphic near 1~1 = 1. A Cauchy integral now splits log F into a
Taylor part plus a Laurent part, and K and H are simply the
exponentials of these. So again, we have an example of a one-
dimensional integral (when n = 1) and its non-linear general-
ization (for general n).

2. WHITTAKER'S FORMULA AND GENERALIZATIONS. In this section we

shall look at a particular example of an integral transform which
has a non-linear generalization. It arises from the famous
formula of E. T. Whittaker [1] for the general solution of
Laplace's equation in three dimensions,

0. (1)

Namely, take an arbitrary function f(Y,e) of a complex variable

Y and a real variable e, and put

21T
cf>(x,y,z) J f(x cos e + y sin e + iz, e)de. (2)
0

Then cf>(x,y,z) satisfies (1), and furthermore every solution of

(1) is obtainable in this way.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 NON-LINEAR INTEGRAL TRANSFORMS 309

Whittaker's procedure can be reformulated in a more geomet-

rical fashion as follows. Let us complexify the Euclidean 3-
space, so that x, y and z become complex coordinates on c3
with complex line-element ds 2 = dx 2 + dy 2 + dz 2 . Let T be the
space of complex planes in c3 which are null (j.e. isotropic
with respect to the complex line-element). It is helpful to
introduce an index notation which makes use of the fact that
SL(2,C) is locally isomorphic to S0(3,C) Namely, let xAB
denote the coordinates on c3, with A, B being two-dimension-
al indices, and xAB being symmetric in AB. The line-element
2 AB CD
is ds = gABgCDdx dx , where gAB = -gBA is the symplectic
form preserved by SL(2,C). It follows that null planes in
are precisely those given by

(3)

where ~,. ~ 2 and Y are three complex constants (with ~, and

2
~ 2 not both zero). Clearly (~~,.~~ 2 .~ Y) determines the same
plane as c~,.~ 2 ,Y) if ~ is any non-zero number, so we see
that the space T is a two-dimensional complex manifold. [In
fact, T is the holomorphic tangent bundle of the Riemann sphere

P1 (C) , with ~,. ~ 2 being the usual homogeneous coordinates on

P1 (C). The space T may also be obtained as the space of

oriented geodesics in see ref. 2.]
There is, therefore, a correspondence between the spaces c3
and T expressed as follows: a point in T corresponds to a
null plarie in c 3 ; and a point in c3 corresponds to a
holomorphic curve in T of the form (3), with xAB fixed.
Whittaker's formula (2) may now be rewritten as

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 310 R. S. WARD

( 4)

where f(Y,nc) is a function holomorphic on some region in T,

having the homogeneity property

(5)

The contour of integration in (4) is a circle on the Riemann

sphere (whose homogeneous coordinates are nA), avoiding what-
ever singularities the integrand f may have. The formula (4)
expresses a correspondence between solutions of ~~ = 0 on
and "free" holomorphic functions on T. To be more precise about
this correspondence requires the language of sheaf cohomology.
The exact statement is that there is an isomorphism between the
two vector spaces

0}

and
W_ 2 = H1 (T,0(-2)),

this latter space w_2 being the first cohomology group of T,

with coefficients in the sheaf of holomorphic func;tions with the
homogeneity property (5) [3]. The map from w_2 to vo is
given by ( 4)' while the inverse map from vo to w_2 is also an
integral transform.
So Whittaker's formula uses a geometric correspondence (be-
tween and T), and an integral transform, to solve ~~ = 0

on Certain other linear systems of equations on c3 can be

dealt with in a similar way. For example, the cohomology group

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 NON-LINEAR INTEGRAL TRANSFORMS 311

where 0 is the sheaf of functions homogeneous of degree zero

(i.e. satisfying (5) with A- 2 deleted), is isomorphic to

where aAB = a/axA 8 . That is, an element of v1 consists of a

scalar field ~ and a 1-form ~. satisfying

curl ~ = grad ~. ( 6)

Note that (6) implies 6~ = 0; it is a generalization of the

Laplace equation.
Actually, the above statement (about w0 and v1 being
isomorphic) is not true as it stands; the correct statement is
wo • V1/-, where is the equivalence relation of "gauge-
equivalence": namely

(~.~) - (~ + grad~.~),

~ being any scalar field on c3.

The nice thing about this example is that it has an inter-
pretation in terms of line bundles and connections. In particu-
lar, note that wo may be thought of as the space of topologi-
cally-trivial holomorphic line-bundles on T. This immediately
suggests a non-linear generalization, namely that of going from
line-bundles to vector-bundles. What one ends up with is a
correspondence between two sets Vn and defined as
1
follows.
w0 n is the set of holomorphic rank-n vector-bundles E

over T, satisfying the condition that the restriction of E to

any curve of the form (3), is trivial. And Vn is a set of
1
pairs ( ~. ~). where is an n x n matrix of scalars on c3

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 312 R. S. WARD

(all holomorphic). One should think of ~ as the coefficients

of a connection on a (trivial) rank-n vector-bundle over
defining a "covariant derivative"

( 7)

The fields ~ and ~ satisfy a non-abelian version of (6),

namely

( 8)

And, as before, there is a gauge-equivalence on Vn

1

(9)

where ~ is now an n x n matrix of scalars. Of course, (9) is

geometrically natural: it is just the transformation induced by
an automorphism of the bundle over c3 mentioned above.
The result is that there is a natural one-to-one correspon-
dence between W0 n and v 1 n /-. i.e. between holomorphic
vector-bundles over T and solutions of non-linear equations (8)
on This correspondence involves non-linear integral trans-
forms. Indeed, to go from Vn to Wn involves the transform
1 0
described in example A of section 1 : to obtain the "patching
matrix" for the bundle E, one has to solve a set of ordinary
differential equations of the form f'(t) = A(t)f(t), where
A(t) is built out of and cp • And to go from Wn to Vn
0 1
involves the transform of example B in section 1: given the
patching matrix G(l,TIA) of the bundle E, one has to "split"
the matrix

AB
F(s) = G(x TIATIB,Tic)\
TIC .. ( s •1 )

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 NON-LINEAR INTEGRAL TRANSFORMS 313

for each fixed xAB in order to obtain the fields ~ and ~.

The details of all this may be found in, for example, [2-4].
The equations (8) are known as the Bogomolny equations, and
are relevant to "magnetic poles" in non-abelian gauge theory.
The "non-linear integral transform" outlined in this section has
proved very useful in constructing and understanding the solu-
tions of these equations.

3. MORE EXAMPLES. [Link] previous section described non-linear

integral transforms between "free functions" on the one hand, and
solutions of a system of non-linear partial differential equa-
tions on the other. Can one find other, similar, examples of
this?
First, the Whittaker formula (2) generalizes to arbitrary
dimension: there is an integral formula like (2) which solves
the Laplace equation in m dimensions, for any rn~ 3. It in-
volves an (m-2)-dimensional integral, and can be interpreted
geometrically as a transform between em and an (m-1)-dimension-
al complex manifold T (which, as before, is the space of com-
plex null hyperplanes in Cm). So the integral transform gener-
alizes to arbitrary dimension. But when one tries to "non-
linearize" this, as was done in section 2 in the case m = 3,
one gets stuck. The reason is basically that there is no obvious
way of "non-linearizing" an integral of dimension greater than
unity. The two examples of section 1 involve line integrals. So
we have to restrict our attention to line integrals, at least
until somebody comes up with a non-linear version of higher-
dimensional integration.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 314 R. S. WARD

Shortly after Whittaker's result became known, H. Bateman

[5] discovered that one could solve the Laplace equation (or wave
equation) in four dimensions by a one-dimensional integral. His
result can be understood in terms of integral transforms between
c4 ) and the complex projective three-space
known (in this context) as "twister space" [6]. These integral
transforms can be non-linearized in various different ways, in
order to obtain solution procedures for various non-linear
partial differential equations in four dimensions. These include
the self-dual gauge-field equations [7] and self-dual Einstein
equations [6].
What about dimensions greater than four? It seems unlikely
(from a consideration of the initial-value problem) that one
could express the solution of the wave equation or Laplace equa-
tion in more than four dimensions, in terms of a one-dimensional
integral. However, there are generalizations, in arbitrarily
large dimensions, of the sort of transforms we have encountered
above. These transforms relate to solutions of sets of partial
differential equations which are either overdetermined (involving
more equations than unknown functions), or whose characteristic
surfaces are cones of degree greater than two (as opposed to the
Laplace equation, Bogomolny equations, etc. where the character-
istics are quadric cones). Some of the details of these higher-
dimensional examples may be found in [8,9].

4. CONCLUSIONS. We have seen several examples where integral

transforms involving line integrals admit non-linear
generalizations. It remains to be seen how far this sort of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 NON-LINEAR INTEGRAL TRANSFORMS 315

thing can be carried, and, in particular, whether transforms

involving higher-dimensional integration can be "non-linearized."

BIBLIOGRAPHY
1. E. T. Whittaker, "On the partial differential equations
of mathematical physics," Math. Ann 57 (1903), 33-355
2. N. J. Hitchin, "Monopoles and geodesics," Commun. Math.
Phys. ~ (1982), 579-602.
3. P. E. Jones, PhD thesis, Oxford, 1984.
4. R. s. Ward, "A Yang-Mills-Higgs monopole of charge 2,"
Commun. Math. Phys. 79 (1981), 317-325.
5. H. Bateman, "The solution of partial differential equa-
tions by means of definite integrals," Proc. Lond. Math. Soc. (2)
l (1904), 451-458.
6. R. Penrose and R. S. Ward, "Twisters for flat and curved
space-time." In General Relativity and Gravitation, Vol. 2, ed.
A. Held, Plenum, New York, 1980, pp.~3-328.
7. M. F. Atiyah, "Geometry of Yang-Mills fields (Scuola
Normale Superiore, Pisa, 1979).
8. R. S. Ward, "Completely-solvable gauge-field equations
in dimension greater than four," Nucl. Phys. 8236 (1984), 381-
396.
9. s. Salamon, "Quaternionic manifolds," Symposia Math. 26
(1982), 139-151.
DEPARTMENT OF MATHEMATICS
DURHAM UNIVERSITY
DURHAM, ENGLAND

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

INTEGRAL GEOMETRY AND TVISTOR THEORY

R. 0. WELLS, JR.

1. INTRODUCTION. Integral geometry arises in many different ways

as the diversity of lectures at this conference shows. One major
development has been that of the Radon transform. This has been
discussed a great deal at this conference and a good recent
survey is given in Helgason [13]. The classical Radon transform
consists of the integration of a function f defined on R3 to
obtain a new function Rf defined on the 4-dimensional space
L(R3) of all lines in R3. This is analogous to the Fourier
transform and has many similar and useful properties. Many of
these were exploited in Fritz John's study of hyperbolic partial
differential equations (see John [18]).
In a series of papers in the late 1960's Roger Penrose
introduced a variant of the Radon transform. If we consider
P3 • P3 (c), 3-dimensional complex projective space and let
L(P 3 ) be the set of complex projective lines in P3 , then
Penrose introduced a transform from P3 to L(P 3 ) (for refer-
ences see the discussion below). One difference between the
transform of Penrose (now called the Penrose transform) and the
classical Radon transform is that the Penrose transform involves
an integration over a complex projective line which is differ-
entiably the same as a 2-sphere. We'll say more about this
integration process below.
© 1987 American Mathematical Society
0271-4132/87 $1.00 + $.25 per page

317
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 318 R. 0. WELLS, JR.

It turned out that the Penrose transform was well-suited to

study various problems in mathematical physics. Just as the
Radon transform of F. John generalized solutions of the ultra-
hyperbolic equation in 1942, Penrose's ideas have yielded solu-
tions of numerous equations of mathematical physics. These
include both linear ~nd nonlinear differential equations and
systems.
The basic observation of Penrose which allows one to relate
integrals over projective lines in P3 to physical problems is
that the space L(P 3 ) is a 4-dimensional complex manifold which
is in a natural manner the complexification of the conformal
compactification of 4-dimensional real Minkowski space. The
space L(P 3 ) is tautologically the same as the space G(2,C 4 ),
the Grassmanian manifold of 2-planes in c4 , which is a four-
dimensional complex manifold. We let M4 denote 4-dimensional
Minkowski space (R 4 equipped with a Lorentz metric of signature
(+ - - -)). The conformal compactification of M4 which is
denoted by M is a useful theoretical tool in theoretical
physics (see Penrose-Rindler [28]). Mathematically this was
first described in Kuiper [20] and is similar to the classical
compactification of the complex plane given by stereographic
projection.
The geometry of this space of lines was studied first by
Grassmann, as the name Grassmannian manifold indicates. This line
space was latei studied by Klein who discovered a natural con-
formal structure on it which was induced from its Plucker embed-
ding into P5 . If we let T be the four-dimensional space of
homogeneous coordinates for P5 and let T be equipped with an
Hermitian quadratic form of signature (+ + - -) then T is a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 319

representation space for SU(2,2) which is a 4-1 covering group

of the conformal extension of the Lorentz group. This is an
example of a space of twistors, a representation space for
SU(2,2) just as spinors form a representation space for
SL(2,C).
Thus, in analogy with the Radon transform, we can consider
functions (or suitable generalizations thereof as we shall see)
defined on some portion of P3 • These can then be integrated
over the "lines" in P3 to yield a new furiction defined on some
portion of L(P 3 ) which can in turn be restricted to M4 yield-
ing functions or, as the physi~ists say, fields on M4 • The
notion of a field is a generalization of a vector field and is
simply a section of some appropriate bundle and hence is locally
a vector-valued function.
If we consider a rational function of degree -2 (in terms
of the homogeneous coordinates of P3 ), then we can define

(1)

where the variable ~ is a local parameter for the projective

line Lx where x 6 L(P 3 ) and the integration denotes a contour
integral around the poles of f restricted to the projective
line Lx (recalling that Lx is a copy of the Riemann sphere).
This contour integral is analogous to the line integral of the
Radon transform

Rf(x) = J fdt
L
X

where t is a parameter on LX for X 6 L(R 3 ). The basic

difference is that in the Penrose transform we have a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 320 R. 0. WELLS, JR.

contribution from the residue defined by the function f.

The fundamental and striking fact first discovered by
Bateman in 1904 and rediscovered by Penrose in 1968 is that $(x)
must satisfy the 4-dimensional wave equation when restricted to
M4 , 1.
. e.,

where (X ,
0 X
1 2
X ,
3
X ) are coordinates in Minkowski space.
0

This is also related to a two-dimensional integral representation

of Whittaker [41], cf. Whittaker-Watson [42]) somewhat earlier,
which relates to an integral over the whole two sphere, instead
of a residue integral.
This construction was generalized by Penrose to obtain
vector-valued and spinor-valued fields satisfying generalizations
of the wave equation. Namely if f is homogeneous of degree
-2s -2 then a suitable variation of (1) yields a solution of the
massless Dirac equations of spin s (see Penrose [29], Penrose-
MacCallum [31], Wells [38], Eastwood-Penrose-Wells [9]). These
equations include for s = 1/2 the Dirac-Weyl equation of a
neutrino, for s = 1 Maxwell's equations of electrodynamics, and
for s = 2 the linearization of Einstein's field equations of
general relativity. It is quite striking to see solutions of
these a priori distinct equations of theoretical physics be
generated in an automatic manner by calculating residues of
rational functions of three complex variables restricted to com-
plex lines where the degree of the rational function determines
which equation is satisfied. It turns out that two different
rational functions could yield the same field and it was

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 321

discovered by Atiyah and Penrose that the ambiguity could be

understood if one considered the rational functions as being ~ech
cocycles for cohomology classes with coefficients in the sheaf of
locally homogeneous functions of a specific degree m (the sheaf
of holomorphic sections of the m-th power of the hyperplane
section bundle, a generalization of rational functions of a fixed
degree which is denoted by O(m)).
The fact that the linearization of Einstein's equations has
a twistor representation is a precursor to the profound discovery
of Penrose (Penrose [30]) that a nonlinear variation of this
method would generate solution·s of Einstein's nonlinear field
equations. This nonlinear graviton construction, as it was
called by Penrose, was modified in a natural manner to yield a
twistor representative of the Yang-Mills equations. These equa-
tions are significant nonlinear generalizations of Maxwell's
equations, which, at the pertubation level of quantum field
theory, yield excellent models for elementary particles.
These two themes of solutions of Yang-Mills equations and of
solutions of the Einstein equations have dominated the work in
twistor geometry of the past few years. I'll mention in partic-
ular the work on magnetic monopoles by Ward [36], Nahm [24],
Hitchin [16], and Donaldson [6]. Quite recently Michael Atiyah
announced some new work with some of his collaborators on the
dynamics of monopoles. The previous works mentioned above were
concerned primarily with the study of static monopoles. Th~ new
work described by Atiyah (in a lecture in Leningrad, October
1984) used the moduli space description of the set of all static
monoples as a fundamental tool in getting a first order approxi-
mation to the dynamics of monopoles.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 322 R. 0. WELLS, JR.

In a different direction there has been work by various

authors on the extension of the twistor representation of the
solutions of the massless field equations to the case of seneral-
ized solutions of these same equations. The integral geometry
ideas can be extended to the setting of distributions and hyper-
functions, generating weak solutions of these equations of hyper-
bolic type in real Minkowski space and its compactification. The
first hints of the twistor representation of solutions to weak
solutions of the massless field equation was in the earliest
papers of Penrose on twistor theory. These were incorporated
into the first general results concerning hyperfunction solutions
to the massless field equations on compactified Minkowski space
given in Wells [37], [39]. These were then improved in the paper
of Bailey-Ehrenpreis-Wells [1], in which a representation of
hyperfunctions on affine Minkowski space was presented. These
results were in turn reinterpreted and improved upon, using the
viewpoint of representation theory in the recent work of Dunne
who is reporting on his work elsewhere in this volume (Dunne
[7]).

The last topic I'd like to mention is the extension of

twistor geometry to supermanifolds. Supermanifolds were intro-
duced in physics in the context of supersymmetry and super-
gravity. A number of mathematicians have made great efforts to
provide good mathematical theories which incorporated the physi-
cists' fundamental desire to have spaces with two types of local
coordinates and the same time: commuting and anticommuting.
These relationships of the independent variables are to mimic the
commutative and aniicommutative relations of the normal field
theories defined on space-time. The objective is to have a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 323

unified geometric basis for understanding fermionic and bosonic

fields. The usual matter fields (electrons, protons, etc.) are
fermionic, whereas that of gravity is bosonic. What is desired
is a uniform setting for both types of theories, and this is the
basic genesis for supersymmetry and supergravity (see e.g. Wess-
Bagger [40], for a discussion of this from a physicist's point of
view). From the viewpoint of a mathematician, there have been
various theories developed recently. One viewpoint is that which
goes back to Berezin, Leites, and Kostant, and which has been
developed further by a number of researchers (see the books by
Berezin [3], Leites [21], and Manin [22] for further references
to the subject. The viewpoint taken by these authors is that a
supermanifold is a ringed space, i.e. a usual manifold with a
structure sheaf which is more general than a subsheaf of the ring
of continuous complex-valued functions. The prototypes of such
ringed spaces arise in algebraic geometry and several complex
variables in which case one deals with ringed spaces which are
infinitesimal neighborhoods or are the structure sheaf of spaces
with singularities. In both cases, one might have nilpotent
elements in the structure sheaf of rings. The structure sheaf of
a supermanifold will have the form of a sheaf of Grassmann-
algebra valued functions. The local generators of the Grassmann
algebra become the "anticommuting coordinates" of the geomet~y.

In particular one can have power series expansions of the fields

defined on the manifold in terms of the local coordinates of the
underlying topological manifold, as well as with respect to the
ring structure and "anticommuting coordinates" of the stalks of
the structure sheaf.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 324 R. 0. WELLS, JR.

Witten noted in his ve~y seminal pape~ [43] that the twistor
correspondence could be extended to the supermanifold setting and
gave some hint of applications of these ideas. The recent book
by Manin [23] gives a very insightful description of this combi-
nation of two very pretty geometric ideas: twister geometry and
supergeometry. We will say more about this development below.
There is an alternative approach to supermanifold, initiated
by Rogers [32], and developed further in the papers of Boyer-
Gitler [4], Volovic [34, 35], and Rothstein [33], among others.
The basic app~oach of these authors is to have a manifold in the
classical sense which incorporates the information contained in
the total space of the sheaves used in the Berezin-Kostant-Leites
approach. Roughly speaking, the total space of the G~assmann-

valued sheaves is equipped with a topology of a manifold, instead

of the usual sheaf topology (etale' space topology over the base
manifold). One has local coo~dinates which are Cartesian
products of the even and odd parts of a Grassmann algebra, and
one defines superdifferentiable functions to be differentiable
with respect to these vector-valued coordinates. This is com-
pletely analogous to being differentiable with respect to complex
coordinates giving rise to the notion of holomorphic. The super-
geometry is defined in te~ms of the superdiffe~entiable func-
tions, just as complex geomet~y is defined with respect to the
~ing of holomorphic functions. There are several variations on
this theme (see the papers cited above), but they all have the
flavor of a classical manifold with a specific pseudogroup struc-
ture in the classical sense of E. Cartan. There are subtle
~elations between these two approaches to supergeometry, and
there has been considerably more development of the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 325

sheaf-theo~etic app~oach. It's not clea~ at this time which will

be of more use fo~ unde~standing the va~ious supe~symmet~ic

models of theoretical physics. On the othe~ hand, it needs to be

said that the supe~symmet~ic field theo~ies of the physicists are
still awaiting any kind of expe~imental justification. The~e a~e

many aspects of the theo~y which a~e quite appealing, hence the
la~ge inte~est in the subject by both physicists and mathemati-
cians. We will t~y to give some indication of one aspect of
supe~geomet~y late~ in this pape~.

2. INTEGRAL GEOMETY. Integ~al geomet~y usually involves integ~a-

tion ove~ some subsets of one space to give a function on the

space of these subsets. This is most often accomplished by means
of (eithe~ explicitly o~ implicitly) a double fib~ation

X <-\l- Z _v_> Y.

The data is defined on, say, the space X, and pulled back by ll

to Z, and then integ~ated ove~ the fibe~s of v to give a

function on Y. That is, if f is defined on X, then define
the integ~al-geomet~ic t~ansfo~m of f

f -> <P ( X ) = I v -1 (x) jl *f

The fact that the pull-back function ll*f is constant on the

fibe~s of the mapping ll will imply, in gene~al, that the t~ans-

fo~med function will satisfy some diffe~ential equation on the

space Y. The~e a~e gene~al conditions on the double-fib~ation

so that the~e is a gene~ic t~ansve~sality between the two fib~a-

tions (see the a~ticle by V. Guillemin in these p~oceedings which

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 326 R. 0. WELLS, JR.

elaborates this point (Guillemin [11]), and this is what is

needed for a good integral-geometric transform. In the case of
twistor geometry, where there is a family of such double fibra-
tions, these conditions always hold. We will now describe these
double fibrations in more detail.
Suppose that r4 is a four-dimensional twistor space, i.e.,
r4 is a four-complex-dimensional vector space equipped with an
Hermitian form~ of signature (+ + - -). Consider the flag
manifolds

of subspaces

with

If one flag manifold is defined with respect to a set of indices

which is a subset of the set of indices for a second flag mani-
fold, then the second flag manifold is always fibred over the
first, as there is a natural mapping which ignores the subspaces
corresponding to the deleted indices. For instance we have the
natural double fibration where the mappings are of this sort:

If we consider the subspaces where ~ is positive definite,

negative definite, or zero, we obtain diagrams of the sort

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 327

where a = +, -, or 0. The domains F(2,T 4 )± are tube domains

with distinguished boundary F(2,T 4 ) 0 which is compactified
Minkowski space. The boundary values of holomorphic fields de-
fined in F(2,T 4 )± on the distinguished boundary gives rise to
the hyperfunction fields alluded to above. They are represented
in terms of this double fibration in terms of hyperfunction data
on the twistor manifold F(1,T 4 ) 0 (a five-dimensional differ-
entiable manifold diffeomorphic to s2 x s3). In the papers of
Mantini [23] and Dunne [7], these twistor manifolds and their
associated Penrose transforms will be discussed in more detail.
As mentioned above Dunne develops a representation-theoretic
point of view with respect to these tube domains, while Mantini
tackles the specific problem of unitary representations in this
type of context, and develops an L2 -cohomology theory on these
spaces in a very suitable manner. The paper by Ward in these
proceedings considers different variants of higher dimensional
versions of the Penrose transform.
The basic flag diagram which is important for nonselfdual
problems is

We note that there is an embedding

and the formal neighborhoods of this embedded submanifold play a

role in the representation of (not necessarily self-dual) Yang-
Mills fields on F(2;T 4 ) (see Isenberg-Yasskin-Green [17],
Witten [43], Henkin-Manin [15], Eastwood [8], Eastwood-Pool-Wells
[10], Henkin [14], and Buchdahl [5]. We'll leave the details to

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 328 R. 0. WELLS, JR.

the papers referenced above, but note that it is the geometry

which was generalized to the supergeometric setting in Witten's
paper, and which was then amplified in great detail in the new
book by Manin. In the next section we will give a brief descrip-
tion of the supergeometric twistor theory.

3. SUPERGEOMETRY. The adjective "super" in supermanifold, super-

symmetry, etc. is equivalent to nz 2 -graded," and it often seems
that the latter designation might be more appropriate. However,
both the mathematical and physical literature has been quite
consistent in using the adjective "super" to desc~ibe the phe-
nomena in question. We will give a brief overview of this type
. of geometry and indicate how it relates to twistor and integral
geometry.
Let A = A0 00 A1 be a z2 -graded ring. We will say that an
element is even if it belongs to A0 and odd if it belongs to
A1 , and it is homogeneous if it is either even or odd. We
define fo~ a homogeneous element a of A o(a) degree (a) = 0
if a is even and = 1 if a is odd. The ring A is super-
commutative if
ab = (-1 )o(a)o(b)ba,

for all homogeneous elements a and b of A. An A-module T

is a z2-graded bimodule which satisfies

at = (-1 )o(t)o(a)ta,

for a 6 A, t 6 T. If T • T0 00 T1 , then we are using the same

conventions for degree, etc., as in the case of a graded ring.
There is a natural mapping of A-modules, the parity change
functor

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 329

IT : {A-modules} ---> {A-modules}

defined by setting

(ITT) O : = T1
( ITT ) 1 : = TO •

Note that A is an A-module itself, and, as such, llA is a new

A-module (but no longer a ring!).
An example of such a ring A is a Grassmann algebra

A A*(V) 2 AP(V)
p
for some vector space v, and

A = 2 AP(V) (9 2 AP(V),
p even p odd
A0 <9 A1

A free module of rank plq over A is of the form

T = Ap <9 (llA)q.

An example would be to let A = C, the commutative ring of

complex numbers. Note that a commutative ring with no odd
elements is an example of a supercommutative ring. Then define

which we call super complex Euclidean space of dimension ££ rank

min.
A complex superspace is of the form

(M,O) = (M,0 0 al 01 )

where 0 is a 'supercommutative sheaf of rings, and (M,0 0 ) is a

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 330 R. 0. WELLS, JR.

complex-analytic space (not necessarily reduced), and 01 is a

coherent sheaf of o0 -modules.
If we define 1 = o, + o,2 ' then GrM is a graded manifold
with structure sheaf

Gr 0 (i) Gr. 0
i>O 1

Let Mrd = (M, Gr 0 0). We say that the superspace M is smooth

if

a) Mrd is reduced and smooth,

b) Gr 0 = S( Gr 1 0) - the symmetric algebra of Gr 1 0 over the
ring Gr 0 0.
c) Gr 1 0 is locally free over Mrd'
d) OM is locally a grassmann algebra over the usual algebra
of holomorphic functions on M.

These definitions are similar to the algebraic geometer's notion

of a ringed space with nilpotent elements (see e.g., Harshorne
[12]), which are useful for studying both singularities and
formal neighborhoods of submanifolds.
Given ·a super Euclidean space, e.g.,

then this is a superspace in the above sense of ringed spaces,

and it has superrank (which is the same as its superdimension)
41N. The superdimension is defined as the superrank of a local
coordinate system, and this super Euclidean space is an example
of such a coordinate system. The reduction of this space is
given by

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 331

(the usual 4-dimensional complex Euclidean space). One can

consider a Grassmannian manifold of the form

where p < 4, q < N. One finds that one can cover this
Grassmannian manifold with suitable coordinate charts which are
isomorphic to copies of cP< 4 -p)!q(N-q). Thus we would. say that
this superspace is a supermanifold of superdimension
p(4-p) I q(N-q).
We can now consider such a super twistor space, i.e., a
superspace whose reduction is isomorphic to c4 • Consider such a
space of the form where the superscripts denote the
superrank of this super Euclideqn space. It might have the
additional structure of a suitable bilinear form, for instance,
but we won't get into that in this paper. We then see that its
reduction will be a twistor space in the usual sense, i.e.,

We can ask for all double fibrations of superspaces of the form

L <-- f --> M

which reduce to a double fibration of complex manifolds

which is isomorphic to the double fibration of twistor manifolds

We call such a triple (1, R, M) above an admissible triple if

it has this type of a reduction. Manin has a classification of

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 332 R. 0. WELLS, JR.

such triples (Manin [22]). They form the basis for supergeo-
metric twister theory. The inspiration for Manin's work stems
from the original paper of Witten [ ] discussed earlier. One
defines super flag manifolds, just as in the usual complex geome-
try (of. the discussion of the Grassmannian manifold above)

This is the set of flags

s1 c ••• c sr'

where Si is a free submodule of T4 1N of superrank Pilqi. We

can now give several examples of such super flag manifolds which
has reduction to

Mrd - G(2IO;T 4 1°) • G2,4(C):

(a) F(2I0,2IN;T 41N ), dim 414N,

(b) F(2I0,4IO;T 41N ), dim 414N,
(c) F(O!N,2IN;T 41N ), dim 412N,
(d) F(2IO,;T 41N ), dim 412N,
(e) F(2IN,;T 41N ), dim 414N,

There are additions to this list obtained by using the parity

change functor as well as the isotropic submanifolds of certain
bilinear forms, which we won't go into here. One can see the
pattern above; if we ignore the "odd part" of the superdimension
under consideration, the subspace being described all agree with
the Grassmannian G(2I0; 4 1°).
Manin finds a complete list of all admissible triples.
We'll write down one of them to give the flavor of the situation.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 333

Manin designates this diagram by AM and remarks that this and

its curved version corresponds to the most important supersym-
metry and supergravity models of physicists. The case covered by
Witten [43] is A3 , and in this context as well as in others one
can study solutions of the Yang-Mills equations on M4 14M (super
Minkowski space) by considering holomorphic vector bundles
("super version") on the space L5 1 2 N (the space of super null
lines in super Minkowsi space). We'll leave any further details
to Manin's book and the other papers on the subject, as we only
wanted to sketch a few highlights here to give some flavor of the
geometry involved. The double fibrations are used in exactly the
same manner as in the usual twistor theory. The fundamental
issue is the appropriate vanishing theorem (a la Kodaira or Bott)
along the fibers of the double fibration. This is what allows
one to "integrate along the fibers" in this context, but specif-
ically one uses the Leray spectral sequence in this general
context, just as in the more classical case.
There remains a lot of work to be done to relate the super-
symmetric quantum field theory (especially the perturbation
aspects) on super Minkowski space M4 14N to the geometry of
bundles on the corresponding space of super null lines L5 12 N.
One major task is to try to understand basic differential geome-
try and Hodge theory in the supergeometric setting. There is
some progress here (e.g. Serre duality, Penkov [25], and a super

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 334 R. 0. WELLS, JR.

version of the Bott-Borel-Weil theorem (Penkov-Skornyakov [27],

Penkov [26]). But the classical notion of harmonic forms and
·solving partial differential equations on supermanifolds are
still in their infancy.

BIBLIOGRAPHY

1. T. Bailey, L. Ehrenpreis, R. o. Wells, Jr., "Weak solu-

tions of the massless filed equations," Proc. R. Soc. Lond., A384
(1982), 403-425. --
2. H. Bateman, "The solution of partial differential
equations by means of definite integrals," Proc. Lond. Math. Soc.
(2) l (1904), 451-458.
3. F. A. Berezin, Introduction to Analysis and Algebra with
Anticommuting Variables, (Russian), Pulications of Moscow Univ.,
Moscow (1983).
4. c. P. Boyer, s. Gitler, "The theory of G-infinity-super-
manifolds," Trans. Amer. Math. Soc., 285 (1984), 241-267.
5. N. Buchdahl, "Analysis on analytic space and non-self-
dual Yang-Mills fields," Trans. Amer. Math. Soc. (To appear).
6. s. K. Donaldson, "Instantons and geometric invariant
theory," Comm. Math. Phys., 2J. (1984), 453-460.
7. E. G. Dunne, "Hyperfunctions in representation theory
and mathematical physics," Contemporary Mathematics, Integral
Geometry (Proceedings of Bowdoin College Conference, Amer. Math.
Soc. Providence, R. I. (1984).
8. M. G. Eastwood, "On the twister description of massive
fields," Hoc. R. Soc. Lond.,! 374 (1981), 431-445.
9. M. G. Eastwood R. Penrose, and R. o. Wells, Jr., "Coho-
mology theory and massless fields, Twister Theory and Its Appli-
cations (in Russian), (Ed. V. s. Vladimirov), MIR, Moscow (1983),
250-308.
10. M. G. Eastwood, R. Pool, and R. 0. Wells, Jr., "The
inverse Penrose transform of as solution to the Maxwell-Dirac-
Weyl field equations," J. Funct. Anal., (in print).
11. v. Guillemin, "Perspectives in Integral Geometry," Con-
temporary Mathematics, Integral Geometry (Proceedings of Bowdoin
College Conference, Amer. Math. Soc., Providence, R.I. (1984).
12. R. Hartshorne, Algebraic Geometry, Springer-Verlag,
Berlin-Heidelberg-New York (1977).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 INTEGRAL GEOMETRY AND TWISTOR THEORY 335

13. S. Helgason, The Radon Transform, Birkhauser, Boston

(1980).
14. G. M. Henkin, "Tangential Cauchy-Riemann equations and
Yang-Mills-Higgs-Dirac fields, Proceedings International
Congress, Warsaw (1982), (to appear).
15. G. M. Henkin, Y. I. Manin, "Twistor description of
classical Yang-Mills-Dirac fields," Phys. Lett., 95B (1980), 405-
408. -
16. N. J. Hitchin, "On the construction of monopoles." Comm.
Math. Phys., 89 (1983), 145-190.
17. J. Isenberg, P. B. Yasskin, and P. s. Green, "Non-self-
dual gauge·fields," Phys. Lett., 78B (1978), 462-464.
18. F. John, Plane Waves and Slherical Means, Springer-
Verlag, ·New York-Heidelberg-Berlin 1981).
19. B. Kostant, "Graded m~nifolds, graded Lie algebras, and
prequantization," Lecture Notes in Mathematics, 570, Differ-
ential-Geometric Methods in Physics, Springer-Verlag, Berlin-
Heidelberg-New York (1977T; 177-300.
20. N. Kuiper, "On conformally flat spaces in the large,"
Ann. of Math., 50 (1949), 916-924.
21. D. A. Leites, Theory of Supermanifolds, (Russian),
Karelskii Affiliated of Acad. Sciences USSR, Petrozavodsk (1983).
22. Y. I. Manin, Gauge Fields and Complex Geometry, Nauka,
Moscow (1984).
23. L. Mantini, "An L2-cohomology analogue of the Penrose
transform for the oscillator representation," Contemporary
Mathematics, Integral Geometry, (Proceedings Conference Bowdoin
College), Amer. Math. Soc., Providence, R.I. (1984).
24. W. Nahm, "All self-dual multimonopoles for arbitrary
gauge groups, CERN report, TH 3172 (1981).
25. I. B. Penkov, "D-modules on supermanifolds," Invent.
Math. 11 (1983), 501-512.
26. I. B. Penkov, "An introduction to geometric representa-
tion theory of complex simple Lie superalgebras, (Preprint,
Sofia, 1985).
27. I. B. Penkov, "Cohomologie des D-modules tordues
typiques sur les supervarietesdes drapeaux, (Preprint, Sofia,
1984).
28. R. Penrose and W. Rindler, Spinors and Space-time,
Vol. I, Cambridge Univ. Press, Cambridge (1984).
29. R. Penrose, "So~utions of the zero-rest-mass equations,"
J. Math. Phys., .lQ (1969), 38-39.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 336 R. 0. WELLS, JR.

30. R. Penrose, "Nonlinear gravitons and curved twistor

theory," General Rel. and Grav., 1 (1976), 31-52.
31. R. Penrose and M. A. H. MacCallum, "Twistor theory: an
approach to the quantizition of fields and space-time," Phys.
Reps., 6C (1972), 241-316.
32. A. Rogers, "A global theory of supermanifolds," J. Math.
Phys., l l (1980) (Preprint), 1352-1365.
33. M. Rothstein, "On the resolution of some difficulties in
the theory of G-infinity manifolds, (Preprint, Univ. of
Washington, Seattle) (1984).
34. I. V. Volovich, "1\-supermanifolds and bundles,
(Preprint) (1984).
35. I. V. Volovich, "Super-self-duality for supersymmetric
Yant-Mills theory," (Preprint) (1984).
36. R. s. Ward, "A Yang-Mills-Higgs monopole cif charge 2,"
Comm. Math. Phys., 79 (1981), 317-325.
37. R. 0. Wells, Jr., "Complex manifolds and mathematical
physics;" Bull. Amer. Math. Soc. (NS), 1 (1979a), 296-336.
38. R. 0. Wells, Jr., "Cohomology and the Penrose trans-
form," Complex Manifold Techniques.!.!!. Theoretical Physics, (Eds.:
P. Lerner and P. Sommers), Pitman, London (1979b), 92-114.
39. R. 0. Wells, Jr., "Hyperfunctions solutions of the zero-
rest-mass field equations," Comm. Math. Phys., 78 (1981), 567-
600. --
40. J. Wess and J. Bagger, Supersymmetry and Supergravity,
Princeton Univ. Press, Princeton, N. J. (1983).
41. E. T. Whittaker, "On the partial differential equations
of mathematical physics," Math. Ann., 21. (1902), 333-355.
42. E. T. Whittaker and G. N. Watson A Course of Modern
Analysis, 4th Edition, Cambridge Univ. Press, Cambridge, N. J.
(1902).
43. F. Witten, "An interpretation of classical Yang-Mills
Theory," Phys. Lett. 788, No. 4 (1978), 394-398.
DEPARTMENT OF MATHEMATICS
RICE UNIVERSITY
HOUSTON, TEXAS 77251

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 [Link]
Contemporary Mathematics
Volume 63, 1987

SOME INVERSE PROBLEMS OF POTENTIAL THEORY

LAWRENCE ZALCMAN 1

The first (integral) part of this paper deals with certain

aspects (primarily questions of uniqueness) of the inverse prob-
lem of potential theory. In the second (geometry) part of the
paper, we mention several attractive open questions concerning
the characterization of circles and spheres. Throughout, the
emphasis is on what is not known: on unsolved problems.

1. NEWTON'S THEOREM. According to Newton's theorem, the gravita-

tional attraction exerted by a solid homogeneous ball on any
point outside it is identical to that exerted by a point mass (of
mass equal to the volume of the ball) placed at the center of the
ball. This result, which plays an important role in the Princi-
pia Mathematica, has a curious historical significance. While
Newton had most of the results in the Principia by 1664, he began
to publish his work only in 1684. It has been suggested that the
long delay was a result of Newton's inability to find a satis-
factory proof for the theorem stated above. Today, of course, it
is not difficult to give a proof using calculus. If one allows
function theory, it becomes irresistibly easy.

1 Preparation of this paper was supported in part by NSF Grant

MCS 8200902.

© 1987 American Mathematical Society

0271-4132/87 $1.00 + $.25 per page

337
Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 338 LAWRENCE ZALCMAN

Indeed, a unit mass placed at the point x attracts the

point y(! x) with a force whose magnitude is --~ 1 ~2 and whose
direction is jx-yj'
~
x-yl
Thus, the gravitational f eld at ! y in-
duced by a point mass at x-y It follows in
x is given by
lx-yl3'
routine fashion that the field induced by a homogeneous ball B
of unit density is given byJ x-y dx. We may assume that B
B lx-yl3 1
is centered at 0. Observe that for y 0 B the function ~
is harmonic on B. Hence, by the mean value theorem for harmonic
functions,

J dx y e B,
v1 B -yx::yr =
1
TYT

where V is the volume of B. Multiplying by V and taking the

gradient of both sides yields

ye B,

which is simply the assertion of Newton's theorem.

2. A CONVERSE. Let P be a solid, homogeneous body in R3

which induces on its exterior a gravitational field identical to
that of a point mass. Must P be a ball? The hypothesis that
P is solid is taken to mean that R3\P is connected and that P
is the closure of its interior. Unless otherwise stated, we
shall always assume that our bodies have unit density.
We may assume the point mass is at 0. By assumption,

J x-y
dx =
-y
c(---.;) ye P. (1)
P lx-yl 3 IYI~

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 339

Integrating, we obtain

ye P,

where we have used the fact that R3\P has only one component.
Letting y ~ m shows that d = 0; thus

c y t P. (2)
TYT

Since the left hand side of (2) defines a bounded c1 function

of y on all of it follows that 0 must lie in the
interior of P. Taking the inner product of each member of (1)
with y and adding the result to equation (2) yields

0 ye P.

By Newton's theorem, the same equation must also hold for B,

the largest ball about 0 contained in P. Subtracting, we
obtain

y e P. (3)

This holds even for y 6 aP, since the left hand side is con-
tinuous as a function of y. Now choose y 6 aP n as and ob-
serve that the integrand in (3) is then positive. It follows
that P c B, whence P = B as required.
It seems remarkable that this result was obtained only
within the past decade. It was first proved (under an additional
hypothesis of boundary smoothness) in 1976, by A. V. Kondra~kov,

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 340 LAWRENCEZALCMAN

in a paper so obscure that it seems to have escaped the attention

of even Mathematical Reviews. Later it was rediscovered, and
given several different proofs, by Aharonov, Schiffer, and my-
self. The proofs given in these papers yield considerably more
than the converse to Newton's theorem. For instance, Kondraskov
shows that a homogeneous solid with regular boundary whose ex-
ternal field agrees with that of an ellipsoid (of the same den-
sity) must coincide with the ellipsoid; he also deals with the
case of radial mass densities on the ball. Aharonov, Schiffer
and I were also able to handle radial densities, as well as
analogues of the converse to Newton's theorem for bodies with
cavities and collections of solids. The extremely elementary
proof given above is due to Don Zagier.

3. A POTENTIAL PROBLEM. Newton's theorem and its converse show

that if the potential

of a homogeneous solid P coincides on its exterior with that of

a solid ball B of the same density, then P = B. The assump-
tion that is known everywhere off P is too restrictive --
if R3\P is known, P is known -- and quite unnecessary. In-
deed, since up is harmonic off P, it is uniquely determined
on all of R3\P by its values on any open subset. Thus, it
suffices to assume that the potentials of P and B agree near
infinity.
This suggests the following problem. Suppose P1 and P2
are homogeneous solids whose corresponding potentials agree near

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 341

infinity. Must P 1 - P2 ? The hypothesis that the potentials

agree near infinity implies, via harmonic continuation, that they
agree throughout 0, the unbounded component of R3\(P 1 u P2 ).
This, in turn, is equivalent to the requirement that

J v(x)dx - J v(x)dx (4)

p1 p2

for all functions v harmonic on (a neighborhood of) Q = R3\n.

For v(x) = rx=Yf
1 (y 6 0), (4) is just the assertion that the
potentials agree on 0. In general, modifying v to be smooth
and of compact support but still harmonic on Q, we have

J
p1
v(x)dx fP (- 1fif
1 J ~v(ll
jy-xj
dy)dx
1

k Jo cJP 1 ~)~v(y)dy
I"' Y1

k Jo cJP ~~:yj)~v(y)dy
2

(- 1
1fif
JP ~ v ( Y)_) dy) dx
ly-xj
2
v(x)dx.

The relation (4) shows at once that P1 and P2 must have

equal volumes (take v s 1) and nonempty intersection (otherwise
take v • 1 on P1 , v s 0 on P2 ). It is tempting to surmise
that (4) actually implies P1 = P2 . Unfortunately, nothing of
the sort is the case.

4. POTENTIAL EQUALITY VERSUS ACTUAL INEQUALITY. It will be

convenient to discuss the relation (4) first for subsets of the

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 342 LAWRENCE ZALCMAN

plane. Already in the geophysical literature of a quarter-

century ago, it was observed that there exist distinct configura-
tions in R2 , neither of which divides the plane, which satisfy
( 4). Indeed, for j = 1• 2 let be a closed disc and Aj a
closed annulus concentric with and disjoint from 8j and having
the same area as 8j. If 81 lies in the unbounded component of
lies in the unbounded component of and
A1 n A2 consists of two components, then
P1 - (A 1 u 8 2 )\(A 1 n A2 ) is the union of three disjoint closed
Jordan regions, as is P2 = (A 2 u 8 1 )\(A 1 n A2 ). The mean value
theorem for harmonic functions shows that

Jr u dx = Jr u dx
A1 u 8 2 A2 u 8 1

for any function harmonic on the complement of the unbounded

component of It is now clear that (4) holds for
P1 and P2 •
A modification of this construction yields Jordan regions

p 1 • p2 which satisfy ( 4). Let 8j and Aj be as before

(81 n 82 = cp) but assume now that 81 u 82 lies in a single

component T of R2 \(A 1 u A2 ) and that A1 n A2 consists of
two components c, • c2. One may then choose a subset s of
T\(8 1 u B2 ) in suc.h a way that P1 - (A 1 u 82 u S)\C 1 and
P2 = (A 2 u 81 u S)\c 1 are both Jordan regions. (The choice of
S is not unique and can be effected in many different ways.) As
before, it follows that (4) obtains for P1 and P2 •
A condensation of singularities argument applied to the
previous construction yields an uncountable collection of Jordan
regions, all lying in a bounded subset of the plane, which give
rise to the same potential near infinity. Suppose we have

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 343

constructed p1 and p2 as above. We may assume that the line

joining the. centers of B1 and B2 is parallel to the x-axis
and that the deleted component c, of A1 n A2 lies above this
line. Attach to P1 and P2 an almost rectangular "stem" W
which abuts A1 u A2 near c2 in such a way that Pj - Pj u w
( j • 1, 2) is again a Jordan region. (The stem W may be taken
to have its base on the x-axis and two sides parallel to the y-
axis; the remainder of its boundary consists of two circular arcs
forming part of the outer boundary of A1 u A2 . Each set Pj
thus resembles somewhat the profile of a fruit tree partially
consumed by a plague of locusts.)
Let R • [0,1] n • 1,2,3, ... construct
X [-1,0]. For
pj
(n) (j .
1 • 2) as above so that -p1 (n) u p2
(n) is contained

in {(x,y) : 0 ~ y ~ x, 2-n < X < 2 1-n} and R u P. ( n) is

J
connected. To each function f from the positive integers
to the set {1,2} we may associate the Jordan region

Pf • R u u.
n=l
-(n)
Pf(n)"

Since {1,2}N is uncountable, there are uncountably many such

domains; and it is clear that if u is harmonic on the set
{(x,y) -1 S y S X, 0 S X S 1} then r
Jp
u dx does not depend
f
on f. In particular, all Pf have the same potential near
infinity.
The adaptation of the construction given above to R3
presents no special difficulties. The Bj are now balls and the
Aj spherical shells. The only point to notice is that A1 n A2
now consists of a single component whose removal disconnects Aj.
Accordingly, we delete only that part c of A1 n A2 lying on

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 344 LAWRENCEZALCMAN

one side of some fixed plane through the centers of B1 and s2 •

The remainder of the construction proceeds as before.
That nonuniqueness can occur in configurations without cavi-
ties seems first to have been observed by the mathematical geo-
physicist Celmi~s in 1957. The first published example of two
Jordan regions for which (4) holds was given by Makoto Sakai in
1978; I have been told that a similar example had earlier been
constructed (though not published) by P. s. Novikov. Quite a
different sort of nonuniqueness, for domains convex in a particu-
lar direction but having variable density, has been exhibited by
Victor Isakov. The idea of replicating the basic construction in
order to increase the degree of nonuniqueness is due to Carl
FitzGerald.

5. POLYDROMY. Nonuniqueness of the sort exhibited above is

closely though obscurely related to the polydromy of the poten-
tial up. More specifically, suppose the potentials u = u
p2
(say) near infinity (and hence on the unbounded component n of
the complement of P1 u P2 ). It is clear that u continues
analytically to a single-valued harmonic function on the comple-
ment of Pj (j = 1 ,2); however, these continuations do not neces-
sarily agree throughout their overlap. It remains an open ques-
tion of considerable interest whether nonuniqueness can occur in
the absence of the polydromy of the associated potential.

QUESTION. Suppose P1 and P2 are homogeneous solids whose

potentials agree on all of R3 \(P 1 u P2 ). Must P1 = P2 ? If
not, can one find distinct homogeneous solids P 1 , P2 , the
complement of whose union is connected, whose potentials agree on
R3\(P 1 u P2 )? In the Soviet Union, the first of these questions

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 345

is taken as the formulation of the (exterior) inverse problem of

potential theory.
Questions involving the analytic continuation and polydromy
of Newtonian potentials were studied half a century ago by R.
Wavre. Among more recent work in this area, we may mention that
of Strakhov, Zhdanov, and Smith.

6. POSITIVE RESULTS. The examples of Section 4 demonstrate

conclusively the necessity of placing additional constraints on
one or both of the bodies in order to be able to conclude unique-
ness. Already in 1938, P. S. Novikov showed that if P1 and P2
are starlike with respect to some point and their exterior poten-
tials coincide, then P1 • P2 • In particular, the correspondence
between homogeneous solids and their exterior potentials is one-
to-one when restricted to the class of convex bodies. Subsequent
refinements extend this result to the class of bodies convex in a
fixed direction. (Some boundary regularity is required for these
results.)
Theorems of this sort leave a good deal to be desired from
the practical point of view. In a typical application, the
external potential of an unknown homogeneous solid P may be
measured and found to coincide with the potential generated by,
say, a convex body. In· the absence of additional information on
P, there is clearly no warrant to apply Novikov's theorem.
Thus, practical necessity, wishful thinking, and the converse to
Newton's theorem all point toward the:

CONJECTURE. Let P1 and P2 be homogeneous solids whose

external potentials coincide near infinity. If at least one of
the solids is convex, P1 = P2 . This problem seems very

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 3~ LAWRENCEZALCMAN

difficult. Beyond the result for balls and ellipsoids mentioned

in Section 2, nothing is known.
Finall~ let us note that in the discussion of this and
previous sections it has been assumed (largely for convenience)
that all solids have unit density. The questions·posed also make
sense, and are interesting and important, in case P1 and P2
have differing (constant) densities.

1. THE INTERIOR PROBLEM. Analogues of the results mentioned in

Sections 2 and 6 above are also available for the inverse problem
for interior potentials. For instance, if the Newtonian poten-
tial of a homogeneous solid P agrees on an open set of P with
that of a homogeneous ellipsoid E, then P = E. Similarly, two
homogeneous convex bodies whose interior potentials agree on a
common region must be identical; indeed, it suffices for the
bodies to have a (common) real analytic density; see [4], [5], [lOL
If P1 and P2 are homogeneous solids whose potentials
agree on P1 n P2 , must P1 ~ P2 ? Must two homogeneous solids
whose interior potentials agree with the same real entire
function on R3 coincide? This last question is due to
H. S. Shapiro (private communication).

8. FLOATING BODIES. A homogeneous solid will float in equil-

ibrium (without turning) in water for every given orientation.
Must it be a ball? This problem was posed over half a century
ago by s. M. Ulam. The two-dimensional analogue of this question
has a negative answer, see [2].
Denote the solid by B, its density by p, and the density
of water by a. A necessary and sufficient condition for

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 347

equilib~ium is that the volume of the subme~ged pa~t A confo~m

to the Law of A~chimedes

oV(A) = pV(B)

and that the line th~ough the cente~s of g~avity of A and B

be pe~pendicula~ to the su~face of the wate~.

Letting the density p tend to ze~o leads to the following

question: if a body ~ests in equilib~ium in eve~y position on a
flat ho~izontal su~face, must it be a ball? Answer: yes [25].

9. DRUMS IN THE NIGHT. A thin elastic memb~ane M of unifo~m

a~eal density o is st~etched to a unifo~m tension T and held

fixed at its bounda~y r, a simple closed plane cu~ve. The
small t~ansve~se vib~ations of M can be modeled as solutions
u(x,t) of the wave equation on D, the ~egion bounded by r,
which vanish on r:

1 au
Au=---
2
X 6 D, t > 0
c2 at 2
(5)
u(x,t) = 0 x 6 r, t Cl: o.

He~e c = IT70 is the wave velocity and A is the Laplacian

with ~espect to x = (x 1 ,x 2 ).
Suppose some solution u of (5) has the property that
g~ad u • (ux ,u ) vanishes identically on a simple closed cu~ve
1 x2
Y c D u r. Must r be a ci~cle?

In case r is a ci~cle (of ~adius R, say, about the

o~igin), the function u(x,t) • J 0 (klxl)eickt will satisfy (5)
if kR is a ze~o of the Bessel function J0• Since J 0 = -J 1 ,

g~ad u • ~kJ 1 (klxl)eickt g~ad lxl. Thus if J 1 (kr) • 0, g~ad u

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 348 LAWRENCE ZALCMAN

will vanish on the circle of radius ~ concentric with r.

Choosing k sufficiently large yields solutions of (5) which
vanish on a family of such circles.

10. MATZOH BALL SOUP. Consider the heat conduction problem for a
homogeneous solid B. Denoting by T(x,t) the temperature at
x 6 B and time t, we have

X 6 8°, t ) 0

T 0 X 6 B , t • 0

T x 6 as, t >o

where ~ is the Laplacian with respect to the space variables.

If B is a ball, the solution is radially symmetric; and so
the isothermal surfaces do not vary with time. (Of course, the
temperatures on them vary.) Is the ball the only bounded solid
having invariant isothermal surfaces? If one allows unbounded
solids, another solution is given by the infinite right circular
cylinder, which corresponds to the spherical solution in two
dimensions.

BIBLIOGRAPHY

1. Dov Aharonov, M. M. Schiffer, and Lawrence Zalcman,

"Potato kugel," Israel J. Math.~ (1981) 331-339.
2. H. Auerbach, "Sur un probl~me de M. Ulam concernant
l'~quilibre des corps flottants," Studia Math. I (1938) 121-142.
3. Aivars Celmipi, 11 Direkte Verfahren zur Auswertung von
Schweremessungen bei zweidimensionaler Massenverteilung," Geofis.
Pura Appl. 38 (1957) 81-122.
4.
Pierre Dive, "Attraction des ellipsoides homog~nes et
r~ciproque d' un thoor~me de Newton," Bull. Soc. Math. France 59
(1931) 128-140.

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 SOME INVERSE PROBLEMS OF POTENTIAL THEORY 349

5. Pierre Dive, "Sur l'identit' de deux corps poss,dant le

meme potential newtonian dans une r'gion int,rieure commune,
C. R. Acad. Sci. Paris 195 (1932) 597-599.
6. L. Flatto and D. J. Newman, "An inverse drum problem,
(Problem 73-24*)," SIAM Rev • .1.2. (1973) 788.
7. V. M. Isakov, "On the uniqueness of a ~elution of the
inverse problem of potential theory," (Russian) Dokl. Akad. Nauk
SSSR 245 (1979) 1045-1047; translated in Soviet Math. Dokl. 20
(1979r-387-390. --
8. A. V. Kondra~kov, "On the uniqueness of the reconstruc-
tion of certain regions from their exterior gravitational poten-
tials," Ill-posed Mathematical Problems and Problems of Geo-
physics, Novosibirsk (1976) 122-129 (Russian). ----
9. M. s. Klamkin, "A phy~ical characterization of a sphere
(Problem 64-5*)," SIAM Rev. §_ ( 1964) 61.
10. Wladyslaw Nik1iborc, "Eine Bemerkung tiber die Vo1umpo-
tentiale," Math. z. 35 (1932) 625-631.
11. P. S. Novikoff, "On uniqueness for the inverse problem
of potential theory," (Russian) c. R. (Dokl.) Acad. Sci. URSS
(N. S,) ~ (1938) 165-168.
12. A. I. Prilepko, "Inverse problems of potential theory
(elliptic, parabolic, hyperbolic, and transport equations),"
(Russian) Mat. Zametki 14 (1973), 755-767; translated in Math.
Notes 1! (1973) 990-996--.
13. A. I. Prilepko, "Uniqueness of the solution of the
exterior inverse problem of the Newtonian potential," (Russian)
Differentsial'ney Uravneniya 2 (1966), 107-124; translated in
Differential Equations£ (196b) 56-64.
14. Makoto Sakai, "A moment problem on Jordan domains,"
Proc. Amer. Math. Soc. 70 (1978) 35-38.
15. Russell A. Smith, "A uniqueness theorem coricerning
gravity fields," Proc. Cambridge Philos. Soc. 57 (1961) 865-870.
16. Russell A. Smith, "Nonuniqueness in the inverse gravity
problem for infinite polygonal cylinders," J. Inst. Math. Appl.
£1 (1978) 61-65.
17. L. N. Sretenskil, "On the uniqueness of determination of
the form or an attracting body from the values of its external
potential," (Russian) Dokl. Akad. Nauk SSSR 22. (1954) 21-22.
18. V. N. Strakhov, "Notes concerning the plane gravimetry
problem," (Russian) Izv. Akad. Nauk SSSR Ser. Fiz. Zemli 1970
no. 12, 32-44; translated in Izv. Acad. Sci. USSR Phys. Solid Earth
§_ (1970) 774-779.
19. S. M. Ulam, A Collection of Mathematical Problems,
(Interscience) New York (1960).

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 350 LAWRENCE ZALCMAN

20. R. Wavre, "Sur les polydromies de certains potentiels

newtoniens prolonges," Math. z. TI (1933) 739-748.
21. R. Wavre, "Surles polydromies que presentent les
potentiels newtoniens lorsqu'ils sont prolonges au travers des
corps generateurs," Camp. Math. 1 (1934) 69-84.
22. R. Wavre, "Sur le probl~me inverse de la theorie du
potentiel et les fonctions [Link] multiformes," Counnent. Math.
Helv. ~ (1933/34) 317-327. ·
23. R. Wavre, "Sur les corps potentiellement equivalents et
les fonctions harmoniques multiformes," Comment. Math. Helv. 7
(1934/35) 131-140. -
24. M. s. Zhdanov, "Some problems in the theory of interpre-
tation of gravitational anomalies depending on three coordi-
nates," Izv. Akad. Nauk SSSR Ser. Fiz. Zemli 1974 no. 9, 32-46;
translated in Izv; Akad. Nauk SSSR Phys. Solid Earth 10 (1974)
565~571 . --
25. Luis Montejano, "On a problem of Ulam concerning a
characterization of the sphere," Studies in Appl. Math. 53 (1974)
243-248.
DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE
BAR-ILAN UNIVERSITY
52 100 RAMAT GAN, ISRAEL

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 Recent Titles in This Series
(Continued from the front of this publication)

157 Alfio Quarteroni, Jacques Periaux, Yuri A. Kuznetsov, and Olof B. Widlund, Editors,
Domain decomposition methods in science and engineering, 1994
156 Steven R. Givant, The structure of relation algebras generated by relativizations, 1994
155 William B. Jacob, Tsit-Yuen Lam, and Robert 0. Robson,Editors, Recent advances in
real algebraic geometry and quadratic forms, 1994
154 Michael Eastwood, Joseph Wolf, and Roger Zierau, Editors, The Penrose transform and
analytic cohomology in representation theory, 1993
153 RichardS. Elman, Murray M. Schacher, and V. S. Varadanijan, Editors, Linear algebraic
groups and their representations, 1993
152 Christopher K. McCord, Editor, Nielsen theory and dynamical systems, 1993
151 Matatyahu Rubin, The reconstruction of trees from their automorphism groups, 1993
150 Carl-Friedrich BOdigheimer and Richard M. Hain, Editors, Mapping class groups and
moduli spaces of Riemann surfaces, 1993
149 Harry Cohn, Editor, Doeblin and modem probability, 1993
148 Jeffrey Fox and Peter Haskell, Editors, Index theory and operator algebras, 1993
147 Neil Robertson and Paul Seymour, Editors, Graph structure theory, 1993
146 Martin C. Tangors, Editor, Algebraic topology, 1993
145 Jdfrey Adams, Rebecca Herb, Stephen Kudla, Jian-Shu Li, Ron Lipsman, and Jonathan
Rosenberg, Editors, Representation theory of groups and algebras, 1993
144 Bor-Luh Lin and William B. Johnson, Editors, Banach spaces, 1993
143 Marvin Knopp and Mark Sheingorn, Editors, A tribute to Emil Grosswald: Number
theory and related analysis, 1993
142 Chung-Chun Yang and Sheng Gong, Editors, Several complex variables in China, 1993
141 A. Y. Cheer and C. P. van Dam, Editors, Fluid dynamics in biology, 1993
140 Eric L. Grinberg, Editor, Geometric analysis, 1992
139 Vinay Deodhar, Editor, Kazhdan-Lusztig theory and related topics, 1992
138 Donald St. P. Richards, Editor, Hypergeometric functions on domains of positivity, Jack
polynomials, and applications, 1992
137 Alexander Nagel and Edgar Lee Stout, Editors, The Madison symposium on complex
analysis, 1992
136 Ron Donagi, Editor, Curves, Jacobians, and Abelian varieties, 1992
135 Peter Walters, Editor, Symbolic dynamics and its applications, 1992
134 Murray Gerstenhaber and Jim Stashefl', Editors, Deformation theory and quantum
groups with applications to mathematical physics, 1992
133 Alan Adolphson, Steven Sperber, and Marvin Tretkofl', Editors, p-adic methods in number
theory and algebraic geometry, 1992
132 Mark Gotay, Jerrold Marsden, and Vincent Moncrief, Editors, Mathematical aspects of
classical field theory, 1992
131 L.A. Bokut', Yu. L. Ershov, and A. I. Kostrikin, Editors, Proceedings of the International
Conference on Algebra Dedicated to the Memory of A. I. Mal'cev, Parts 1, 2, and 3, 1992
130 L. Fuchs, K. R. Goodearl, J. T. Stafford, and C. Vinsonhaler, Editors, Abelian groups and
noncommutative rings, 1992
129 John R. Graef and Jack K. Hale, Editors, Oscillation and dynamics in delay equations,·
1992
128 Ridgley Lange and Shengwang Wang, New approaches in spectral decomposition, 1992
127 Vladimir Oliker and Andrejs Treibergs, Editors, Geometry and nonlinear partial
differential equations, 1992

(See the AMS catalog for earlier titles)

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 This page intentionally left blank

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]
 ISBN 0-8218-5071-7

Licensed to AMS.
License or copyright restrictions may apply to redistribution; see [Link]