Real-Time Listening and the Act of Mental Pointing: Deictic and Indexical Claims

Mark Reybrouck
University of Leuven

ABSTRACT

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deitic and indexical claims. Mind,
Music, and Language, 2, 1-17.
[Link]



Music can be considered as an artifact, as a static structure that can be objectified in imagination without being
presented to the senses. We can think, e.g., of the third symphony of Bruckner or a Beatles song as one single
thing. This is a reification of music, considered outside of time, to be dealt with at a virtual level of reality and
relying on symbolic replicas of the sounds. Real music, on the contrary, is a temporal and sounding
phenomenon, evolving in real time and impinging on the senses. As such, it calls forth processes of sensemaking and engagements, allowing the listener to enact a musical experience and to react even bodily to the
sounds (Krueger 2009, 2011; Schiavio et al. forthcoming).
This contribution elaborates on this distinction by arguing for an experientialist as against a merely
conceptual-symbolic approach to music. The latter approach is characterized by distancing and polarization
between the listener as a subject and the music as an object, to use object Werners and Kaplans (1963)
terminology (see Reybrouck 2005b, 2006 for a critical discussion); the former approach is dynamic-vectorial
and directive by its very nature, stressing the field of pointing rather than the symbolic field of meaning. This
field of pointing also called the deictic field of meaning (Bhler 1934; Hanks 2005) is an interesting frame of
reference. It provides operational tools for the listeners making sense out of the perceptual flux by raising two
major questions: what are the entities he/she is pointing at and how does he/she delimit the denotable things
as signified (Reybrouck 1999)? The questions are related to listening strategies rather than to an objective analysis
of the music, taking seriously the idiosyncrasies of individual listeners and their individual listening learning
histories (Delalande 1998, 2013).

Deixis and Deictic Terms

The concept of deixis goes back to Bhler (1934) who drew an explicit analogy between gestural and linguistic
means for showing direction or place. Conceiving of two basic types of linguistic expressions namely deictics
(or pointing words) as opposed to symbols (or naming words) he presented as a main thesis that deictic
expressions refer to a deictic field of language whose zero point (the Origo) is fixed by the person who is
speaking (I), the place of utterance (here) and the time of utterance (now). Deictic terms, accordingly, are words
that pick out or point to things in relation to the participants in a speech situation (Clark 1978). Terms as
this/that, here/there, I/you, my/your, etc. are typical examples. They are related to the notion of
indexicality (Nunberg 1993; West 2011) and the notion of pointing and its correlates (Kita 2003). Deixis, as a
source of reference, thus locates individual elements in context, rather than simply tagging them. It has its
origin within the speaking situation, with the meaning of deictic expressions depending crucially on when,
where and by whom they are used. Each ordinary speech situation, therefore, can be systematized in terms of
personal, spatial and temporal deixis (the so-called socio-spatio-temporal axes). Deictic expressions, therefore, have
to be defined with reference to the speech event, its participants and its settings in order to allow an operational

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 2

description of space/time moments and their relations to the position and time of utterance (Fillmore 1972,
1982).
Deictic procedures, further, focus the hearers attention towards a specific item of the deictic space
(Ehlich 1982) and realize a form of joint attention, the sharing of overlap in the focal attention of the parties of a
referential exchange (Diessel 2006; West 2011; Grassmann & Tomasello 2009; Seemann 2011). The means for
doing so comprise a set of procedures, varying from such evolved linguistic devices as anaphora (referring back
to text, e.g. this, that...) and deixis (referring back to context) to simple ostensive pointing (Bruner 1983, p. 68).

Deictic and Indexical Elements In Music

Introducing deictic elements in music can be done primarily by focusing on the concepts of deictic space and
temporal and spatial deixis. Taking as a starting point Lakoffs metaphorical treatment of time as space (Lakoff
1993, p. 216), time can be understood in term of things (entities and locations) and motion. With respect to
location, the present time is conceived mostly as being at the same location, future times are located in front of
and past times behind the observer who functions as the deictic center. With respect to the motional aspect
there are two possibilities: time is moving with respect to the observer or the observer is moving with respect
to time (see Reybrouck 2004 for a discussion). The distinction is related to the perspective the listener takes on
the music (Gody 1997), especially with respect to his/her windows on the time-line and the more conceptual
or experiential way of listening. There is, in fact, a distinction between a kind of panoramic listening with the
music unfolding in time and the listener functioning as a stationary center as against the moment-to-moment
scanning of the temporal unfolding of the sonorous articulation over time with the listener moving along with
the music. The former represents the music at a glance, reducing the sequential aspects to a simultaneous image
that can take a symbolic stance, the latter keeps step with the discrete particulars and idiosyncrasies of its
unfolding, and, as such, does more justice to the musical experience proper.
Both modes of processing are illustrated in figure 1, which shows a waveform notation of Farewell
to Stromness by Peter Maxwell Davies. The upper pane depicts the sonorous articulation of the whole piece
(4.20 minutes of length) and offers a panoramic view of the whole unfolding. The little box at the left side of
the upper pane, however, zooms in in the lower panes (left and right channel of the stereo track) and shows an
unfolding of about 32.50 seconds. When playing back the fragment, the vertical cursor at the left side moves to
the right allowing the listener to keep step with the unfolding in real time and to abandon his/her stationary
position in favor of a position that moves along with each new now moment.

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 3

Figure 1. Depiction of the temporal unfolding of Farewell to Stromness by Peter Maxwell Davies both as a panoramic
view (upper pane) and as a zoomed in window (lower pane).

In what follows, both modalities are brought together (Reybrouck 2005a, 2005b), leaning heavily on the
theoretical groundings of William James who elaborated in depth on the relationship between percept and
concept. In his little-known but very important doctrine of radical empiricism he states that the significance of
concepts consists always in their relation to perceptual particulars:
We extend our view when we insert our percepts into our conceptual map ... but the map remains superficial through
the abstractness, and false through the discreteness of its elements ... Conceptual knowledge is forever inadequate to the
fullness of the reality to be known. Reality consists of existential particulars as well as of essences and universals and
class-names, and of existential particulars we become aware only in the perceptual flux. The flux can never be
superseded. (James 1968, p. 245)
The perceptual flux, however, can be transformed into a succession of objects, as advocated by Pierre Schaeffer
(1966) who conceived of sonorous objects (objet sonore) as enabling a representation of delimited segments of
the fluctuating musical substance in a more solid form. This transition is not without problems, but the concept
of resolution may be helpful here. According to Gody, it offers the possibility of thinking a musical object in
different temporal representations, from real time versions to extremely compressed, i.e instantaneous or synoptic kinds of
representations, which have also been called outside time representations of musical objects (Gody 1997, p. 11). Such a

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 4

shift in resolution allows the emergence of relatively stable forms on the basis of a rather unstable, complex
and distributed substrate, allowing a quantal orientation of meaning (Gody 1997, p. 54, 149) as a
hermeneutical extension of listening. This grasping of a meaning out of a flux allows the listener to interpret
something as something etwas als etwas, to use a Heideggerian expression (1977, p. 149) and to conceive of
this quantal and stable forms in categorical terms.
Pointing, as a deictic tool, therefore, can be directed to delimited segments of the sonorous unfolding,
but can keep step with the perceptual flux as well. As such it offers the possibility to bridge the gap between
discrete and continuous decoding, somewhat analogous to the difference between categorical and auditory
perception (Handel 1989, p. 4, 274), with the former referring to the perceptual events as separate entities and
the latter to the acoustic properties of the perceptual events, which are distributed over time.
Both approaches are helpful in dealing with sounding music. Listeners, in fact, must be able to foresee
and to remember this requires a kind of synoptic overview but the experiential approach also requires a
continuous scanning of the sonorous unfolding in search for discrete particulars and idiosyncrasies. Conceiving
of the listener as a pointing device, moreover, provides means for giving an adequate identification of the place
and time in this unfolding. This location process is depicted schematically in figure 2. At the actual now
moment a kind of moving origo the listener can recollect in memory the previous events as well as
anticipate future events.

Figure 2. Schematic depiction of deictic listening with the actual now moment as a moving origo and the recollection of
previous sounding events and anticipation of future events.

The result of such recollecting and anticipating behavior is the setting up of a deictic space with a basic
reference point (Klein 1982). Deictic spaces, in general, are set up by summing up all possible denotata the
things or events that can be pointed at of local deictics or localities (e.g. rooms, streets, cities, countries), but
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 5

the denotata need not be localities. They can be abstract places in a train of thought as well, as deictic spaces
are sets of elements (the possible denotata) provided with some structure. It is possible, therefore, to conceive
of music as a deictic space as well, and to use this conceptual tool in order to guide the listener in his or her
involvement with the music. Listeners, in fact must go from here to there in a kind of mental journey
through the music (Delige 2001; Reybrouck 2010), and in order to do this, they must rely on a kind of
cognitive representation of the crossed space. This can be done by working out a cognitive map or route-description
that is the outcome of the structuring of previous and actual experiences. Cognitive maps, further, may be
differently structured for different individuals, but even if they are vague, incomplete, or even wrong in some
respects, they are very informative as to the way how individuals structure their own routes and how their
attention may be focused on different objects or events (Klein 1982, p. 170).
Route-descriptions, further, can be handled in anaphoric terms, which means that participants of a
referential exchange refer back (catadeixis) to things just treated (this, that...) or to things that are to be
treated immediately (anadeixis) (Ehlich 1982). In order to do this, they must have access to the flow of
discourse as a whole, where parts may be re-taken up and anticipated. To quote Bhler (1982): [the] whole must
be accessible to sender and receiver, so that a wandering is possible, comparable to the passing of ones gaze over an optically present
object. ... The adequate production and reception of any piece of music, for example, requires not exactly this, but something
similar (p. 21).

Pointing as a Selection Device

The act of pointing is a primitive marking system for singling out the noteworthy. By emancipating itself from
an object-oriented movement (merely grasping) to an act of pointing (Vygotsky 1978, p. 56) it realizes an
internal reconstruction of an external operation, which can go so far as to mentally pointing to things that are
not physically present (James 1968, p.155). Such an act of pointing (externally or internally reconstructed)
mostly begins with the emergence of a kind of quality in combination with an insistent particularity, e.g. this is
important, that is difficult (Whitehead 1968). The multiplicity of facts, however, requires a selection in dealing
with them, and this selection requires the notion of relative importance. To quote James (1968): Only new-born
babies, or men in semi-coma from sleep, drugs, illness, or blows, may be assumed to have an experience pure in the literal sense of a
that which is not yet any definite what, tho ready to be all sorts of whats ... But the flux of it no sooner comes than it tends to fill
itself with emphases, and these salient parts become identified and fixed and abstracted; so that experience now flows as if shot
through with adjectives and nouns and prepositions and conjunctions (p. 215).
We take thus as an essential aspect the construction of reality and the elevation of some entities to a
special level of prominence within predications. This designation is one of the pivotal constructs of cognitive
grammar, as advocated by Langacker (1987), but it is essential also for all constructivist claims of reality. To quote
Varela (1979): ..all of these items that we come to consider more or less permanent must, at some point, have been isolated and
individuated in the field of our experience. This isolating and individuating nesessarily had to be achieved by us, for it is we who
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 6

say that we are aware of them. That is, we must have differentiated them and cut them out from the rest of our experiential field and by that very act the rest of our experiential field became their environment (p. 273).
This singling out of entities is dependent upon the conceptual organization of the parties of a
referential exchange. It demands that both speaker and listener know three things about objects: that they have
an independent existence, apart from speaker and listener, that they can be individuated and that they belong to
classes (Clark 1978). One can question the first assumption (independent existence) as it challenges modern
conceptions about cognition and categorization (Dubois 1991). The act of singling out, however, is important,
as is obvious from the use of pragmatic anaphora (deictic expressions that are accompanied by a pointing gesture,
e.g. I bought that [pointing] yesterday) (Jackendoff 1988).
Pragmatic anaphora refer to designated objects (things), but also to other ontological category features
such as places, directions (paths or trajectories), actions, events, sounds, manners, amounts and numbers and all
entities that can be raised to the level of identity and individuation. As such, they can be applied to the realm of
music, especially with respect to the problem of musical sense-making in real time listening and the role of
selective focusing and the general tendency toward event-perception as discrete, yet continuous things that can
be pointed at (Handel 1989; Reybrouck 2005a).

Pointing as a Predication Process

Pointing to something uses this thing or event as an ostensive component. It is done frequently in language
learning and in providing names for identifiable objects (Bateson 1985, p. 125; Kalagher & Yu 2006; Pizzuto &
Capobianco 2008). Pointing, however, is predicative as well, if at least some generality can be ascribed to what is
pointed at. Predication, further, is either direct or mediated, dependent on whether or not some physical
stimuli are presented to the senses, somewhat analogous to the problem of immediate as against represented
knowledge (James 1968, p.152) and the distinction between direct knowledge and symbolism (Whitehead
1927, p.13). Direct knowledge provides first-hand knowledge, symbolic knowledge elicits consciousness,
beliefs, emotions and usages, respecting other components of the experience proper. The role of presentational
immediacy, therefore, is critical in setting out the borders between perception and consciousness. The latter, in
fact, involves complex mediated processes that supplant the immediacy of natural perception, as stressed
already in the early theoretical writings of Vygotsky (1978).
Listening, as a cognitive activity, is dependent both on perception and consciousness. At the
perceptual level it can be defined as an articulation through time, with conceptualizations by the listener being
added as the music proceeds. As such, it combines an experiential-perceptual and symbolic-conceptual
approach (Reybrouck 2005a), both of which can be described in more operational terms in relying on
Langackers concepts of processual predication and episodic nominalization. The former (processual predication)
follows the temporal evolution of a situation, and involves a continuous series of states that represent different
phases of the process as occupying a continuous series of points in conceived time. The span of time during
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 7

which the evolution is tracked to is referred to as its temporal profile (Langacker 1987, p. 244). The latter
(episodic nominalization), refers to just a single instance of the process and can be considered as a thing or
event that can be characterized as a bounded region in some domain. A beep, for instance, is bounded in both
pitch and time. Duration is also limited for a blip and a flash, but a flash must be almost instantaneous, whereas
a blip can persevere (Langacker 1987, p. 191).
Both concepts can be easily applied to the actual unfolding of music through time by cultivating the
global qualities of delimited segments of musical unfolding rather than the more abstract kinds of structures
encountered in some music theory (Gody 1997). Such global qualities are dependent on the size of the
segment under consideration at any given moment, but they can only emerge on the basis of a distributed
substrate, as a collection of more elementary units in succession. What matters is the recognition of the dual
status of the musical substance as both having a distributed substrate and a more or less unitary emergent
quality (Gody 1997, p. 37).
Starting from the actual unfolding of music through time, listeners can thus select and (mentally)
point to delimited segments of this unfolding, extending the position in a time-series from discrete slices of
time to larger temporal spans. Or as Gody (1997) puts it, it is possible to establish means of obtaining
knowledge at different temporal levels (a single tone, a phrase, an entire movement) and, dependent upon the
listeners strategies, to move between different musical objects of different sizes, to zoom and pan both in and out and
sideways and filter out different kinds of information (p. 40). These claims, again, are somewhat related to
Langackers conception about focal adjustments such as selection, perspective and abstraction. Focal adjustments of
selections determine which facets of a scene are dealt with. Perspective relates to the position from which a
scene is viewed (in the case of the visual domain). It influences the object of our cognition and related
problems such as figure/ground alignment and the viewpoint that is taken where a figure can be defined as a
substructure perceived as standing out from the remainder (the ground) and accorded special prominence as the pivotal entity
around which the scene is organized and for which it provides a setting (Langacker 1987, p. 120). The viewpoint, on the
other hand can be understood as subsuming the notions vantage point and orientation, with the former
referring to the position from which a scene is viewed, and the latter to the alignment with respect to the axes
of the visual field. The concept of abstraction, finally, pertains to the level of specificity at which a situation is
portrayed (Langacker 1987, p. 117).

Deixis and Categorization

Categorizing is a cognitive activity of utmost importance. As such, it has received a lot of theoretical grounding.
According to James, e.g., it substitutes a conceptual order for a perceptual one (James 1968, p. 234). Or as
Bruner et al. have put it: To categorize is to render discriminably different things equivalent, to group objects and events and
people around us into classes, and to respond to them in terms of their class membership rather than their uniqueness. (Bruner,

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 8

Goodnow & Austin 1956, p. 1). In order to manage complex environments, we therefore categorize so that we
must not react to every stimulus as if it should have its own and single signature.
Since these early definitions, much research has been done on categorization. The pioneering work of
Rosch (1978) still stands out by offering some workable hypotheses with respect to its underlying mechanisms,
such as the principle of cognitive economy and the principle of reality. The former stresses the importance of
providing the maximum of information with the least cognitive effort; the latter claims that the perceived world
is not unstructured, but that it consists of real and natural discontinuities and co-occurrent properties. What
matters, then, is the process of recognition through identification and differentiation.
This process of recognition, further, involves units and their relations, both of which can be structured
with respect to previous experiences (Langacker 1987), which may serve as a standard for an act of comparison
in which some facet of a current experience functions as a target. When the difference between standard and
target approximates zero, the overall event is one of recognition; when the similarity is only schematic, the
recognition amounts to categorization.
Musical sense-making, accordingly, involves processes of categorization. In contrast to categorization
in general, however, things are rather complicated with music so that several questions should be raised
(Reybrouck 2003, 2009). What are, e.g., the units and the categories to be delimited? What kind of categorical
judgments are to be made (nominal or processual)? And what are the ontological and epistemological
assumptions? Are musical categories to be handled in terms of realism or nominalism? Should we conceive of
them as real sounding things (realia) or as names to be assigned to them (nomina)? Or stated in different terms:
should we deal with them in an empirical or mentalist way?
The questions are related to the distinction between classical (objectivist) and modern conceptions of
categorization with a shift from a mere focus on perceptual attributes and names to categorizations in terms of
functional qualities and action words (for an overview, see Dubois 1991; Reybrouck 2012; Schiavio et al.
forthcoming). A chair, for example, affords sitting for an adult person, but it does not for a child for whom it
affords only crawling or hiding under. Different perceivers who vary in their morphological and kinematic
details, as well as in their goaldirected activities and individual learning histories will therefore respond
differently to what it affords for them (Mazet 1991; Krueger 2014). This brings us to an affordance-based
approach to perception where what matters is not merely the world in its objective qualities, but the world as
perceived by observers.
There is no space to elaborate on this affordance-based approach in this contribution (see Reybrouck
2012; Krueger 2014), which is centered around the special status of music as a temporal art and the role of
mental pointing with a gradual shift from mere pointing as a physical act over classifying to identifying with a
corresponding transition from a nominal to a processual approach.
Taking as a starting point the sonorous articulation through time, as a substrate for pure experience, it
is possible to put a categorization grid on the sonorous unfolding. Such a grid can be limited to an existing grid of
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 9

prototypes (a kind of lexicon of stylistic or panstylistic universals consisting of notes, motifs, phrases, chords,
cadences, rhythmical patterns, metrical groupings, etc.) but it can embrace variable categories (ill-defined
categories without fixed boundaries) as well. Categories, however, must have at least some cue validity, so that
they can lean themselves to identification and differentiation (Goldstone 1994). In the field of music, however,
much research still has to be done (see e.g. McAdams 1993; Zbikowski 2002). The pioneering work of
Schaeffer (1977) and Chion (1983, 1988) on the classification of sound and more recent work on the analysis of
electroacoustic music (Landy 1999; Couprie 2004) should be mentioned here. Some questions, however, can be
raised which may be important for future research.
A first problem concerns the distinction between discrete and continuous categories. Most
categorizations, as nominal predicates, refer to discrete things, rather than to events, which are continuous in
their unfolding. Music, however, is a temporal art, with musical things being considered as events rather than
things. Listeners, in fact, do not perceive the acoustical environment in terms of phenomenological
descriptions but as ecological events (Balzano 1986; Lombardo 1987; Reybrouck 2005a). The distinction is
related to the basic difference between continuous and discrete processing with eventsand also auditory
events being continuous in their unfolding but discrete in their labeling. Events moreover, are basic building
blocks in perception this is the event perception hypothesis which behave as sequences of stimuli that are
extended in time and which function as units in perception and memory. As such, there is a difference between
the recognition of a sounding object or an event as a discrete entity and the experience proper of its
sonorous articulation through time. As soon as we recognize something as something, we stop acoustical
processing in favor of conceptual processing which allows us to conceive of events in a propositional way.
Such propositional processing is much quicker and less demanding as to its processing effort as it is much
easier to select and delimit events and to pick them up in an act of episodic attention than to deal with them in
an act of sustained attention. This, again, is in a nutshell the core assumption of cognitive economy (Reybrouck
2005a).
Listening to music, accordingly, can be considered in terms of event perception, which, in turn, can
be considered as a kind of top-down processing of the music with schemata, which are imposed on the sound
(Gody 2006). What we hear are not acoustic properties but acoustic events, which receive significance as the result
of a process of semiotization of the sonic world. As such, it is possible to conceive of musical events as higherorder variables, which can be described as having time-varying complex acoustic properties with temporal
constraints, which may involve a transition from high-frequency (in the range of about 10 milliseconds) or
high-resolution processing to perceptual units (in the range of 2-3 seconds) which allow event identification
over time (Wittmann 1999; Wittmann & Pppel 1999-2000). Most musical events, e.g., have a clearly defined
beginning and ending and have a gross temporal patterning as well (Handel 1989). As such they hold a position
between invariance and change, allowing processes of discrete labeling and categorization.

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 10

A second problem is the scope of predication. Music has temporal as well as atemporal aspects of
organization, but in both cases there are plenty of componential elements that must be handled at once.
Langackers concepts of sequential and summary scanning are interesting here. They provide contrasting modes of
cognitive processing for structuring a complex in ways that are experientially quite different: Summary scanning
is basically additive, and the processing of conceptual components proceeds roughly in parallel. All the facets of the complex scene are
simultaneously available, and through their coactivation ... Sequential scanning ... involves the successive transformations of one
configuration into another (Langacker 1987, p. 248).
Both modes of processing are depicted schematically in figure 3, which illustrates the conceptual
flexibility to process a complex scene in two different modes. It is possible to structure the scene as a process
by activating the component states sequentially and to experience them successively with the passage of
processing time. It is possible, however, to activate also the component states simultaneously by means of
summary scanning and to superimpose them as to form a single gestalt.

Figure 3. Sequential and summary scanning (adapted from Langacker 1987). The upper part of the figure shows the
sequential states of a falling ball. The lower part shows the summing up of these states in one simultaneous image.

The combination of sequential and summary scanning, further, is somewhat related to the gesturaldynamical reconstruction of the music (Gritten & King 2013). Mentally pointing to the music, in fact, is mostly not
episodic, but involves a continuous process, a succession of gestures with a minimal span of time. The listening
process, then, can be described in terms of internalized gestures that follow the articulation of the sounds and
that may receive a kind of unit status and become the object of motor categorizations (Reybrouck 2001;
Gody 2003, 2006). These categorizations, however, have a hybrid status in the sense that they rely on a kind
of continuous knowledge representation that is analogue and procedural rather than declarative. It is the kind
of knowledge that comes from physically generating forms as illustrated in the idea of phoronomy from the
Greek for to drag as it was introduced by Kant in his Metaphysical Foundations of Natural Science (1970/1786,
p. 30, 41; see also Walker 1995) and elaborated in mathematics, for example by emphasizing the act of tracing a
geometrical form, such as a curve, rather than looking at it as a static depiction or object. The phoronomic
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 11

understanding of a shape thus comes from a continuous and indivisible experience of movement (DogantanDack 2007).
The phoronomic approach, finally, can be related also to the conceptual framework of embodied or
enactive cognition (Johnson 1987; Lakoff & Johnson 1999; Schiavio et al., forthcoming). Music, in this view, if not
described in terms of perceptual properties, but in terms of schemas and frames which take the body as a point
of reference. One of these schemas the sourcepathgoal schema is closely related to the phoronomic
approach. It states that every time we move anywhere there is a place we start from (the source), a destination
(end point), a path (a sequence of contiguous locations connecting the source and the destination), and a
direction (toward the destination) (Lakoff 1988). It is not difficult to apply this schema to the act of listening to
music. This work, however, mostly still has to be done.

Conclusion
Starting from the distinction between music as structure and music as a process, this contribution argues for an
experientialist approach to music. Relying on the conceptual framework of deixis and the related notions of
pointing and indexicality, it conceives of listeners as deictic devices that are able to locate themselves in the
context of the perceptual flux by selective focusing on actual now moments and by recollecting previous ones
in memory and anticipating future ones in imagination. In doing so, they can set up a deictic field with
temporal, spatial and personal coordinates, which can be elaborated in more operational terms by working out
a cognitive map or route-description of the sounding music. The elaboration of their maps may differ between
individual listeners and even between different listening moments of the same listener, dependent on the
perspective taken on the music (instantaneous or synoptic overview as against moment to moment scanning of
the temporal unfolding) and the level of abstractness and distance with respect to the sounding music. Rather
than thinking in terms of dichotomies between continuous and discrete decoding of the music, between
sequential and summary scanning, and between processual predication or episodic nominalization, it is argued,
further, that mental pointing to the music can be selective, predicative, categorical and even enactive and
gestural-dynamic. None of them should claim to hold the truth, but the combination of these approaches
should result in a richer musical experience that does justice to the definition of music as a temporal and
sounding art.

References
Balzano, G. (1986). Music perception as detection of pitch-time constraints. In V. McCabe & G. Balzano (eds.),
Event Cognition: An Ecological Perspective (pp. 217-233). London: Hillsdale.
Bateson, G. (1985). Mind and Nature. London: Fontana Paperbacks.
Bruner, J. (1983). Childs Talk: Learning to Use Language. Oxford: Oxford University Press.

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 12

Bruner, J., Goodnov, J., & Austin, G. (1956). A Study of Thinking. New York: Wiley.
Bhler, K. (1982). The deictic field of language and deictic words. In R. J. Jarvella & W. Klein (eds.), Speech,
Place and Action: Studies in Deixis and Related Topics (pp. 9-30). Chichester: John Wiley & Sons.
Bhler, K. (1934). Sprachtheorie: Die Darstellungsfunktion der Sprache. Jena: Fischer.
Chion, M. (1983). Guide des Objets Sonores. Paris: Editions Buchet/Chastel.
Chion, M. (1988). Du son la chose. Analyse musicale, 11, 52-58.
Clark, E. (1978). From gesture to word: On the natural history of deixis in language acquisition. In J. Bruner &
A. Garton (eds.), Human Growth and Development (pp. 85-120). Oxford: Clarendon Press.
Couprie, P. (2004). La place de l'informatique et du multimdia dans l'analyse des musiques lectroacoustiques.
In R. Barbanti, E. Lynch, C. Pardo & M. Solomos (eds.), Music, Arts, Technologies: Toward a Critical Approach
(pp. 361-374). Paris: L'Harmattan.
Delalande, F. (1998). Music analysis and reception behaviours: Sommeil by Pierre Henry. Journal of New Music
Research, 27, 13-66. [Link]
Delalande, F. (2013). Analyser la Musique, Pourquoi, Comment? Paris: Ina Editions.
Delige, I. (2001). Introduction. Similarity perception <--> Categorization <--> Cue Abstraction. Music
Perception, 18, 233-243. [Link]
Diessel, H. (2006). Demonstratives, joint attention, and the emergence of grammar. Cognitive Linguistics, 17, 463489. [Link]
Dogantan-Dack, M. (2007). Practice and theory: Ways of knowing music. Systems Research in the Arts, 8, 7-12.
Dubois, D. (Ed.) (1991). Smantique et Cognition. Catgories, Prototypes, Typicalit. Paris: Editions du CNR.
Eco, U., Santambrogio, M. & Violi, P. (Eds.) (1988). Meaning and Mental Representations. Bloomington: Indiana
University Press.
Ehlich, K. (1982). Anaphora and deixis: Same, similar, of different? In R. J. Jarvella & W. Klein (eds.), Speech,
Place and Action: Studies in Deixis and Related Topics (pp. 315-338). Chichester: John Wiley & Sons.

Fillmore, Ch. (1972). Anstze zu einer Theorie der Deixis. In F. Kiefer (ed.), Semantik und Generative Grammatik I
(pp. 147-174). Frankfurt: Athenum Verlag.
Fillmore, Ch. (1982). Towards a descriptive framework for spatial deixis. In R. J. Jarvella & W. Klein (eds.),
Speech, Place and Action: Studies in Deixis and Related Topics (pp. 31-59). Chichester: John Wiley & Sons.
Gody, R. I. (1997). Formalization and Epistemology. Oslo: Scandinavian University Press.
Gody, R. I. (2003). Motor-mimetic music cognition. Leonardo, 36, 317-319.
[Link]
Gody, R. I. (2006). Gestural-sonorous objects: Embodied extensions of Schaeffers conceptual apparatus.
Organised Sound, 11, 149-157. [Link]
Goldstone, R. L. (1994). The role of similarity in categorization: Providing a groundwork. Cognition, 52, 125-157
[Link]
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 13

Grassmann, S. & Tomasello, M. (2009). Young children follow pointing over words in interpreting acts of
reference. Developmental Science, 13, 252-263. [Link]
Gritten, A., & King, E. (2013). New Perspectives on Music and Gesture. Farnham: Ashgate Publishing.
Handel, S. (1989). Listening: An Introduction to the Perception of Auditory Events. Cambridge: MIT Press.
Hanks, W. (2005). Explorations in the deictic field? Current Anthropology, 46, 191-220.
Heidegger, M. (1977 [1927]). Sein und Zeit. Tbingen: Max Niemeyer Verlag.
Jackendoff, R. (1988). Conceptual semantics. In U. Eco, M. Santambrogio & P. Violi (eds.), Meaning and Mental
Representations (pp. 81-97). Bloomington: Indiana University Press.
James, W. (1976). Essays in Radical Empiricism. Cambridge: Harvard University Press.
James, W. (1968). The Writings of William James: A Comprehensive Edition. J. McDermott (ed.). New York: Random
House.
Johnson, M. (1987). The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University
of Chicago Press.
Kalagher, H., & Yu, C. (2006). The effects of deictic pointing in word learning. Proceedings of the 5th International
Conference of Development and Learning. Bloomington.
Kant, I. (1970). The Metaphysical Foundations of Natural Science. J. Ellington (trans.). Indianapolis: Bobbs Merrill.
Kita, S. (2003). Pointing: Where Language, Culture, and Cognition Meet. Mahwah: Erlbaum.
Klein, W. (1982). Local deixis in route directions. In R. J. Jarvella & W. Klein (eds.), Speech, Place and Action:
Studies in Deixis and Related Topics (pp. 161-182). Chichester: John Wiley & Sons.
Krueger, J. (2009). Enacting musical experience. Journal of Consciousness Studies, 16, 98-123.
Krueger, J. (2011). Doing things with music. Phenomenology and the Cognitive Sciences, 10, 1-22.
[Link]
Krueger, J. (2014). Affordances and the musically extended mind. Frontiers in Psychology, 4, 1003.
[Link]
Lakoff, G. (1988). Cognitive semantics. In M. Santambrogio & P. Violi (eds.), Meaning and Mental Representations
(pp. 119-154). Bloomington: Indiana University Press.
Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortonyi (ed.), Metaphor and Thought (pp. 202251). Cambridge: Cambridge University Press.
Lakoff, G., & Johnson, M. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought.
New York: Basic Books.
Landy, L. (1999). Reviewing the musicology of electroacoustic music. Organised Sound, 4, 61-70.
Langacker, R. (1987). Foundations of Cognitive Grammar, Volume 1. Stanford: Stanford University Press.
Lombardo, T. J. (1987). The Reciprocity of Perceiver and Environment: The Evolution of James [Link] Ecological
Psychology. Hillsdale: Lawrence Erlbaum.

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 14

Mazet, C. (1991). Fonctionnalit dans lorganisation catgorielle. In D. Dubois (ed.), Smantique et Cognition:
Catgories, Prototypes, Typicalit (pp. 89-100). Paris: Editions du CNRS.
McAdams, S. (1993). Recognition of sound sources and events. In S. McAdams & E. Bigand (eds.), Thinking in
Sound: The Cognitive Psychology of Human Audition (pp. 146-198). Oxford: Clarendon Press.
Nunberg, G. (1993). Indexicality and deixis. Linguistics and Philosophy, 16, 1-43.
[Link]
Pizzuto, E., & Capobianco, M. (2008). Is pointing just pointing? Gesture, 1, 82-103.
[Link]
Reybrouck, M. (1999). The musical sign between sound and meaning. In I. Zannos (ed.), Music and Signs:
Semiotic and Cognitive Studies in Music (pp. 39-58). Bratislava: ASCO Art & Science.
Reybrouck, M. (2001). Musical imagery between sensory processing and ideomotor simulation. In R. I. Gody
& H. Jrgensen (eds.), Musical Imagery (pp.117-136). Lisse: Swets & Zeitlinger.
Reybrouck, M. (2003). Musical semantics between epistemological assumptions and operational claims. In E.
Tarasti (ed.), Musical Semiotics Revisited. Acta Semiotica Fennica XV (p. 272-287). Imatra: International
Semiotics Institute.
Reybrouck, M. (2004). Music cognition, semiotics and the experience of time: Ontosemantical and
epistemological claims. Journal of New Music Research, 33, 411-428.
[Link]
Reybrouck, M. (2005a). A biosemiotic and ecological approach to music cognition: Event perception between
auditory listening and cognitive economy. Axiomathes, 15, 229-266. [Link]
Reybrouck, M. (2005b). Body, mind and music: musical semantics between experiential cognition and cognitive
economy. Transcultural Music Review, 9. [Link]
Reybrouck, M. (2006). Music cognition and the bodily approach: Musical instruments as tools for musical
semantics. Contemporary Music Review, 25, 59-68. [Link]

Reybrouck, M. (2009). Similarity perception as a cognitive tool for musical sense-making: Deictic and ecological
claims. Musicae Scientiae, 13, 99-118. [Link]
Reybrouck, M. (2010). Music cognition and real-time listening: Denotation, cue abstraction, route description
and cognitive maps. Musicae Scientiae, 14, 187-202. [Link]
Reybrouck, M. (2012). Musical sense-making and the concept of affordance: An ecosemiotic and experiential
approach. Biosemiotics, 5, 391-409. [Link]
Rosch, E. (1978). Principles of categorization. In E. Rosch & B. Lloyd (eds.), Cognition and Categorization (pp. 27-47).
Hillsdale: Erlbaum.
Schaeffer, P. (1966). Trait des Objets Musicaux. Paris: Editions du Seuil.
Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 15

Schiavio, A., Menin, D., & Matyja, J. R. (forthcoming). Music in the flesh. Embodied simulation in musical
understanding. Psychomusicology: Music, Mind and Brain.
Seemann, A. (2011). Joint Attention: New Developments in Psychology, Philosophy of Mind and Social Neuroscience.
Cambridge: MIT Press.
Vygotsky, L. (1978). Mind in Society: The Development of Higher Psychological Processes. M. Cole, V. John-Steiner, S.
Scribner & E. Souberman (eds.). Cambridge: Harvard University Press.
Walker, C. (1995). Karl Jaspers and Edmund Husserl III: Jaspers as a Kantian Phenomenologist. Philosophy,
Psychiatry, and Psychology, 2, 65-82.
Werner, H., & Kaplan, B. (1963). Symbol Formation: An Organismic-Developmental Approach to Language and the
Expression of Thought. New York: John Wiley.
West, D. (2011). Deixis as a symbolic phenomenon. Linguistik Online, 50, 6. [Link]
Whitehead, A. N. (1927). Symbolism: Its Meaning and Effect. New York: Capricorn Books.
Whitehead, A. N. (1968). Modes of Thought. New York: The Free Press.
Wittmann, M. (1999). Time perception and temporal processing levels of the brain. Chronobiology International,
16, 17-32. [Link]
Wittmann, M., & Eppel, E. (1999-2000). Temporal mechanisms of the brain as fundamentals of
communication - with special reference to music perception and performance. Musicae Scientiae, 3, 13-28.
[Link]
Zbikowski, L. (2002). Conceptualizing Music: Cognitive Structure, Theory, and Analysis. Oxford: Oxford University
Press.

Reybrouck, M. (2015). Real-time listening and the act of mental pointing: Deictic and indexical claims.
Mind, Music, and Language, 2, 1-17.
[Link]
2015 Mark Reybrouck

Reybrouck 16