Chapter 3: Observation and Modes of Experience
Read Our Reality, What an Experience (1) and What an Experience (2) prior to reading this part.
Theories of Observation
The most common theory of observation is an idea of "measurement" in which events are recorded at each instant. This idea is useful for measurements made in the laboratory and for tracking electrical signals in the brain but it does not explain our observation. It does not explain observation because we cannot observe anything in no time at all and the idea that our observation is a succession of instantaneous, fixed states bears little relationship to the geometrical form of the view in our Experience.
The graphic below represents how describing the world as a succession of fixed states removes time from the idea of observation (the greyed out zone). Such a pattern of measurements might record the latest data to be loaded into our brain but this does not describe our Experience.
Another common theory of observation is "dualism" in which our observation exists as a result of a supernatural soul.
The soul, by being outside time and space, can take a look at what is going on across a range of times. The biggest problem with the dualist "soul" is that it explicitly rejects any physical explanation of Experience by creating two independent worlds, the spiritual and the material. Rather than waiting for a physical description of Experience "dualism" proposes that our observation is non-physical.
In the following text an "Observational Theory" of Experience will be described. This will rely on what we actually have in Experience.
The Observational Theory attempts to fit what Experience is like to an idea of arrangements of events in time and space.
What is "time passing" like?
What is the passing of time like? We have a largely constant past at any moment that is expanded by events in our extended present.
If time exists as a single direction for arranging events then even random or unpredictable events are already written on our timeline. If we say “hi Jane” the word “hi” is fixed in time and the word “Jane” is also fixed in time a half second later so what moves from the “hi” to the “Jane”? Only our observation point needs to move. In fact only the observation point can move because, being a point, it is not fixed in time.
Observe your Experience whilst looking at this page at this moment and then observe your Experience whilst looking away. The version of your experience when you looked at this page does not travel forward in time. That version is now several seconds in the past and the version of you looking at the page is getting ever more historical.
So what moves into the future? The version of you, now, does not move because it is fixed in time.
Only something outside of time and space, such as a geometrical point, can move in time. Moving the observation point is equivalent to moving the view. The view extends in time which means that Experience will contain a bit of glum Joe even when Joe is happy.
The problem of "becoming" is even more acute if we reject the idea of time as a dimension because there is no time to know anything.
Depth
One of the most remarkable features of our Experience is that things are seen as if viewed from one side. Viewing from one side entails separation from the object being viewed and this separation is evident as our depth of vision. What is depth like?
If we move our fingers in and out towards this page or push our hand away from us we know the time extended motion as depth. Our sense of depth is closely related to movement. We can see this motion as depth in the Ouchi Illusion and the Necker Cube.
In the Necker cube the whole frame appears to move to change the 3D form of the cube.We do not have true three dimensional vision (we can never see the back of objects without turning them round), what we call 3D vision is a movement over time such as a movement in the image that occurs as our eyes take in the Necker cube. It takes about half a second to swap from one form of the Necker cube to another. When viewing objects in the world around us binocular vision accomplishes the motion of objects in the visual field by convergence and divergence of the eyes and refocusing. This motion provides a time extended (temporal) component to vision that is known to us as depth.
The judgement of depth should not be confused with the direct Experience containing depth. Our “judgement” of depth, as opposed to the depth sensation that occurs directly in our Experience, uses many other cues besides the temporal sense of depth, as an example, small images of people are judged to be far away and large images are judged as closer.
Ouchi illusion where apparent relative motion appears as depth as the disk floats.
When objects in Experience can move in front of other objects we also call this depth. The Ouchi Illusion is uncanny because reason tells us the disk cannot be floating. Notice that the periods of movement in the Ouchi Illusion are again 0.5 to one second long.
What we call depth is due to the presence of time extended objects in Experience.
Although nothing actually moves into our viewing point we can appear to move things towards and away from it. When I move my hand towards and away from my head in the world outside my body the model of this in my brain has my hand approaching and receding from a viewing point somewhere near the centre of the model of my head. This is why we really, really want to explain “seeing” as a flow from the visual stuff in our Experience into a viewing point that “sees”. But, of course, we never see anything flow into the point. The point is pure geometry. Everything that happens happens out there, in Experience and beyond.
Notice that something moving away in Experience is moving in time. When we move a hand towards this page the part of our visual image that contains the image of the hand simply becomes a tiny bit smaller over a short period of time. The “moving away” aspect is mainly the time extension of the muscle movements in and around our eyes and bodies. What we call depth of view is actually time extended motions in the view.
When we stand on the edge of a precipice the depth sensation is increased by the imagining of falling.
The separation between the observation point and the content of Experience is like a separation in time although we have learnt to explain it as a separation in space because it correlates with real separations in space outside our bodies. (It is probably the case that the separation is between earlier and later parts of the view connected at the observation point).
More about Depth
We get the information for seeing depth from parallax etc so that the brain can have “3D” visual experience. But have you ever wondered how Experience itself has depth? Our sense of depth is less obvious than we might expect. Most of the sense of depth is due to binocular vision (having two eyes) but when we look through both eyes, then shut one eye, very little changes. Take a look.
When we look at autostereograms the sense of depth is more acute than in real life. The only problem is that autostereograms can be hard to see. One of the simplest autostereograms is provided on Wikipedia:
(Courtesy: Wikipedia Autostereogram)
If you are new to autostereograms please refer to the Wikipedia article to find out how to adjust your eyes to see them. I am looking at this image with slightly crossed eyes to get a 3D effect, this makes the central row of tigers furthest away, but I can, with more effort, invert the depth sequence by using a distant focus (“wall eyed” vision). The 3D effect looks much better wall eyed but the autostereogram works either way.
Once you have the “3D” view notice how the image is heavily re-created by the brain. A whole new central column is inserted to create rows of 7 columns even though the original image has only 6 columns. Most people don’t notice the extra column, take a look.
The images in each column are created from combinations of the overlaid images from two adjacent columns. The far left and far right columns are exempt from these overlays. (Coloured dots have been added to the sharks in the image above to aid your analysis). The overlays are “smart” in the sense that both red and yellow coloured dots can be seen on the central upper shark, the image from neither eye obscures important content. The content of the image is synthesised by the brain on the basis of visual data.
There is much more to be noticed about autostereograms but the important property that will be discussed here is the sense of depth. This sense is immediate.
What property of the image conveys immediate depth? Certainly we can move our head and see parallax: the images in the rows “nearest” to us move more rapidly than those further away as we move our heads from side to side. But I also see the depth when my head is stationary so parallax is not the reason for immediate depth.
Is the sense of depth due to focus? The row that is the subject of my focus is clear whereas the other rows are slightly blurred. The image is divided into focal planes. If I shut one eye the focal plane effect disappears. The autostereogram is designed to have this effect because each row requires a slightly different vergence of the eyes to create clear, overlaid images.
However, clarity or focus cannot be the whole answer because there are no monocular autostereograms. It would be easy to create an image with two blurred rows and a central clear row but this would not create a sense of depth.
The immediate depth of the autostereogram in Experience must be something. We call this something “depth” by analogy with the world outside our brain. Depth is an independent direction for arranging events. It is obvious that the brain contains a synthetic image about the world within the wider space and time of Experience so what serves the role of depth in Experience?
Our Experience is time extended so distance might be represented directly by the time between events as well as, or instead of, the spatial distance between them. Is it possible that the rows in the autostereogram are, say, 0.1 secs apart rather than a centimetre or two apart?
If you wave a piece of paper across one eye at rates slower than about twice a second the depth is lost. Pick up a sheet of paper and try this experiment of rapidly covering and exposing an eye.
Given that visual splashes seem to be extended in time for less than a tenth of a second this suggests that the data for the sense of depth comes from a slower process than simple vision, such as vergence (the converging or diverging angle of the eyes) which takes about 0.5 secs to be established. It also shows that the contribution of the eye that is periodically covered by the paper can remain for 0.5 secs or so.
Time extension is similar to depth because it occurs in an independent direction for arranging events that is present in every dot of the apparent sphere of Experience (ie: we hear time extended notes at the position of each instrument in an orchestra). However, as will be explored in Chapter 4, time extension is not straightforward but seems to involve another, time-like direction for arranging events.
Is it possible that the sense of depth is an example of time extension rather than actual spatial depth? The idea is shown in the diagram below:
The time-like direction for arranging events is extended at every dot on the two dimensional surface of visual Experience.
This idea would have consequences such as the time extension of your moving arm having the quality of depth as the arm moves away from you and, more contentiously, the 0.5 secs of sound extension of words and bars of tunes having the quality of depth.
Is time extended sound arranged in an apparent direction away from the observation point? The time extension of a word is only about 0.5 secs and words are not continuous. This makes it very difficult to tell whether the “g” of “get” is slightly nearer than the “t”. Neither the “g” nor the “t” stay in place long enough to tell what is occurring.
If depth in Experience is due to the extension in time of visual Experience then it would be compatible with the idea of the brain as a set of varying electrical pulses and fields. However, using a fifth, time-like dimension to account for this is awkward - it is not impossible in physics so might be permitted at this stage as a “conceivable” description.
Eye scanning
When we use our eyes alone to scan a scene in our Experience it is as if we are clarifying places in the view every time our eyes stop. Unlike head movements, eye scans leave the general form of the image intact. Eye scans increase the resolution of the new centre of gaze compared with the previous resolution of that area of the view. Given that nothing actually flows from the view in Experience into our eyes in Experience what is really happening is that there is an increased resolution/definition of the part of the model that is the centre of gaze.
The existence of eye scanning means that the view has the qualities of a surface separated by an apparent space from the observation point. The space is "apparent" because, as seen above with the Ouchi illusion, depth is a correlate of motion and hence time extension. It is the angular motion relative to the centre of gaze that creates the sensation of depth that accompanies eye scanning.
What is our observation?
Consider the image below:
Imagine you had cut out the outline of the “d” above and had thrown the cut-out through the air to be carried away in the wind. Anyone picking up the cut-out would have no idea whether they had a “d”, “b”, “p” or “q” in their hands. The immediate “sidedness” of a two dimensional pattern requires three dimensions because we, the observer, have to be separate from the pattern, receiving simultaneous input from right and left, to see that one part, such as the bulge in this “d”, is on the left or on the right. That the content of our observation has an immediate left and right, a “sidedness” or “handedness”, suggests that our observation consists, in part, of a three dimensional geometry. “Dimension” sounds very grand but a dimension is simply an independent direction for arranging things, the three spatial dimensions are up-down, forward-back and left-right. Time is also a dimension with events arranged from the past to the future, and, as discussed above, the apparent forward-back in our Experience may be events spread in time rather than in space.
In the world outside our bodies and brains we could easily devise a machine to tell the difference between a “b” and a “d” as examined from a particular place. The “particular place” for the machine is most important because from another place the “b” will look like a “d”. The machine would need to transfer data so that the left and right sides of the image can be compared. The immediate comparison would require data acquired simultaneously from both sides of the "b" or "d". So the minimal machine for immediately distinguishing a "b" from a "d" requires a three dimensional framework with a preferred viewing direction and a simultaneous comparison of points in the "b" or "d". The single bit generated to indicate that the upright part of the "b" was on the left or the lack of a bit for the upright on the right provides the signal for"b"or "d". However, this is nothing like our observation. The single bit is analogous to the observation point but is not like an observation point. And, of course, a single bit is meaningless at any instant so fails to fulfill the requirement of immediately distinguishing “b” from “d”.
If an image is simply transferred from one place to another the problem of "sidedness" is not resolved. Resolving the immediate sidedness of an object requires that the object, or a copy of the form of the object, is loaded into a system that has the properties of a "view".
Whether or not we see a “b” or a “d” depends on our viewing direction. The direction of view within the three dimensions in our Experience is set by the location of the viewing point.
Immediate sidedness occurs at an observation point
We always view as if from the point. Our immediate grasp of the “sidedness” of objects requires that our Experience has at least three dimensions and also a viewing point and this confirms the description based on what vision is like. Vision is like viewing from a point.
It is clear that there is a viewing point but nothing is seen flowing into the point. So how can it be a viewing point? How can a separation in space be bridged without any movement, without any flow?
An example of how to bridge space without movement is to mark two points on a sheet of paper with ink dots and to bring them into contact by folding the paper. Anything that happens on one dot now happens no distance at all from the other dot. What was separated in two dimensions can be brought into contact by adding another dimension, a “dimension” being an independent direction for arranging things. Like the dots on the folded paper are separated in two dimensions but adjacent in three, the content of a visual image could both be distributed in three dimensional space and also at a point if another dimension, a fourth dimension, exists. Fortunately the world is indeed four dimensional so this “folding” of the visual image onto a four dimensional point is conceivable (See Appendix).
The reason for this digression on sheets of paper and dots is to show that there are possible physical explanations for the viewing point, not that the explanation given above is the explanation. We do not know how we can have Experience arranged as if it is a view from a point but we do know that physical explanations are conceivable. One day there will be a physical explanation of how we have a viewing point.
A separate viewing point allows a "b" to be immediately different from a " d" in the view. It also allows an immediate difference between a "b" and a "p" or a "d" and a "q" even though a cut out of a "b" can be laid down as any of these letters.
Once we have constructed an object in our Experience as, say, a “p” it can remain identified as “p” even if it is stuck to a board and rotated. Of course, the “p” will become a “d” with rotation but having once been identified it can become a rotated “p”. The difference between a “d” and a rotated “p” is of interest because they are objectively the same. There is an immediate connection between the form of the “p” and its attributes. An attribute of our “p” is the previous Experience containing its rotation. In the same way as our viewing point has many simultaneous objects present this is also true of each point in the view. The entire “p” object is connected simultaneously to its attributes at the viewing point as well as at the “p” itself. See the section on Qualities above for more on the connectedness of things.
We have come a long way in describing our Experience. Our visual Experience is something in our heads constructed by our brains based on data acquired from our two separate retinas which respond to light focussed into images. The images contain data about the world around us. The images are merged so that our Experience is a single model of the world with an immediate sidedness and is stuff arranged around a point. This might require four independent directions for arranging things (dimensions).
The existence of a viewing point means that the things we experience subtend angles at the point. The separations between things in our experience are angular separations. We label the separation of events from the viewing point as a separation in space but it might truly be a separation in time or almost no separation at all with our experience being arranged tightly around a point. We should keep an open mind on the possibility that within our Experience time and space might be substituted for each other or mixed or even replaced by a way of arranging things or a direction for arranging things that we have not yet described.
There is a paradox in having a view in our brains based on data from an object. The data appears to be separate from the viewing point but the data can still be "seen". Our immediate impulse is to assume that something flows from the object into the viewing point but nothing is in the viewing point. The data is all "out there" in the view in our brains.
Were the data present in our brains without a view there would be no viewing direction and the immediate "sidedness" that characterises our experience would be lost. The existence of "sidedness" means that there is more data in our Experience than that which corresponds to a collection of dots in Experience. Look at this page as an example, we immediately have, relative to the viewing point, up-down and left-right bits of the page in the view whereas if the data that composes the page were the totality of our experience at this moment (ie: if we were the paper and ink or screen) there would be less data present, the immediately evident up-down, left-right etc. being absent.
The extra data in the view does not appear in the page but is a property of the view.
For each dot in Experience there is data from relations to surrounding dots and data corresponding to the separation from the viewing point (depth). We are also unable to have data in our Experience for no time at all, Experience needs time. As we saw above, depth in Experience also needs time.
The space-time geometry of Experience will be explored later.
How can arrangements be in Experience?
How do we have arrangements of events now. How can the arrangement of the letters in a word on this page be an arrangement now? At any instant there is no way to compare whether the “h” in “hello” is to the left or the right of the word on the page. However, we have more than an instant in our Experience and so we can immediately see or hear an arrangement of things because the analysis of the sequence is extended in time.
We have already seen that Experience contains events extended in time as well as space. "hello" begins now and ends then, the word on the page is scanned from left to right and scans are things extended in time. That we have evident arrangements in Experience is a feature of Experience being extended in time yet viewed or heard as if from the observation point now. The arrangement occurs as a succession of angles at our observation point whether it is an arrangement in space or an arrangement in time.
The contents of Experience are arranged around a point. It is as if the contents are simultaneously at a point but "out there" in Experience. We do not observe anything flowing into the point so whatever creates the simultaneous Experience must be "out there". Each patch in the contents of Experience "out there" must be connected simultaneously to every other patch for it to appear as if there is a simultaneous connection at a point.
Continued with Modes of Experience