ASTR 5110, Majewski [FALL 2019]. Lecture Notes

## Observing Through Earth's Atmosphere: Speckles, Speckle Interferometry and Related Techniques (Plus an Introduction to Convolution, Deconvolution and Autocorrelation)

On this web page I include not only a focus on speckles and speckle interferometry, but other related aspects and image processing techniques relevant to your understanding and completing the speckle interferometry lab exercise.

Some References:

• Sections 4.5.11 and 4.5.12 of Buil, CCD Astronomy, has one of the nicest, pedantic description of the image processing techniques used for speckle interferometry and is highly recommended.

• Chapter 4 of Gordon Walker, Astronomical Observations is also very good.

• Matthew Hoffman's thesis webpage, http://www.cis.rit.edu/research/thesis/bs/2000/hoffmann/thesis.html.

• Hal McAlister's CHARA webpage, http://www.chara.gsu.edu/CHARA/specklejpgs.html, from which I shamelessly copied numerous images.

• Kitchin, Section 2.1.

• Appendix A of Lena has a summary of Fourier analysis.

• Schroeder, parts of Chapter 11 and 16.

## A. The McCormick Observatory Specklescope (Speckle Camera)

##### (Top) The main parts of the speckle camera for the McCormick 26-inch telescope, with the detector not shown (it lives at the focal plane of the microscope objective). (Bottom) The ray diagram for the system.
Camera Parts:

• Filter and Filter Slide: Allows insertion of Δλ = 70 nm wide filter centered at λ = 535 nm to decrease the wavelength smearing of speckles. Since speckles have sizes of λ / D, where D is the telescope diameter, need to limit the λ range to maintain coherent speckle sizes.

• Retractable Knife Edge: Used to focus the telescope to put focal plane in proper place with respect to microscope objective. With the knife edge inserted into the beam, one can also image the wave front at the objective.

• Microscope Objective with Rack and Pinion Focus: Microscope objective magnifies image by 4X so that we sufficiently sample the diffraction-limited Airy patterns of the speckles on the detector.

• Iris Diaphragm: The iris diaphragm acts as a pupil stop to vary the effective entrance pupil of the telescope (allowing us to see the effect of making the 26-inch a smaller net aperture).

• Detector (not shown above): The detector records the images of the speckles. As of 2019 we have two options for this. One is a commercial DSLR camera with a CMOS detecrtor that can be operated via a laptop computer to not only snap still pictures of varying integration time but to give a live continuous video feed. Alternatively, we have a CMOS-based QHY camera that operates in a similar manner.

The figures below show the system in a disassembled and an assembled state with the Canon camera attached:

## B. Viewing the Wavefront

Before discussing speckles in the image plane, it is worth reviewing what the pupil corresponding to a speckled image looks like -- a constantly varying arrival of tilted wavefronts.

You will see these wavefronts in the course of doing the first steps of the speckle lab.

Knife-Edge Test

Your first task will be to do a knife-edge focus of the 26-inch refractor on a bright star.

• First, we need to find a bright star with which to illuminate the pupil (the objective lenses) of the telescope.

• When you look through the "raw" speckle camera (i.e., the camera base WITHOUT the magnifier and videocamera) you will be looking at the pupil of the 26 inch as illuminated by the light of the star. You should see the pupil as a shimmering disk.

• The principle of the knife edge focus is demonstrated below:

##### The knife edge test. The top image shows the relative configuration of the knife-edge (blue) and the converging beam (red). The bottom shows the image of the pupil you will observe in each case. From http://home.att.net/~starastronomy/Astrophotos/Articles/Knife-edge_Focus/knife-edge_focus.html.

• Your goal is to place the focus of the telescope primary at the position of the knife-edge.

• If converging rays from the image of the star meet before the knife-edge, then, as the knife-edge is brought into the beam, the image of the pupil will appear to be occulted from the direction that you are bringing in the knife-edge.

This is the relative configuration shown by the left blue knife-edge above.

• If the converging rays come into focus after the knife-edge, then, as the knife-edge is brought into the beam, the image of the pupil will appear to be occulted from the opposite direction that you are bringing in the knife-edge.

This is the relative configuration shown by the right blue knife-edge above.

• When the focal point of the objective is at the same distance as the knife-edge you cannot see any preferred direction of occultation of the pupil by the knife-edge.

Diffraction Off Knife-Edge

The next thing you should look for is diffraction off of the knife-edge.

• You may only see one or two diffraction fringes.

Viewing the Disturbed Wavefront Directly

Our eyes can only see light intensities (E 2 of the electromagnetic wave), colors and (under certain conditions -- see Haidinger's brush -- or with appropriate filters like most sunglasses) polarization. Our eyes cannot detect phase differences.

However, the technique described here translates phase differences of light into amplitudes that we can see.

After you focus the telescope and look for the diffraction of the knife-edge, leave the knife-edge halfway in the beam and look for the Schlieren pattern.

• The Schlieren pattern should be obvious as the constantly shifting patterns of light and dark patches ("flying shadows").

• The sequence of frames below is from a 30-inch Cassegrain telescope, so of similar size to the McCormick refractor.

Thus, this represents what you should see, except the pupil of a refractor will not have a central obscuration!

##### From Roosen & Meisner, 1999, ASP Conf. Ser. 194, p.303.

• What is creating these patterns?

The following picture illustrates the instrumental set-up that made the pictures.

It is the same as what you will have at McCormick, except the lens and CCD array at the bottom are replaced with the lens of your eye and your retina (you can ignore the "video signal" boxes on the right for now).

##### Adapted from Roosen & Meisner, 1999, ASP Conf. Ser. 194, p.303.

• At the top of the figure on the left we see the wrinkled wavefront that has been distorted by the atmosphere.

It hits the objective of the telescope, which refracts the incoming rays to a focus.

However, because of the wavefront corrugation, different pieces of the wavefront will be focused to different points in the focal plane (making different, virtual speckles in that plane).

• At this point, the knife-edge acts as a spatial filter:

• As may be seen, some portions of the wavefront are tilted to the right (dashed lines in the wavefront and in the ray trace) and some to the left (solid lines in the wavefront and the ray trace).

(In principle, the objective will make different images of the starlight from each the left-tilting and right-tilting segments of wavefront -- each one creating its own speckle image in the focal plane.

But note, with the knife-edge assembly we are not looking at the focal plane of the objective, but instead we are looking at an image of the illuminated objective itself.)

• Those pieces of the wavefront deflected to the left, however, are now blocked by the knife-edge while those deflected to the right are able to pass the knife-edge spatial filter.

• After passing the knife-edge, the rays diverge again and our eye sees them as an image of the pupil.

But those parts of the illuminated pupil that tilt rays to the left are NOT represented, and create shadows in the pupil image (the rays tilted to the right create the bright patches).

Thus the alternating bright/dark parts of the Schlieren pattern are created.

• The shifting patterns show directly the motion of the turbulent atmospheric layers in front of (along the line of sight of) the telescope.

• Note the final image in the sequence of five above -- it shows the sum of 500 Schlieren frames, averaging over the turbulence.

The resulting average pattern on the right should have been evenly illuminated but actually has ended up showing wavefront tilts that come from the telescope optics.

• Here is a Schlieren photograph taken with the 200 inch telescope:

##### From Meinel's Chapter 3 in Stars and Stellar Systems I: Telescopes, eds. Kuiper & Middlehurst.

Were the 30-inch and 200-inch telescopes at the same site taking Schlieren data at the same time, one would expect more Schlieren modulation in the 200 inch image (why??).

However, the 30-inch data were taken at a poorer observing site (Minnesota) compared to Palomar.

This results in about the same number of modulations in the two image (why?).

The Schlieren method can be reproduced in the lab, and is often used to study fluid flows, like the aerodynamics of supersonic/hypersonic flows in wind tunnels, or convective currents off of heat sources.

## C. Brief Review of Speckles

### (Very Brief) History of Speckle Interferometry

Speckle interferometry was developed by the French astronomer Antoine Labeyrie.

• Labeyrie noted that high frequency information in astronomical images was being degraded, but not lost, on short timescales.

• Defining work in the field were photographic observations Labeyrie made at the Palomar 200-inch in the 1970s (because of the low quantum efficiency of film, large telescopes were required to capture speckle images of even bright stars).

##### Speckles of the star Vega made in the groundbreaking observations by Labeyrie on the 200-inch.

An important application of speckle interferometry has been in the study of double stars.
• Speckle interferometry allows the possibility of resolving close binaries near the diffraction limit of the telescope.

• Determining binary star orbits is one of the key ways to obtain stellar masses.

• An example of speckles of binary stars with differing separations:

##### Speckle images from WIYN telescope, from Matthew Hoffman's webpage, http://www.cis.rit.edu/research/thesis/bs/2000/hoffmann/thesis.html.

The determination of the separation and orbit of Pluto's moon Charon was first determined using speckle interferometry:

### The Number of Speckles

Recall this figure from before:

##### From Lena, Observational Astrophysics.

Note that there are several problems with the figure:

• The equation is inverted.

• The number of areas of coherence at the pupil is actually N ~ 4 A / ( π r02 ) ~ ( D / r0 )2.

(The creator of the figure has forgotten that r0 is a diameter.)

• There is a difference between the number of speckles and the number of areas of coherence.

• If the number of areas of coherence (Fried cells) is N the number of speckles is reduced to ~ N / 2 (or to an even smaller number), because each pair (and, actually, any combination) of in-phase points tend to produce only a single detectable speckle. (Think about the Airy function of a full pupil really being the superposition of all the two-point subaperture interference patterns --- i.e. multiple Young's two-slit experiments). So the above math is actually an upper limit to the number of speckles.

## D. The Notion of Convolution

In imaging (or any other kind of data collection) the final image (or other kind of data) that we obtain is affected by numerous sources of degradation/noise. For example, in the case of an image in the focal plane we have:

• photon/Poisson noise

• background noise

• atmospheric/dome seeing

• telescope optics (diffraction, aberrations)

• filtering

• detector optics

• detector noise

Many of these affects can be thought of as a function that convolves with the "truth function'' to create a degraded image.

Removing artifacts of the noise and uncovering the real signal is usually difficult and not completely achievable.

• Removing noise from the real signal is called the inverse problem.

• In imaging this is called deconvolution.

The convolution of two functions (usually denoted by an " * " symbol) is defined by the integral:

##### From Lena, Observational Astrophysics. The shaded curve represents the product of the two functions at one point x in the input function f(x), whereas h(x) will be the integral over the shaded region to produce one single value h at x.
The convolution can be considered as a linear transformation determined by g(x) and applied to f(x).

This transformation represents the behavior of many physical systems that impose a linear operation (e.g., g(x)) on an input signal (e.g., f(x)) to lead to an output signal (e.g., h(x)).

Practically, what is happening is that at each position x the final linear transform h(x) is given by the sum over all x values of the "stationary" function f(x) multiplied by the "roving" function g displaced to be centered on the x position in question.

To understand convolution simply, one can imagine, for example:

• The true image of a distant star (i.e. a point source or 2-D Dirac delta function) convolved with the PSF of the telescope (Airy diffraction), yielding a net Airy pattern.

• The true image of a star (2-D Dirac delta function) convolved with the PSF of the telescope (Airy function) and the wavefront tilts in the atmosphere (complicated displacement or shifting functions, see below), yielding speckles.

• The line spread function of spectrograph optics degrading the true spectrum of a source:

##### From Kitchin, Astrophysical Techniques.
The figure below shows how the observed spectrum is "smeared" by the instrumental profile, causing line features to broaden.

From Kitchin, Astrophysical Techniques.

Mathematically, we have:

where:

• O1) is the intensity of the observed spectrum at λ1,

• T2) is the intensity of the true spectrum at λ2,

• I12) is the response of the spectrograph at a spectral distance of (λ12) from λ1.

• Many image processing filters often used to modify the appearance of images depend on convolutions.

One of the most useful filters, particularly when working with noisy data, is one that smooths the data using a smoothing filter, also called a moving average or boxcar filter (which invokes the possible oblong shape of the kernal in two dimentions, if desired).

As an equation, and written in discrete (rather than integral) form, the boxcar filter calculates a moving average as follows:

The figure below (with the equation above, taken from this website) shows how this is actually a convolution, with the kernal (shown in the middle panel) being a ``tophat function" that is simply the multiplicative constant 1/(2M+1) spanning 2M + 1 pixels (selected as 2M + 1 = 5 pixels at a level of 1/(2M+1) = 0.2 for the illustration below) and 0 elsewhere:

Compare the input and the resulting, smoother, post-convolution version of the data.

Here is a figure (taken from this site, as are many of the following images and equations) demonstrating how a two-dimensional boxcar filter is applied, using a 3x3 kernal:

Here, mathematically, is what the roving filter, or kernal K, looks like for the 3x3 boxcar function:

Again, it is a "tophat" filter -- now in two dimensions -- that is a multiplicative constant of 1/9 in the nearest 3x3 grid of pixels to a given position and a multiplicative 0 value elsewhere.

Here is the discrete, 2-dimensional formulation of how to apply this filter (with a kernal sized [n+1] x [n+1], where n is an even number):

• One can also apply a median filter version of the boxcar, which replaces each pixel by a median, rather than average, value.

This is particularly useful when you have cosmic rays or other spikey noise that would problematically distort averages over large numbers of pixels in the kernal (think about what the effect of one extremely higher than typical value in a pixel will do to a boxcar-smoothed image).

• For fun, here are convolution kernals used in other often-used image processing effects and their application to an image (click on each to see detail; images taken from this site):

 RAW DATA BOXCAR SHARPEN EMBOSS

I hope that this gives you some feeling for how convolutions work.

• The following example shows how an asymmetrical line spread/point spread function can result in a net phase shift as well as a modulation of the brightness function.

##### From Smith, Modern Optical Engineering, Second Edition.

In terms of atmospheric seeing, such phase shifts are seen in the scattering of speckles across the image plane.

In imaging, this may have effects on the positions (astrometry) in images longer than the coherence time.

In spectroscopy, the phase shifts seen above may effect the derivations of such things as the radial velocity of a source based on the measured wavelength shift.

• The point of this section is to get you to start thinking of the various things that happen to wavefronts in terms of a set of convolutions, or operators, acting on the true wavefront.

Later, we will explore deconvolution, or the removal of these effects, when we think we have a good handle on the nature of the kernal function.

## E. More Complicated Speckle Fields

Below is a set of other famous images of bright stars from Labeyrie's experiments:

##### Labeyrie et al.'s (1974, ApJ, 194, L147) famous set of speckle images taken with the Palomar 200-inch telescope. Betelgeuse, Capella and Vega speckle photographs taken with a 20 nm bandpass at 500 nm. The f/ratio was 200 and the exposure times were 0.01 seconds. Note each image shows a different speckle structure. Adapted from Walker, Astronomical Observations.

• The image of Vega (α Lyrae, right) shows the speckle pattern of a star whose angular diameter is much smaller than the Rayleigh limit of the telescope.

The speckles are sharp, reflecting the diffraction limit of the telescope.

• The image of Betelgeuse (α Orionis, left), a supergiant star, shows speckles that are bigger than for Vega.

In this case, the angular size of the disk of Betelgeuse (0.06 arcsec) is larger than the diffraction limit of the telescope, and the speckles resolve the star.

This example shows another application of the speckle technique -- measuring stellar diameters.

• Capella (α Aurigae, center) is a close binary star with angular separation larger than the Rayleigh limit.

The speckles all appear double.

### Binary Stars

If binary stars have an angular separation smaller than the isoplanatic angle, all of the speckle patterns are correlated and seen double.

##### From the CHARA website, http://www.chara.gsu.edu/CHARA/specklejpgs.html.

• We have the convolution of the actual image of the double star with the speckle "function" or speckle pattern.

• The "isoplanatic patch" is the angle on the sky over which the turbulence pattern is statistically the same.

• If the double star is entirely contained within the envelope of the isoplanatic patch, then the speckle function will be close to identical for each star.

• In principle, can measure separation and position angle of double stars by measuring separations of the speckles.

• But we can apply more sophisticated techniques, described below.

The two following images from Hal McAlister's CHARA group shows the appearance of the speckle patterns for a "wide" and a "close" binary.

## F. Information Extraction From "Lucky Shot" Imaging

As mentioned above, we can recover information near the diffraction limit from "instantaneous" (i.e., short exposure, t < ~ 0.01 s) images.

• This was well-known to old-time, skilled visual observers (like Burnham, van Biesbrock, etc.) in the nineteenth and early twentieth centuries who were able to resolve very close binaries by eye, and measure their separation with nothing more than a filar micrometer.

See this website for a nice description on the use of the filar micrometer.

• These observers took advantage of transient moments of exquisite seeing (large r0 ) to "capture" the visual impression of the resolved binary.

• This was easier with D ~ r0 , to get one "speckle".

• In the modern regime where data are recorded at high speed, one can pick out those moments with the best, largest r0 and register and coadd only these.

--> Obviously, the more "picky" one is, the fewer frames you have to choose from and the longer one needs to observe to get a required number of frames.

• If D > r0 and you are working with small angular source images, then you can "pick apart" the speckle pattern and coadd individual speckles.

• If the image is larger than the isoplanatic angle, you can pick apart image sections with the sharpest resolution, and composite them back together.

• The following images represents this kind of work with a 12-inch Meade Schmidt-Cassegrain and a video-camera, by Ron Dantowitz, as presented in the August 1998 issue of Sky & Telescope.

##### The Shuttle docked at the Russian Mir station in 1996.
Some beautiful "lucky imaging" results can be found here.

A related technique that has been implemented on some telescopes is "fast shuttering".

• The following telescope arrangement includes a pick-off mirror sending the image of a bright star to a fast readout camera. This camera is connected to a computer and together they monitor:

• those moments when the PSF is sharpest.

• the mean centroid position of the star (image wander).

• When the seeing is good, the shutter opens, sending the full field image to the science CCD camera.

Before reaching the focal plane, the image position is adjusted by a tip-tilt mirror or lens, to make sure all passed data are registered.

• This technique only really works well when the seeing is good already.

If the seeing is poor, the shutter hardly ever opens, depending on imposed limits.

• My impression is that fast shuttering is not being used as much these days (and never really achieved the popularity expected), because more modern telescopes at choice sites already exceed the seeing potential of this method when used on older telescopes.

## G. Information Extraction From Autocorrelation in the Image Domain

In some cases it is worthwhile searching for occurrences of specific patterns within an image, i.e. shape recognition, or compare the degree of similarity of two images.

• A number of astronomical programs utilize this method to do image classification of astronomical sources (e.g., star/galaxy separation, or galaxy typing).

A simple way to do this is by undertaking a correlation, by shifting one image with respect to another and measuring the degree of similarity of the two images at each shifted position.

Buil's book (referenced at the top of this webpage) gives a simple example of a cross-correlation:

• On the left is a test image and on the right is the pattern to be cross-correlated.

• The result of the cross-correlation is as follows:

• For good measure, the convolution of the cross-correlation with the test pattern looks like this:

And the difference of this with the original image is this:

Below is another example of a cross-correlation showing a commonly used methodology for determining the Doppler shift (radial velocity) of a source by cross-correlating against a shifting template kernal that has notch features at known rest-wavelengths of expected spectral lines. On the right is shown the resulting cross-correlation ``power spectrum'' that (in this case, where the kernal features are shown as positive and the actual astronomical lines are in absorption) is at a minimum at the shift corresponding to the velocity (Vstar).

More commonly this is done by cross-correlating with a kernal that is the spectrum (real or synthetic) of a star with known Doppler velocity and the power specrtrum will be a positive function with a peak at the velocity difference between the kernal and target sources.

##### Finding the radial velocity of a star by cross-correlating against a "rest-frame" line template. From http://nexsci.caltech.edu/workshop/2018/presentations/20180724-SaganWorkshopRV-SharonXuesongWang.pdf =.

A way to search for repeated patterns within a single image is to cross-correlate an image against itself.

This is known as autocorrelation.

It is a particularly effective way of picking out the repeated double star patterns in speckle images of binaries.

• The figure below shows 1-D cross-sections of a double star image through the center of the two stars:

• Imagine shifting pairs of cross-sections against one another at different shifts.

It is typical to point-by-point multiply the two functions against one another as a measure of the correlation.

Only in three shifted positions are there non-zero correlations (cases B, D and F).

In case D the correlation is the strongest since the true components are exactly superposed.

• The final autocorrelation function looks like this:

• One can measure the distance between the two components (stars) by one-half the distance between the two secondary peaks.

• Below is an actual example (in two dimensions) of an autocorrelation of a speckle image of a double star.

##### From the CHARA website, http://www.chara.gsu.edu/CHARA/specklejpgs.html. Note that the arcsecond scale of the left and right images are not the same.

• And finally, here is the promised autocorrelation of the Pluto-Charon speckle images from Beletic et al. (1989):

## H. Solving the Inverse Problem with Deconvolution: Information Extraction in the Frequency Domain

For the convolution of two functions we discussed above you should know that the inversion of the equation

is not possible in a direct way.

However, it can be done in Fourier, frequency space using Fourier transforms.

• Recall that the Fourier transform (in one dimension) and its inverse are given by:

The notation in this case is that F(s) is the Fourier transform of f(x), but an alternative notation is to denote the transform by a tilde (as below).

(In practice, since we sample our data at discrete intervals, and not over infinity, we always approximate the Fourier transforms by discrete forms that replace the integrals by sums; see discussion in Lena or Kitchin or many other sources. These computations are now generally done by the so-called Fast Fourier Transform -- FFT -- algorithm).

We have already discussed one important Fourier transform pair in this class, that between an entrance aperture and its diffraction pattern, which is its focal plane conjugate, the "point spread function":

##### From this site.
Technically, the right-hand function is not the Airy function, but the so-called sinc function. We observe the modulus (square) of this of course (shown below).

##### From this site.

We can take advantage of a remarkable feature of convolutions:

• The convolution theorem: The convolution of two functions corresponds to the multiplication of their Fourier transforms.

• Thus, in the example above we had the line spread function convolved with the spectrum of a source:

Then, taking Fourier transforms, we have:

and the true spectrum (or whatever) can be deconvolved by inverting the latter equation and taking its Fourier transform:

• Typically, we work with the power spectra

where

Let's revisit the Labeyrie bright star images, now with their Fourier transform power series.

##### Labeyrie et al.'s (1974, ApJ, 194, L147) famous set of speckle images taken with the Palomar 200-inch telescope, with their Fourier transforms below. From Walker, Astronomical Observations.

• The Fourier transform of the (speckled) image of Vega gives the angular frequency response of the telescope to a point source.

Notice that it is broad, showing that in Fourier space, there is power at high frequencies (frequencies needed to make up a sharp PSF).

• The Fourier transform of Betelgeuse is smaller, because the source is resolved with a finite diameter, and therefore there is a high frequency cutoff (because the source is no longer a POINT source, we do not need high frequency Fourier components to make a "pointy", sharp PSF in this case).

• The Capella image is the convolution of a double source image (each star unresolved) with the PSF of the telescope.

The Fourier transform power series shows a classical "two-slit diffraction pattern" due to the double stars multiplied by the Fourier transform of the PSF of the telescope, which gives an envelope that is the same size as that of Vega.

The spacing of the fringes is inversely proportional to the stellar separation, and the orientation indicates the position angle of the binary.

• Here is a more complicated, but interesting Fourier transform of a speckle image of the star system η Orionis:

##### McAlister's (1977, Sky and Telescope, 53, 346) image of speckle images taken with the Mayall 4-m. From Walker, Astronomical Observations.

The fine set of fringes came from the previously known binary, A-B, with 1.5 arcsec separation.

But the broad set of fringes showed that there is a third star in the system, a previously unresolved binary of star A that has only a 0.044 arcsec separation in the images.

Nice properties of working in Fourier space are:

• the removal of the phase differences caused by the speckling (the FT of a Dirac delta function is the unity function).

• the invariance of the "shift" of the Fourier images in the frequency domain even with shifts in the spatial domain:

The latter means we can coadd the FTs of many speckle images and coadd them directly for improved S/N.

In the speckle lab we will take advantage of both of these properties of working in frequency space.

We will also apply the deconvolution theorem to divide out the effects of the telescope pupil MTF (single star PSF), which should have the effect of making the fainter double star fringes easier to see.

Unless otherwise attributed, material copyright © 2005,2007,2009,2011,2013,2019 Steven R. Majewski. All rights reserved. These notes are intended for the private, noncommercial use of students enrolled in Astronomy 511 and Astronomy 5110 at the University of Virginia.