Rolling Shutter Image Sensors - The Finer Points
When we use an image sensor to 'capture' an image with a digital camera, we generally expect (and usually get) a frozen moment in time.
The image we see is a result of the image sensor reacting to light from the scene focused by the lens. However the processes of converting
the photons that make up an image to electrical signals (conversion) and transmitting those millions of electrical signals from the image sensor (readout)
are not instantaneous. And when a scene we wish to capture is changing as fast (or faster) than the sensor conversion and readout rates, the methods
that the image sensor uses for these processes become important to the ultimate image quality that can be achieved.
One of the more common processes for image conversion and readout is known as the "rolling shutter" method. This method,
shown in Figure 1, employs two signals: RESET and READ to define the length of the pixel exposure time during image capture.
Figure 1. - Rolling Shutter image capture
The RESET signal affects all of the pixels in a row and essentially puts the pixels in a state to convert light intensity into an electrical signal. The image
sensor circuits cause this signal to be sequentially applied to each row in the image sensor in order to capture a full frame of image data. At some fixed
interval after the RESET signal, a READ signal is applied to all pixels in a row causing the electrical signals from each pixel in a row to be transmitted
from the image sensor circuits. The READ signal is applied sequentially at the same rate as the RESET signal, producing an effective window of exposure
that propagates (or 'rolls') over the image sensor. It is easy to see that the effective exposure time of an image capture with this method is determined by the
separation (in time) of the RESET and READ signals.
Here's where things get interesting. Because the RESET and READ signals that make up the sensor exposure time take a finite amount of time to propagate
over the image sensor, a rapidly moving image can change in the amount of time it takes for the exposure window to go from one part of the image sensor to another.
The QuickTime video shown below is a good example of this phenomenon:
"Golf Swing #1" : ACSion 2462CL; 1280 X 720 ;Captured @ 60fps; Playback @ 6 fps
The golf club head in the picture is moving quickly through the scene and the image exposure time is short,
so it appears that golf club is bent because the shaft of the club has changed position between the time that the image sensor
exposure window captured the top of the club and the bottom. But wait a minute - something doesn't look right.
If you look at the QuickTime video, you'll notice that the shaft of the club appears to bend in a way that no
club could ever bend without violating the laws of Physics. This is where we have to examine the finer points of
rolling shutters-
The weird behavior of the golf club in the video above can be explained by the sequence of diagrams shown below that
are graphical representations of the motion of the club.
T = 0 T = 1 T = 2 T = 3 Final Image
As the exposure time window (hatched area) rolls down the image, the shaft of the club is moving to the right so that in each consecutive
time window the club shaft has noticeably moved. The final image integrates this motion together to give the impression that the club is bending.
So it appears that the DIRECTION of the rolling shutter on an image sensor is just as important as the duration of the exposure window. We
can test this fairly simply by reversing the direction of the shutter on this sensor by turning the camera upside down. This has the effect of
making the exposure window roll from bottom to top instead of top to bottom. The QuickTime video shown below does just that - the camera
was turned upside down and another golf swing was captured. Take a look at the bend in the golf club with this video compared to the one above.
"Golf Swing #2 : ACSion 2462CL; 1280 X 720; Captured @ 60fps (Camera Upside Down); Playback @ 6 fps