Providing Real-Time Data
For and About Subjects Within a CameraÕs View
Michael Naimark
University of Southern California
Los Angeles, CA
Abstract
This
invention is a specific enhancement of the Òtally lampÓ or Òcamera indicator
lamp,Ó the red light visible on the front on virtually all video cameras when
they are active. The specific enhancement is to make the light only visible to
subjects when they are actually within the view of the camera, which may be called
a Òtrue tallyÓ lamp. It can be achieved by placing the tally lamp behind the
lens, on-axis with and the same size as the camerasÕ sensors, and pulsing on
out-of-phase with the sensorsÕ activity. A preferred embodiment is described,
based on creating through known means a uniform light source that closely
resembles the camera sensors such that it can easily mount like one. Additional
features include modulating the frequency of the true tally lamp to appear
flashing as well as continuous and modulating the color of the true tally lamp,
both enabling additional information to be conveyed to the subjects within the
frame. Finally, the invention described here is a step toward an integrated
on-axis projector/camera, or ÒprocamÓ system, a
lively area today in the computer vision field as a means for enhancing both
cameras and projectors.
Background
A
Òtally lampÓ is a light on the front of a video
camera that lets subjects know when the camera is active. Its original use was
in multi-camera video studios such as newsroom sets, so the commentators could
know at which camera to look as the director in the control room switches
between cameras. Tally lamps on studio video cameras are typically big and red.
The tally lamp of the professional studio camera migrated into
todayÕs consumer video camcorder as a standard item, where it is less prominent
(e.g., a small red LED) and can be disabled via the cameraÕs settings menu.
Many consumer camcorder companies now refer to them as the Òcamera
recording lamp.Ó They are unobtrusive but visible to anyone looking for them.
ItÕs noteworthy that motion picture
film cameras historically do not have tally lamps. In professional movie
production, they are not important because everything is under the control of
the director.
But in todayÕs world, where video
cameras are cheap and ubiquitous, issues of control are not so simple. How
important is it that subjects know when theyÕre being recorded or broadcast? Is
it useful to let your daughter know when youÕre recording during her birthday
party? Is it a simple courtesy to let your classmates know when youÕre
recording a seminar? And are there more serious, sometimes legal, implications
to not letting strangers know when youÕre recording them in a public place? In
our new world, in which surveillance and privacy have become flashpoints for
critical debate, cameras and their technologies are under scrutiny, offering
both challenge and opportunity. For example, one well-supported area of current
research is around technologies that automatically detect and hide faces from
public surveillance cameras. The face data is actually stored under digital
Òlock and keyÓ and only accessible by subpoena if required.
Another possible opportunity lies in
providing more feedback from the camera to its subjects, like tally lamps. But
tally lamps have a problem: they only indicate when the camera is active but
give no indication where the boundaries of the cameraÕs view may be. Potential
subjects in the general region in front of a camera cannot know if theyÕre
actually in the view or not. And with zoom lenses, even if the camera is
pointing near the subject, itÕs impossible to know the size of the field of
view.
Disclosure
The
solution is to place the tally lamp behind the lens, on-axis with and the same
size as the camerasÕ sensors, and pulsing on out-of-phase with the sensorsÕ
activity. If the tally lamp is a uniform light source whose surface dimensions
match those of the sensors, the light refracts through the lens and is visible
in the exact field of view seen by the sensors. It would appear to the subject
as a light coming out of the lens. One might call this a Òtrue tallyÓ lamp.
The
true tally lamp light source would need to be on-axis with the sensors. Many
video cameras today have three camera sensors, for red, green, and blue
(sometimes even a fourth) all on axis through the use of beam-splitting prisms
or half-silvered mirrors. The true tally lamp light source would simply be an
additional element using these known means for positioning multiple on-axis
elements.
In
order for the true tally lamp source to not interfere with the sensors, it
would have to be pulsed on out-of-phase with the sensorsÕ activity. When it is
on, it needs to come on quick enough to have no affect on the functional duty
cycle of the sensors. When it is off, it needs to be completely off such that
the sensors pick up nothing from it. Video cameras have two ÒfreeÓ
opportunities to insert a pulse of light with no affect on the sensorÕs duty
cycle: during the horizontal blanking and during the vertical blanking. For
American video standards, horizontal blanking is about 1.3 milliseconds every
16 milliseconds and vertical blanking is about 10 microseconds every 60
microseconds. Given our eyesÕ retention ability, such pulsing, if of an
appropriately strong intensity, would be more than adequate to see and could
even appear continuous. Additionally, the camerasÕ sensors could have shorter
duty cycles, for example, itÕs common for Òprogrammable shuttersÓ to be as fast
as 1/1,000 second, leaving the remaining 99.9% of the duty cycle to pulse the
tally light source.
Preferred Embodiments
The preferred embodiment would be
based on making, through known means, a thin true tally light source of uniform
light distribution out of a light-emitting semiconductor such as LEDs or OLEDs. It would be fast
enough to pulse adequately, be sufficiently bright to be easily visible through
the lens in daylight, be energy efficient, and produce no heat or other energy
leakage that may affect the sensorsÕ performance. It could be made, through
known means, to resemble camera sensor elements, such as the red, green, or
blue CCDs. The more it resembles these sensor
elements, the easier it is to mount like one.
The mounting would be through the
known means of using half-silvered mirrors, or more commonly, prisms, to split
the beam of light (which is the same in both directions) in such a way to keep
all elements on axis. For simple Òone-chipÓ (CCD sensor) video cameras, such a
beam splitter would allow a second chip to be added and for Òthree-chipÓ video
cameras, such a beam splitter would allow a fourth chip to be added.
The simplest preferred embodiment
for pulsing the true tally lamp is to pulse it on during the vertical interval,
since it is a thousand times longer than the horizontal interval, as well as
being a larger percentage of the duty cycle. It would pulse at 60 hertz and be
on for one-sixth of the duty cycle, resulting in, worst case, a partially
noticeable flicker (depending on what region of oneÕs field of view itÕs in).
In a preferred embodiment, the true
tally light would complement rather than replace a regular tally lamp, since
they provide different but complementary functions: Òcamera activeÓ and Òcamera
pointing at me.Ó If the tally light is red, the true tally light may be a
different color, but red is probably still preferred.
Optional Features and
Future Applications
The simplest additional feature for
a true tally light is to modulate its frequency, which requires no change in
hardware. Hidden during the vertical interval, it will appear almost
continuous, allowing it to flash at human-detectable speeds, like once per
second. At the very least, information could be conveyed to the subjects by
having the light appear continuous or flashing, and additional information if
the light flashes slowly or quickly. For example, this information could be to
distinguish when the camera is active or in standby mode.
A more ambitious additional feature
would result if the true tally light could emit more than one uniform color,
for example, by constructing it of red, green, and blue LEDs.
Even more additional information could be conveyed to subjects if the true
tally light could change color.
Finally, a very ambitious additional
feature would result if the true tally light could emit more than one uniform
color and be modulated on a pixel-by-pixel basis. For one thing, it would allow
specific colors to be conveyed anywhere in the cameraÕs view on a
pixel-accurate basis. It could, for example, Òsend a messageÓ to only one
particular subject (identified through known computer vision techniques).
But even more ambitious, such a
light source could double as a projector, perfectly on-axis with the camera.
Such projector/camera configurations, when used in phase as a closed-loop
system, have numerous applications for expanding both camera capabilities and
projector capabilities. In computer vision, these systems even have a name, Òprocams,Ó and are currently a lively source of activity
(e.g., see www.procams.org). Procam projection can be
both low-energy, invisible to the naked eye but useful to project detectable
structure, for example, to help determine depth, or
high-energy and visible to the naked eye, useful where the camera provides
feedback for the projection itself, for example, to keep the projected frame
aligned when the projection screen is moving in real time.
Some additional in-phase applications
are:
- as a light source for retro-reflective ÒmarkersÓ used
in motion capture;
- if the light source is strong enough, as the means to
actually illuminate the scene and subjects, either as a flash or continuous
light. On-axis illumination of subjects results in absolutely no shadow
effects.
No fully integrated procam system currently exists.
Additional Features
1. Single integrated sensor
+ light source chip
The
original disclosure assumed that sensor chips and light source chips must be
different chips, placed on axis through the use of known means such as prisms
and partially silvered mirrors. There is no reason why a single chip cannot
have both sensors and light source integrated together, for example, through
micro-optical engineering. Such engineering is common for integrating different
color sensors on the same chip. The advantage of an integrated sensor + light
source chip is size: just like single-chip color cameras, no additional prism
or mirror optics is required.
2. Integrated focusing lens
for the light source chip
It
would be most efficient for the light source to ÒaimÓ all of its light out
through the camera lens with minimum scattering in other directions. This can
be accomplished several ways, such as external focusing optics in front of the
light source chip or integrated in the chip itself.
It
may, however, be important to minimize the effect of stopping down the camera
lensÕ aperture on the view-ability of the tally light, for example in bright
sunlight. This can be accomplished by further focusing the light source chip primarily
through the center of the camera lens.
3. Use of Class IIIa lasers
ÒClass IIIaÓ lasers emit no more than 5 milliwatts of power and are the class of lasers used in common laser pointers. These laser
pointers have proven to be safe to both cameras and human eyes, and can be
found for sale on the Internet for one or two dollars apiece. The core of these
lasers is a very tiny, very inexpensive, solid-state laser emitting diode.
Using
laser-emitting diodes as the light source may simplify the design for this invention
due to the coherent nature of laser light. For example, the need for the tally
light source to be the same size as the camera sensors may not be necessary.
Clever placement of the tiny emitter may also allow for smaller alternatives to
sensor-size prisms or half-silvered mirrors.
Class IIIa lasers work very well when used in the Òopposite
direction:Ó when a laser pointer is aimed directly into the lens of a digital
camera, the entire field of view is temporarily neutralized, or Òzapped,Ó at
distances as far as several hundred meters. [See Naimark,
M. (2002). How to Zap a Camera. (self published on www.naimark.net).] ItÕs reasonable to assume that this process may
be generally reversed.
|