7.1 Optical sensing and ShadowSense™: locating shadows

Slide 42 of 157 · Optical sensing and ShadowSense™: locating shadows

Welcome to chapter 7: Optical sensing and ShadowSense: locating shadows. I'm Diggy Breiling with FPD.DEV, The Future of Display and Systems Development. Our goal is to explain the ShadowSense basics: infrared emitters on three sides, typically six sensors on the fourth, and a touch located from its shadows. ShadowSense is a trademark of Baanto International Limited, and the technology and related intellectual property belong to Baanto. Our graphics are teaching illustrations.

Slide 43 of 157 · Moving touch, moving shadow

Let's, let's start with the light. Six sensors are shown along the top edge, with sensors one and six at the corners. Infrared LEDs sit on the other three sides. Each yellow fan starts at the selected LED. The denser LED array here is illustrative. As the touch object moves, watch the shaded shadow sweep across the receiving edge. The dashed paths show light interrupted by the object. Changing the active LED provides another view of the same touch, giving the controller more information about its position.

Slide 44 of 157 · Receiving fields of view

Now look at the sensor fields of view. Sensors one and six look inward from the corners. Baanto's illustrated arrangement labels these as ninety-degree views and the four intermediate views as approximately one hundred and seventy degrees. The purple regions show where a sensor can receive light within the sensing plane. They do not show light emitted by a sensor. A measured shadow direction lies within that field of view. Confirm the actual acceptance angles and usable coverage for the chosen design.

Slide 45 of 157 · Multitouch: moving objects, moving shadows

ShadowSense™ uses analog optical sensors to measure light and shadow directions. It does not need a camera image of the hand. Here, objects A and B move independently in the same sensing plane. Watch how each shadow moves, and how the two shadows can overlap from one light-source position. Selecting another LED changes the view and provides more information to help separate the contacts. The controller combines observations over time to track them. Shadow geometry and density can also help classify objects and reject unwanted contacts, depending on configuration and conditions. This animation illustrates the optical geometry; it does not reproduce Baanto's tracking algorithm.

Slide 46 of 157 · What is distinctive about ShadowSense™ (2 of 2)

ShadowSense™ is designed for strong ambient-light rejection. Baanto specifies operation up to full sunlight at 10,000 foot-candles (about 108,000 lux) for ShadowSense™ Pro. This makes it a useful option where high ambient light overwhelms conventional IR touch frames, causing missed or false touches. It does not mean every IR frame fails outdoors. Compare the intended assemblies under the same solar exposure, incidence angles, temperatures and contamination conditions. A lux value alone does not describe the infrared loading at the receiver; sunlight tolerance and screen readability are separate requirements.

Slide 47 of 157 · Full-array scanning and changing shadow views

Let's, let's follow the light around the frame. First, six LEDs turn on slowly, each giving a different view of the same two moving contacts. The colored shadows move with the objects and change direction when the emitter changes. Watch the six sensor-path indicators respond. Now the pattern visits all fifty-four illustrated LEDs, holding each for just under half a second. The progress strip marks the positions visited. Then the same pattern repeats faster. Combining these different views helps separate overlapping contacts, maintain their tracks, and assess object characteristics. Our pattern, path indicators and display speeds are illustrative. Actual sensing uses analog information and design-specific processing. Baanto also describes two-megahertz modulation within illumination windows. That carrier, emitter selection, complete scans and host reports have different rates.

Slide 48 of 157 · Up to 10,000 times per second: what is counted?

Here is the rate distinction. We say up to ten thousand acquisitions per second in this teaching example. At that upper rate, each acquisition takes a hundred microseconds. If a complete scan needed a hundred sequential acquisitions, it would take ten milliseconds, before extra overhead. Those numbers are an example, not a verified ShadowSense™ schedule. Baanto advertised up to ten thousand frames per second in a 2010 technology announcement, while a separate SDW656 report measured a ten-point-eight-six-millisecond polling period. We must specify what is being counted and confirm the intended product. USB reports are another boundary.

Slide 49 of 157 · Eleven LED views: one touch, then two

Let's, let's start with just one touch object. The LEDs are numbered from one at the upper left, down the left edge, across the bottom, and up the right edge to fifty-four. We compare eleven selected LEDs, each with its own identifying color. Only one LED is shown active at a time. Follow its colored rays to sensors S-one through S-six. A solid path reaches the sensor. Where the object blocks light, the path beyond it becomes dashed gray. The highlighted map row uses that LED's actual frame number. C means clear and X means blocked. You can pause the video at any time to check the table. Trace each LED-to-sensor path and verify that its cell correctly shows clear or blocked. Now the object moves, and the blocked paths change. Then we add a second touch. In our illustrative example of up to ten thousand acquisition reads per second, combining these views builds a detailed, continuously updated estimate of each touch location and helps track multiple contacts. These reads are individual observations, not necessarily complete position updates. Colors and binary path states simplify invisible infrared and richer analog sensor data.

Slide 50 of 157 · Custom frames and integration (1 of 2)

Changing the aspect ratio can change line-of-sight geometry, sensor distribution, mechanical tolerances and electronics placement. A narrow panoramic display is not merely a scaled standard monitor. Ask for coverage at corners and edges and for the smallest detectable object under specified conditions. FPD.DEV's touch-development work includes direct Baanto coordination. A custom program still needs design-specific agreement on frame geometry, electronics, firmware, host interface and acceptance evidence. Evaluate occlusion, cleaning, water, ambient light and infrared coexistence using the final assembly. Retain raw diagnostics where available so a failed interaction can be traced to sensing, tracking or application handling.

Slide 51 of 157 - 1300 nm illumination and NVIS

FPD.DEV can work with Baanto on a custom ShadowSense design using 1300 nanometer LEDs and wavelength-matched receivers. For conventional image-intensifier night vision goggles, 1300 nanometers is beyond the long-wavelength response band. That is the reason for selecting this wavelength: to keep the touch illumination outside the goggles' useful sensitivity. The relevant aircraft lighting standard is MIL-STD-3009. Its predecessor specifications are MIL-L-85762 and MIL-L-85762A. Tables IV and V give the relative spectral responses for Class A and Class B NVIS systems, from 450 through 930 nanometers. The table endpoint is not a universal cutoff for every night vision device, and the standard does not certify a design simply because its LEDs operate at 1300 nanometers. This requires matched emitters, detectors and optical windows. Verify the complete emitted spectrum and compatibility with the specified goggles. Short-wave infrared cameras can detect 1300 nanometers. This is a custom design option, not a specification for every ShadowSense frame.

Slide 52 of 157 · Apply the idea

Why does up to 10,000 acquisitions per second not necessarily mean the same number of complete touch reports? Pause here and consider your answer. An acquisition can be only one illumination step or channel observation. A complete scan may require many steps, followed by contact extraction, tracking and host reporting. In our hypothetical 100-step example, 10,000 step acquisitions per second yields at most 100 full scans per second before extra overhead. Define the rate boundary and verify the actual implementation.

Slide 53 of 157 · Chapter 7 complete

For a closer look at touch emitters and night-vision compatibility, continue with our NVIS Display Engineering course. Next is Chapter 8: Surface acoustic wave: listening to a disturbance. I'm Diggy Breiling with FPD.DEV, The Future of Display and Systems Development. Thank you for joining me.

An acquisition can be only one illumination step or channel observation. A complete scan may require many steps, followed by contact extraction, tracking and host reporting. In our hypothetical 100-step example, 10,000 step acquisitions per second yields at most 100 full scans per second before extra overhead. Define the rate boundary and verify the actual implementation.

Accuracy

Closeness of a reported coordinate to a defined reference position.

Precision

Repeatability or spread of results under specified conditions.

Resolution

Smallest coordinate increment or representable change; not equivalent to accuracy.

Latency

Time between explicitly defined input and output events.

PCAP

Projected capacitive touch; patterned electrodes sense electrical coupling changes.

Self-capacitance

Measurement of an electrode relative to an electrical reference.

Mutual capacitance

Measurement of coupling between transmit and receive electrodes.

Baseline

Reference measurement used to identify changes associated with touch.

SNR

Signal-to-noise ratio; specify the signal, noise measure and conditions.

SAW

Surface acoustic wave sensing through controlled propagation on a surface.

APR

Acoustic pulse recognition; a touch-generated mechanical response is used to locate contact.

DST

Dispersive signal technology; a bending-wave sensing implementation.

EMR

Electromagnetic resonance, used by compatible pen digitizers.

Haptics

Tactile feedback produced by the system.

HID

Human Interface Device class/protocol family; the descriptor defines report interpretation.

Report descriptor

Data describing the structure and meaning of HID reports.

Contact ID

Identifier used to distinguish and track a contact.

I2C

Two-wire synchronous bus; raw I2C communication does not itself implement HID.

SPI

Synchronous serial peripheral interface, with device-specific protocol behavior.

UART

Asynchronous serial hardware; electrical levels and framing must be specified.

Optical bonding

Joining optical layers with a transparent material to manage the interface.

Parallax

Apparent relative position shift caused by viewing separated planes from an angle.

NVIS

Night Vision Imaging System; pronounce 'en-vis' in this course.

Qualification

Evidence against a defined set of requirements for an identified configuration.

Validation

Evidence that the system satisfies its intended use.

Verification

Evidence of conformance to specified requirements.

Ghost touch

Reported contact that does not correspond to an intended physical touch.

Palm rejection

Classification or suppression of unintended hand contact.

On-cell / in-cell

Touch structures at a display-cell surface / integrated within the display structure; verify supplier construction.

Touch plane

Geometric region in which the sensing system detects an object.