Are landmark screens mediocre? A breakdown of the 3D motion-sensing interactive solution.

City landmark LED screens often become mere "visual backdrops" due to their single playback mode. However, the 3D motion-sensing interactive solution uses spatial perception and real-time rendering technology to transform giant screens into interactive urban digital sculptures. This solution, centered around millimeter-level motion capture, translates pedestrian gestures into visual feedback on the screen, reshaping the interactive logic of landmark screens.

A three-layer breakdown of the hardware architecture.

The underlying perception layer utilizes a hybrid sensing solution: LiDAR (scanning range 0-50 meters) builds a spatial point cloud model, a TOF depth camera (accuracy ±2mm) captures human skeletal nodes, and a binocular vision camera (resolution 1280×720) enables dynamic object recognition. In one commercial plaza, 32 distributed sensors formed into a circular array can simultaneously track 200 interactive targets within an 80-meter range. The middle computing layer deploys a cluster of edge servers, equipped with a deep learning pose estimation algorithm (an optimized version of OpenPose), to convert sensor data into 18 skeletal joint coordinates, with processing latency under 30ms. The physics engine (NVIDIA PhysX) calculates the interaction logic between actions and screen content in real time, such as triggering a change in the trajectory of on-screen particle flow with a wave of the hand.

The top display layer utilizes a micro-pitch LED screen with a resolution of less than P2.5, combined with a holographic optical film (92% transmittance), creating a floating 3D image. The screen must support a 120Hz refresh rate and progressive scanning to ensure smooth, interactive images. In one landmark project, a 1,200-square-meter curved screen achieved hairline-level motion feedback with a 16K resolution.

Four interactive logic modes:

Gesture control mode: Recognizes six basic gestures, such as pinching and swiping, corresponding to operations such as zooming and switching. For example, a pinch gesture can reduce the 3D model on the screen from 10 meters to 2 meters. Gait Response Mode: Using pressure sensors and visual recognition, pedestrian paths are converted into on-screen light tracks. In one pedestrian street example, hundreds of people walking simultaneously created a dynamic city map projection.

Body Mapping Mode: Using a motion capture suit, dancers' postures are mapped to virtual characters on the screen in real time, with an accuracy of less than 5mm. This is suitable for interactive performances between stage and landmark screens.

Environmental Interaction Mode: Incorporating meteorological data (such as wind speed and light intensity) automatically adjusts the screen's visual effects. On rainy days, the screen creates a converging water effect, reminiscent of real raindrops.

After implementing this solution at a landmark in Shenzhen, the average daily screen interactions increased from 200 to over 5,000, and the duration of visitor engagement increased eightfold. This demonstrates that 3D somatosensory interaction is becoming a key technology for landmark screens to evolve from "display tools" to "city interaction interfaces," enabling giant screens to truly connect with citizens.