Computational Design of Diverse Morphologies and Sensors
for Vision and Robotics

CVPR 2024 Tutorial

Where: Summit 344, When: June 18 (09:00 - 12:30, half-day), 2024

A clever morphology can make exhibiting intelligence easier. The video shows a fish that appears to be gracefully swimming. The fish is dead. It is the clever morphology of its body that does most of the work in interaction with the surrounding environment (water stream) and makes control and other aspects effortless. Can we design the physical and perceptual morphology of our embodied agents just as cleverly?


Animals exhibit a wide variety of morphologies and sensors, believed to have appeared through billions of years of evolution. Common examples relevant to vision include differences in pupil shapes, the positioning of eyes, various types of eyes, and a varying level of multimodality across animals. Such adaptations are hypothesized to be instances of the so-called Ecological Theory, which posits a strong connection between the specifics of vision and the environment surrounding the agent, its objectives, and its body. How can we replicate this diversity and achieve adaptive design in robotics and vision systems?

In this tutorial, we discuss 1) alternative forms of visual sensors that can be useful for real-world robots and 2) computational approaches to robot and vision design that can achieve the goal of adaptive design automatically, effectively, and efficiently. The tutorial covers topics in sensing, control, simulation, optimization, and learning-based design for various rigid and soft robots and visual sensors. The material is drawn from state-of-the-art breakthroughs in the field and insights from other disciplines. This material is accessible to individuals of all backgrounds and levels of expertise.


09:00 - 09:05 💫 Introduction Amir Zamir
Visual Morphology:
09:05 - 09:20 👁️ Diverse Perceptual Systems in Nature Amir Zamir
09:20 - 10:00 👁️ Solving Vision Tasks with Simple Photoreceptors instead of Cameras & How to Design Them Andrei Atanov
10:00 - 10:30 👁️ Motivations from Biological Vision: Measures and Models of Visual Acuity in Oceanic Animals Sönke Johnsen
10:30 - 11:00 ☕️ Coffee Break
Physical Morphology:
11:00 - 11:30 🦾 Preliminaries and "Classical" Approaches Andrew Spielberg
11:30 - 12:00 🦾 Generative AI and Interfacing with the Real-World Andrew Spielberg
12:00 - 12:30 🏭 Neural Concept: Computational Design for Industry Pierre Baqué,
Neural Concept


Andrew Spielberg
Harvard University


Thanks to for the webpage format.