Description

An accessible, authoritative, and up-to-date computer vision textbook offering a comprehensive introduction to the foundations of the field that incorporates the latest deep learning advances.

Machine learning has revolutionized computer vision, but the methods of today have deep roots in the history of the field. Providing a much-needed modern treatment, this accessible and up-to-date textbook comprehensively introduces the foundations of computer vision while incorporating the latest deep learning advances. Taking a holistic approach that goes beyond machine learning, it addresses fundamental issues in the task of vision and the relationship of machine vision to human perception. Foundations of Computer Vision covers topics not standard in other texts, including transformers, diffusion models, statistical image models, issues of fairness and ethics, and the research process. To emphasize intuitive learning, concepts are presented in short, lucid chapters alongside extensive illustrations, questions, and examples. Written by leaders in the field and honed by a decade of classroom experience, this engaging and highly teachable book offers an essential next-generation view of computer vision.

• Up-to-date treatment integrates classic computer vision and deep learning
• Accessible approach emphasizes fundamentals and assumes little background knowledge
• Student-friendly presentation features extensive examples and images
• Proven in the classroom
• Instructor resources include slides, solutions, and source code

Table of Contents

Preface……………………………………………………………………………………………………………………………………………… xxi

Notation…………………………………………………………………………………………………………………………………………. xxvii

• The Challenge of Vision……………………………………………………………………………………………………………………. 1

I         FOUNDATIONS…………………………………………………………………………………………………………………………….. 33

• A Simple Vision System………………………………………………………………………………………………………………….. 35
• Looking at Images…………………………………………………………………………………………………………………………… 53
• Computer Vision and Society………………………………………………………………………………………………………… 67
• IMAGE FORMATION………………………………………………………………………………………………………………….. 75
• Imaging……………………………………………………………………………………………………………………………………………. 77
• Lenses………………………………………………………………………………………………………………………………………………. 89
• Cameras as Linear Systems…………………………………………………………………………………………………………. 107
• Color………………………………………………………………………………………………………………………………………………. 117
• FOUNDATIONS OF LEARNING……………………………………………………………………………………………… 135
• Introduction to Learning……………………………………………………………………………………………………………… 137
• Gradient-Based Learning Algorithms………………………………………………………………………………………… 151
• The Problem of Generalization……………………………………………………………………………………………………. 161
• Neural Networks…………………………………………………………………………………………………………………………… 175
• Neural Networks as Distribution Transformers………………………………………………………………………… 191
• Backpropagation…………………………………………………………………………………………………………………………… 199
• FOUNDATIONS OF IMAGE PROCESSING………………………………………………………………………….. 217
• Linear Image Filtering…………………………………………………………………………………………………………………. 219
• Fourier Analysis……………………………………………………………………………………………………………………………. 241
• LINEAR FILTERS……………………………………………………………………………………………………………………… 273
• Blur Filters…………………………………………………………………………………………………………………………………….. 275
• Image Derivatives…………………………………………………………………………………………………………………………. 287
• Temporal Filters…………………………………………………………………………………………………………………………… 315
• SAMPLING AND MULTISCALE IMAGE REPRESENTATIONS………………………………………. 325
• Image Sampling and Aliasing………………………………………………………………………………………………………. 327
• Downsampling and Upsampling Images…………………………………………………………………………………….. 345
• Filter Banks…………………………………………………………………………………………………………………………………… 365
• Image Pyramids……………………………………………………………………………………………………………………………. 385
• NEURAL ARCHITECTURES FOR VISION…………………………………………………………………………… 401
• Convolutional Neural Nets…………………………………………………………………………………………………………… 403
• Recurrent Neural Nets………………………………………………………………………………………………………………….. 431
• Transformers………………………………………………………………………………………………………………………………… 439
• PROBABILISTIC MODELS OF IMAGES………………………………………………………………………………. 465
• Statistical Image Models………………………………………………………………………………………………………………. 467
• Textures…………………………………………………………………………………………………………………………………………. 493
• Probabilistic Graphical Models…………………………………………………………………………………………………… 505
• GENERATIVE IMAGE MODELS AND REPRESENTATION LEARNING………………………. 525
• Representation Learning……………………………………………………………………………………………………………… 527
• Perceptual Grouping…………………………………………………………………………………………………………………….. 549
• Generative Models………………………………………………………………………………………………………………………… 559
• Generative Modeling Meets Representation Learning……………………………………………………………… 583
• Conditional Generative Models…………………………………………………………………………………………………… 603
• CHALLENGES IN LEARNING-BASED VISION…………………………………………………………………… 621
• Data Bias and Shift……………………………………………………………………………………………………………………….. 623
• Training for Robustness and Generality…………………………………………………………………………………….. 639
• Transfer Learning and Adaptation…………………………………………………………………………………………….. 645
• UNDERSTANDING GEOMETRY……………………………………………………………………………………………. 657
• Representing Images and Geometry…………………………………………………………………………………………… 659
• Camera Modeling and Calibration……………………………………………………………………………………………… 675
• Stereo Vision…………………………………………………………………………………………………………………………………. 701
• Homographies……………………………………………………………………………………………………………………………….. 721
• Single View Metrology…………………………………………………………………………………………………………………. 731
• Learning to Estimate Depth from a Single Image……………………………………………………………………… 757
• Multiview Geometry and Structure from Motion……………………………………………………………………… 769
• Radiance Fields…………………………………………………………………………………………………………………………….. 783
• UNDERSTANDING MOTION………………………………………………………………………………………………….. 799
• Motion Estimation………………………………………………………………………………………………………………………… 801
• 3D Motion and Its 2D Projection…………………………………………………………………………………………………. 813
• Optical Flow Estimation………………………………………………………………………………………………………………. 823
• Learning to Estimate Motion……………………………………………………………………………………………………….. 835
• UNDERSTANDING VISION WITH LANGUAGE…………………………………………………………………… 841
• Object Recognition……………………………………………………………………………………………………………………….. 843
• Vision and Language…………………………………………………………………………………………………………………….. 869
• ON RESEARCH, WRITING AND SPEAKING………………………………………………………………………… 885
• How to Do Research………………………………………………………………………………………………………………………. 887
• How to Write Papers…………………………………………………………………………………………………………………….. 893
• How to Give Talks…………………………………………………………………………………………………………………………. 903
• CLOSING REMARKS……………………………………………………………………………………………………………….. 909
• A Simple Vision System—Revisited……………………………………………………………………………………………. 911

Bibliography…………………………………………………………………………………………………………………………………………… 921

Index……………………………………………………………………………………………………………………………………………………….. 943