Most commercial photo browsers today have an automatic mechanism to help
users group their photos by event. This automatic event-based photo organization
has not always been available. In the early days, digital photo management was
similar to its analog counterpart where users had to manually organize their photos
into photo albums. This thesis is motivated by the same issues today, but for photos
within an event. People now are more liberal with their photo taking and have even
more photos to manage for each of their events.
To complement event-based photo organization and help users manage photos
in each event, this thesis proposes a chapter-based photo organization where
photos from each event are organized further, i.e. separated into smaller groups
according to the moments in the event. We refer to this task as event photo stream
segmentation. In this thesis, we developed a method to accomplish this exact task.
Our method is based on a hidden Markov model with parameters learned from 1)
a dataset of unlabelled, unsegmented event photo streams and 2) the event photo
stream we want to segment. Our method is unsupervised, relies on features from
temporal, camera parameters and visual information that are fast to compute. Our
approach is based on our novel observation that an event’s photo stream consists of
alternating feature types: features of the photo and features between consecutive
photos. In an experiment with over 5000 photos from 28 personal photo sets, our
method outperforms baseline methods including the state-of-the-art.
This thesis also describes results from the first user study on chapter-based
photo organization. The findings reveal key insights on how people organize their
event photos. For example, users value chapter consistency more than the chronological
order of the photos. The study also reveals common criteria people use
to group their events into chapters. Another novel contribution is the photo layout
study findings where we found that users value the chronological order of the chapters
more than maximizing screen space usage and that users like having chapter
thumbnails, but not at the expense of screen space utilization.
Finally, the work we present culminates in CHAPTRS ver. 2, a publicly available,
fully-implemented chapter-based photo browser that 1) complements eventbased
photo organization by working with users’ existing digital photo libraries
(iPhoto and Aperture), 2) automatically separates events into chapters, 3) presents
the photos with a user interface design and photo layout based on the user study
findings, and 4) allows easy drag-and-drop operations to fine-tune the photo arrangement
with any criteria.
To further research in this area, we used CHAPTRS ver. 2 to build a large public
dataset of anonymous photo features and describe how using the Mac App Store
as a distribution channel allowed us to reach a large number of participants and
their personal digital photo libraries, a feat that would be difficult to achieve with
volunteers or other conventional means.