Event Photo Stream Segmentation: Chapter-Based Photo Organization For Personal Digital Photo Libraries
Research Area: Digital Libraries Year: 2013
Type of Publication: Phd Thesis  
  • Jesse Prabawa Gozali
Most commercial photo browsers today have an automatic mechanism to help users group their photos by event. This automatic event-based photo organization has not always been available. In the early days, digital photo management was similar to its analog counterpart where users had to manually organize their photos into photo albums. This thesis is motivated by the same issues today, but for photos within an event. People now are more liberal with their photo taking and have even more photos to manage for each of their events. To complement event-based photo organization and help users manage photos in each event, this thesis proposes a chapter-based photo organization where photos from each event are organized further, i.e. separated into smaller groups according to the moments in the event. We refer to this task as event photo stream segmentation. In this thesis, we developed a method to accomplish this exact task. Our method is based on a hidden Markov model with parameters learned from 1) a dataset of unlabelled, unsegmented event photo streams and 2) the event photo stream we want to segment. Our method is unsupervised, relies on features from temporal, camera parameters and visual information that are fast to compute. Our approach is based on our novel observation that an event’s photo stream consists of alternating feature types: features of the photo and features between consecutive photos. In an experiment with over 5000 photos from 28 personal photo sets, our method outperforms baseline methods including the state-of-the-art. This thesis also describes results from the first user study on chapter-based photo organization. The findings reveal key insights on how people organize their event photos. For example, users value chapter consistency more than the chronological order of the photos. The study also reveals common criteria people use to group their events into chapters. Another novel contribution is the photo layout study findings where we found that users value the chronological order of the chapters more than maximizing screen space usage and that users like having chapter thumbnails, but not at the expense of screen space utilization. Finally, the work we present culminates in CHAPTRS ver. 2, a publicly available, fully-implemented chapter-based photo browser that 1) complements eventbased photo organization by working with users’ existing digital photo libraries (iPhoto and Aperture), 2) automatically separates events into chapters, 3) presents the photos with a user interface design and photo layout based on the user study findings, and 4) allows easy drag-and-drop operations to fine-tune the photo arrangement with any criteria. To further research in this area, we used CHAPTRS ver. 2 to build a large public dataset of anonymous photo features and describe how using the Mac App Store as a distribution channel allowed us to reach a large number of participants and their personal digital photo libraries, a feat that would be difficult to achieve with volunteers or other conventional means.
Digital version