It can be difficult to properly understand aspects of user behavior on the Web without examining the behaviors in a realistic setting, such as through field studies. In this article, an overview of the experiences in augmenting logged data with contextual information over the course of two separate research projects conducted in the field is presented. One project investigated the privacy sensitivity of normal Web browsing, and the other examined user behavior during Web-based information-seeking tasks. Throughout both projects, the contextual information was collected through participant annotations of their Web usage. Based on experiences in conducting this research, implications of methodological decisions are considered, unanswered questions are highlighted, and considerations for other researchers are provided. These shared experiences and perspectives will assist future researchers planning similar field studies, allowing them to build upon the lessons learned.