| |
Abstract
The use of online questionnaires is rapidly increasing. Contrary to manifold advantages, not much is known about user behavior that can be measured outside the boundaries set by standard web technologies like HTML form elements. To show how the lack of knowledge about the user setting in web studies can be accounted for, we present a tool called UserActionTracer, with which it is possible to collect more behavior information than with any other paradata gathering tool, in order to (1) gather ...
|
| |
Abstract
Recently there has been renewed interest in cross-cultural qualitative research underscoring the epistemological and methodological pitfalls implied in this kind of research. In particular, focus groups, because of their intrinsically relational nature, require an accurate analysis of how the setting influences interpersonal exchanges and people’s attitudes toward participation and, thus, the results achieved. In this chapter, the authors consider how the data collection medium framed the results of a study involving 16 focus groups on HIV/AIDS, 8 conducted with Italians and ...
|
| |
Abstract
Web surveys can suffer from their nonrandom nature (coverage error) and low response rate (nonresponse error). Therefore, web surveys should be supported by mail survey to eliminate these problems. However, using different survey methods together may introduce another problem: the mode effect. This experimental study investigated the mode effect between two survey modes. A randomly selected group of 1,500 teachers were assigned to two experimental groups, one of which received mail surveys, while the other received web surveys. Nonrespondents in both ...
|
| |
Abstract
Unnumbered graphic rating scales theoretically can lead to greater score variance and thus greater score reliability. The use of Web-based graphic sliders overcomes the practical problems of administering graphic scales on paper measures: the excessive costs of scoring and excessive scoring errors arising from the tediousness of scoring done by hand. The study investigated whether graphic scales realize theoretically expected improved score reliability and how coarseness in slider scoring affects reliability. A large sample size (n = 4,407) and diverse respondent ...
|
| |
Abstract
In recent years, Internet surveys have been widely used by researchers and companies as a means of data collection. However, few IS studies have discussed the validation of Internet survey research. From the perspective of the positivist and quantitative research, studies that fail to organize a scientific and rigorous research design will harm their finding?s validity, reliability and generalizability. In turn; their contributions to the literature and implications for business are limited. ...
|
| |
Public Opinion Quarterly, Vol. 50, No. 3. (21 September 1986), pp. 402-413, doi:10.1086/268992
Abstract
This report examines the electronic survey as a research tool. In an electronic survey, respondents use a text processing program to self-administer a computer-based questionnaire. As more people have access to computers, electronic surveys may become widespread. The electronic survey can reduce processing costs because it automates the transformation of raw data into computer-readable form. It can combine advantages of interviews (e.g., prompts, complex branching) with those of paper mail surveys (e.g., standardization, anonymity). An important issue is how the electronic ...
|
| |
In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts (2012), pp. 41-50, doi:10.1145/2212776.2212782
Abstract
Using the example of research conducted in the body modification community, this paper considers some of the methodological issues of researching online communities, especially when those communities are marginalized or non-dominant. Drawing on texts that address ethical ethnographies of subcultures, I focus on boundaries between insiders and outsiders issues of recruitment, and measures of validity. ...
|
| |
Abstract
The paper addresses severalethical issues in online communication researchin light of digital ontology as well as theepistemological questions raised by theblurring boundary between fact and theory inthis field. The concept of ontology is used ina Heideggerian sense as related to the humancapacity of world construction on the basis ofthe givenness of our being-in-the-world.Ethical dilemmas of Internet research thusarise from the tension between bodily existenceand the proper object of research, i.e., onlineexistence. The following issues are beingconsidered: online identity, online language,online consent ...
|
| |
|
| |
Disabil Rehabil Assist Technol In Disability & Rehabilitation: Assistive Technology, Vol. 5, No. 6. (6 December 2010), pp. 401-410, doi:10.3109/17483101003793404
Abstract
Purpose.?A group of Australian researchers seeking an accessible online survey tool discovered to their concern that most commercially available survey tools are not actually ?useable? by a significant number of assistive technology users. Method.?Comparative effectiveness analysis of 11 popular survey tools. A bespoke survey tool was subsequently created to meet all accessibility guidelines and useability criteria as determined by the wide range of assistive technology users with whom the research team was working. Results.?Many survey tools claim accessibility status but this ...
|
| |
Abstract
Social media hold a treasure trove of information. But the secretive methods of ethics review boards are hindering their analysis, says Alexander Halavais. ...
|
| |
Abstract
Aim. This paper reports a study that explored whether active engagement and group interaction could be captured in an online environment. Background. Focus groups have become a common means of capitalizing on group interaction to collect rich responses to questions posed. Whilst their use is well established in the repertoire of qualitative researchers, with changing technology there is the opportunity to use a computer program that facilitates online engagement and interaction to bring together a group of people to explore issues, ...
|
| |
|
| |
|
| |
Social Science Research Network Working Paper Series (13 October 2010)
Abstract
Amazon’s Mechanical Turk is an online labor market where "requesters" post jobs and "workers" choose which jobs to do for pay. The central purpose of this paper is to demonstrate how to use this website for conducting behavioral research. We describe general techniques that apply to a variety of types of research and experiments across disciplines. We begin by discussing some of the advantages of doing experiments on Mechanical Turk, such as easy access to a large, diverse subject pool, low ...
|
| |
In Find Work in Wikipedia. Intelligent User Interfaces (IUI (2007), pp. 32-41
Abstract
Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create communitymaintained artifacts of lasting value (CALV) that support the group’s main purpose and provide value to others. Said communities don’t help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members ...
|
| |
Abstract
Data collected from mobile phones have the potential to provide insight into the relational dynamics of individuals. This paper compares observational data from mobile phones with standard self-report survey data. We find that the information from these two data sources is overlapping but distinct. For example, self-reports of physical proximity deviate from mobile phone records depending on the recency and salience of the interactions. We also demonstrate that it is possible to accurately infer 95% of friendships based on the observational ...
|
| |
Abstract
This review considers two recent trajectories of research on the geospatial web: efforts to develop appropriate methodologies for working with the new forms of geographic information that are part of it, and studies of its cultural, social, and political significance. In both arenas, visualization and visual methods are central. I show how methodologies drawn from quantitative and qualitative approaches to geovisualization in GIScience offer productive ways of working with geoweb-based information in research, and examine recent efforts to use critical visual ...
|
| |
In Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future (2005), pp. 1-10
Abstract
Mobile devices, applications and services have become integrated into people's daily lives on a personal and professional level. Although traditional research methods are being used to understand the use of mobile devices and applications, methodological challenges still exist. Researchers have responded to these challenges in a range of ways, with an emphasis on developing methods that enable new ways of accessing, making available and collecting, data about mobile technology use. This paper identifies, defines, describes and presents, a preliminary framework for ...
|
| |
Abstract
Abstract Ordnance Survey, the national mapping agency of Great Britain, is investigating how semantic web technologies assist its role as a geographical information provider. A major part of this work involves the development of prototype products and datasets in RDF. This article discusses the production of an example dataset for the administrative geography of Great Britain, demonstrating the advantages of explicitly encoding topological relations between geographic entities over traditional spatial queries. We also outline how these data can be linked to ...
|
| |
|
| |
Abstract
The shocking image of the young student Neda Salehi dying, after appearing to have been shot by the Iranian government's Security Forces, dominated the global news and online platforms during the 2009 Iran election crisis. Iranian protestors took to the streets, internet, blogosphere and Twitter to express their discontent about the re-election of President Mahmoud Ahmadinejad. In the days following the election, the global news attention shifted from the situation on the ground to the role of Twitter in the Iran ...
|
| |
Abstract
In recent years, candidates and other political actors have dramatically increased their presence and activities online. Although the notion of these activities reaching beyond a limited set of early-adopters is relatively new, younger citizens have long been at the forefront of new developments on the web and continue to make up a substantial proportion of those seeking political information online. Given longstanding concern over levels of civic and political engagement among young people, questions concerning what young people seeking information and ...
|
| |
In The Sage handbook of online research methods (2008)
|
| |
Abstract
Six incentive experiments were conducted in a nonprofit online panel. In each experiment, the incentive offered for participation was a cash lottery. The control group was not offered any incentive. The cash lottery was offered in two versions: Either the total payout of the lottery was mentioned, or the lottery was split into multiple prizes. Dependent measures included response and retention rates. The results of the six individual experiments were meta-analytically summarized. Cash lotteries relative to no incentives did not reliably ...
|
| |
|
| |
Abstract
A longstanding idea in the literature on human cooperation is that cooperation should be reinforced when conditional cooperators are more likely to interact. In the context of social networks, this idea implies that cooperation should fare better in highly clustered networks such as cliques than in networks with low clustering such as random networks. To test this hypothesis, we conducted a series of web-based experiments, ...
|
| |
Judgment and Decision Making, Vol. 5, No. 5. (2010), pp. 411-419
Abstract
Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by participants recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting participants, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss ...
|
| |
Abstract
The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called randomized experiments, A/B tests (and their generalizations), split tests, Control/Treatment tests, MultiVariable Tests (MVT) and parallel flights. Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior. We provide a practical guide to conducting online experiments, where end-users can help guide the development of features. Our experience indicates that significant learning and return-on-investment (ROI) are seen ...
|
| |
Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc, Vol. 35, No. 4. (November 2003), pp. 614-620
Abstract
This study evaluated the psychometric equivalency of Web-based research. The Sexual Boredom Scale was presented via the World-Wide Web along with five additional scales used to validate it. A subset of 533 participants that matched a previously published sample (Watt & Ewing, 1996) on age, gender, and race was identified. An 8 x 8 correlation matrix from the matched Internet sample was compared via structural equation modeling with a similar 8 x 8 correlation matrix from the previously published study. The ...
|
| |
American Psychologist, Vol. 59, No. 2. (0 2004), pp. 105-117
Abstract
As the Internet has changed communication, commerce, and the distribution of information, so too it is changing psychological research. Psychologists can observe new or rare phenomena online and can do research on traditional psychological topics more efficiently, enabling them to expand the scale and scope of their research. Yet these opportunities entail risk both to research quality and to human subjects. Internet research is inherently no more risky than traditional observational, survey, or experimental methods. Yet the risks and safeguards against ...
|
| |
In Arizona-Nevada Academy of Science Annual Meeting (April 2005)
|
| |
Cyberpsychology & behavior : the impact of the Internet, multimedia and virtual reality on behavior and society, Vol. 5, No. 4. (August 2002), pp. 305-313
Abstract
The Internet can be a valuable data collection tool for organizational psychology researchers. It can be less expensive than traditional paper-and-pencil survey methods, and the potential pool of participants is much larger. In addition, it can be used in situations where traditional data collection methods are not feasible, such as research involving sensitive issues such as negative employee attitudes or deviant behaviors at work. In this study, we examined the organizational attitudes of employees from various companies using (a) a snowball ...
|
| |
Cyberpsychology & behavior : the impact of the Internet, multimedia and virtual reality on behavior and society, Vol. 6, No. 1. (February 2003), pp. 73-80, doi:10.1089/109493103321167983
Abstract
The Internet can be an effective medium for the posting, exchange, and collection of information in psychology-related research and data. The relative ease and inexpensiveness of creating and maintaining Web-based applications, associated with the simplicity of use via the graphic-user interface format of form-based surveys, can establish a new research frontier for the social and behavioral sciences. To explore the possible use of Internet tools in psychological research, this study compared Web-based assessment techniques with traditional paper-based methods of different measures ...
|
| |
Abstract
This study evaluated the psychometric equivalency of Web-based research. The Sexual Boredom Scale was presented via the World-Wide Web along with five additional scales used to validate it. A subset of 533 participants that matched a previously published sample (Watt & Ewing, 1996) on age, gender, and race was identified. An 8 × 8 correlation matrix from the matched Internet sample was compared via structural equation modeling with a similar 8 × 8 correlation matrix from the previously published study. The ...
|
| |
Abstract
The rapid growth of the Internet provides a wealth of new research opportunities for psychologists. Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N = 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and ...
|
| |
Behav Res Methods, Vol. 37, No. 2. (May 2005), pp. 287-292
Abstract
The Web Experiment List (http://genpsylab-wexlist.unizh.ch/), a free Web-based service for the recruitment of participants in Internet-based experiments, is presented. The Web Experiment List also serves as a searchable archive for the research community. It lists more than 250 links to and descriptions of current and past Web experiments. Searches can be conducted by area of research, language, type of study, date, and status (active vs. archived). Data from log file analyses reveal an increasing use of the Web Experiment List and ...
|
| |
Behavior Research Methods, Instruments, & Computers, Vol. 33, No. 2. (1 May 2001), pp. 226-233
Abstract
There are many methodological differences between Web-based studies, differences that could substantially affect the results. The present study investigated whether sample type, offering payment through a lottery, and requiring participants to enter personal information would affect dropout rates and/or the substantive results in a study of jury decision making in capital cases. Asking participants to enter their e-mail addresses increased dropout rates, and offering payment through a lottery tended to do so as well. Participants offered payment tended to be less ...
|
| |
Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc, Vol. 35, No. 2. (May 2003), pp. 217-226
Abstract
Previous research has made a beginning in addressing the importance of methodological differences in Web-based research. The present paper presents four studies investigating whether sample type, financial incentives, time when personal information is requested, table design, and method of obtaining informed consent influence dropout and sample characteristics (both demographics and measured attitudes). Undergraduates were less likely to drop out than nonstudents, and nonstudents offered a financial incentive were less likely to drop out than those offered no incentive. Complex tables, tables ...
|
| |
|
| |
(2001)
Abstract
[Collection of articles, including discussion of issues in online survey research, online experiments, and perception research. Discusses procedures involved in developing an experimental Web laboratory] Internet Science is a new and exciting interdisciplinary field. Its purpose is the conduct of empirical studies which examine the Internet as both an instrument for, and an object of, scientific investigation. This book is the first comprehensive collection of contributions to Internet Science appearing in English, written by highly respected experts from seven different countries. Dimensions ...
|
| |
Abstract
Although Internet-based experiments are gaining in popularity, most studies rely on directly evaluating participants' responses rather than response times. In the present article, we present two experiments that demonstrate the feasibility of collecting response latency data over the World-Wide Web using WebExp-a software package designed to run psychological experiments over the Internet. Experiment 1 uses WebExp to collect measurements for known time intervals (generated using keyboard repetition). The resulting measurements are found to be accurate across platforms and load conditions. In ...
|
| |
Abstract
Internet-based psychological experimenting is presented as a method that needs careful consideration of a number of issues—from potential data corruption to revealing confidential information about participants. Ten issues are grouped into five areas of actions to be taken when developing an Internet experiment (dos) and five errors to be avoided (don’ts). Dos include: (a) utilizing dropout as a dependent variable, (b) the use of dropout to detect motivational confounding, (c) placement of questions for personal information, (d) using a collection of ...
|
| |
Being There Together (November 2010), pp. 205-231
Abstract
This chapter gives an overview of how multi-user virtual environments are increasingly being used for social research, beginning with a typology and review of these. One advantage is that all interactions can be recorded, and another that experiments can be done that would not be possible in face-to-face interactions. The chapter also details different methods that are being used, including participant observation or ethnographies, large-scale logging of populations, analyzing small group interaction. Some examples of small group interaction analysis are provided ...
|
| |
Abstract
Data from Web-delivered experiments conducted in browsers by remote users of PsychExperiments, a public on-line psychology laboratory, reveal experiment effects that mirror lab-based findings, even for experiments that require nearly millisecond accuracy of displays and responses. Textbook results are obtained not just for within-subjects effects, but for between-subjects effects as well. These results suggest that existing technology is adequate to permit Web delivery of many cognitive and social psychological experiments and that the added noise created by having participants in different ...
|
| |
Abstract
In fall 1995, the worldwide-accessible Web Experimental Psychology Lab (http://www.genpsylab. unizh.ch) opened its doors to Web surfers and Web experimenters. It offers a frequently visited place at which to conduct true experiments over the Internet. Data from 5 years of laboratory running time are presented, along with recommendations for setting up and maintaining a virtual laboratory, including sections on the history of the Web laboratory and of Web experimenting, the laboratory’s structure and design, visitor demographics, the Kids’ Experimental Psychology Lab, ...
|
| |
Abstract
Self-control problems have recently received considerable attention from economic theorists. We conducted two studies involving behavioral interventions expected to affect performance, providing some of the first experimental data in this area. In the first we investigate whether evenly spaced interim deadlines lead to higher completion rates for a lengthy task, where procrastination could be a factor. We found that these interim deadlines in fact led to lower completion rates in our set-up; we also found no evidence in the aggregate data ...
|
| |
Social Science Research Network Working Paper Series (16 April 2010)
Abstract
Online labor markets have great potential as platforms for conducting experiments. They provide immediate access to a large and diverse subject pool, and allow researchers to control the experimental context. Online experiments, we show, can be just as valid - both internally and externally - as laboratory and field experiments, while often requiring far less money and time to design and conduct. To demonstrate their value, we use an online labor market to replicate three classic experiments. The first finds quantitative ...
|
| |
Abstract
Two contemporary promises of participatory democracy are addressed in the present article; (a) democratic decisions reached in small group deliberation between lay citizens and (b) the possibility to exploit information and communication technology in democratic dialogue and decision making. Initially, a quasi-experimental approach was used to explore the potential and impact of face-to-face deliberation between citizens. In this design, a random sample of adult citizens was first surveyed and invited to take part in the experiment. The original face-to-face experiment was ...
|
| |
Abstract
Remote usability testing is emerging as a popular approach through which evaluators can test technology interfaces on a large number of participants quickly and inexpensively. Two types of remote studies have been employed, either with a moderator interacting remotely with the participant or in an unmoderated format. But without a moderator, or even when the moderator is present but not collocated with the participant, there is some question as to whether the results can match the validity, reliability, and acceptability of ...
|
