Proposal: experiment to compare open vs paywall publishing

Posted on by

Update 24 August: Designing for Diffusion

this proposal comes out of a discussion today on Twitter, regarding the imminent publication in Science of a paper by Lev Muchnik (Hebrew University of Jerusalem), Sinan Aral (MIT), & Sean J. Taylor (NYU):

actual paper:
“Social Influence Bias: A Randomized Experiment”
Lev Muchnik, Sinan Aral, Sean J. Taylor

Science 9 August 2013: Vol. 341 no. 6146 pp. 647-651
DOI: 10.1126/science.1240466
http://dx.doi.org/10.1126/science.1240466

Abstract (from Science): 

Our society is increasingly relying on the digitized, aggregated opinions of others to make decisions. We therefore designed and analyzed a large-scale randomized experiment on a social news aggregation Web site to investigate whether knowledge of such aggregates distorts decision-making. Prior ratings created significant bias in individual rating behavior, and positive and negative social influences created asymmetric herding effects. Whereas negative social influence inspired users to correct manipulated ratings, positive social influence increased the likelihood of positive ratings by 32% and created accumulating positive herding that increased final ratings by 25% on average. This positive herding was topic-dependent and affected by whether individuals were viewing the opinions of friends or enemies. A mixture of changing opinion and greater turnout under both manipulations together with a natural tendency to up-vote on the site combined to create the herding effects. Such findings will help interpret collective judgment accurately and avoid social influence bias in collective intelligence in the future.

Editor’s Summary (from Science): 

Follow the Leader?

The Internet has increased the likelihood that our decisions will be influenced by those being made around us. On the one hand, group decision-making can lead to better decisions, but it can also lead to “herding effects” that have resulted in financial disasters. Muchnik et al. (p. 647) examined the effect of collective information via a randomized experiment, which involved collaboration with a social news aggregation Web site on which readers could vote and comment on posted comments. Data were collected and analyzed after the Web site administrators arbitrarily voted positively or negatively (or not at all) as the first comment on more than 100,000 posts. False positive entries led to inflated subsequent scores, whereas false negative initial votes had small long-term effects. Both the topic being commented upon and the relationship between the poster and commenter were important. Future efforts will be needed to sort out how to correct for such effects in polls or other collective intelligence systems in order to counter social biases.

Update 8pm PST:

“The effect of free access on the diffusion of scholarly ideas” Heekyung Kim, working paper, 2012. (PDF). Using controlled data from SSRN, comparison of free vs paywalled papers finds 10-15% increase in citations for freely-accessible papers.

Heekyung Kim is a post-doctoral research fellow at the National Bureau of Economic Research, working with Shane Greenstein, Scott Stern, and Josh Lerner and finishing a PhD in Management Science at MIT, with advisors Erik Brynjolfsson and Scott Stern.

Kim is co-Principal Investigator for “The Impact of Free On-line Repositories on the Diffusion of Scholarly Ideas,” awarded by Sloan Foundation, $450K. 2012-2014.


Update 24 August: Designing for Diffusion

I forgot to add the below followup comments to Erik Bryn on 8 Aug.

Beyond just comparing current open/paywalled approaches, I’m interested — thinking as a designer or publisher — in what new things we might build or do to deliberately increase dissemination, impact, etc. For example,

  1. new alerting and social dissemination mechanisms, such as Twitter posts for new article titles & highlights; and
    .
  2. innovations in paywall use such as temporary/dynamic access (what I call “porewall”) for immediate public and journalistic use. E.g., for the rare paper that headlines in mainstream media or Reddit, detect that traffic pattern and automatically make open temporarily. (as the surf/skate culture aphorism says, “if it swells, ride it”).

.

What do you think?  Please feel free to comment below (login w/Twitter, Facebook, or Google+; or use/create a Disqus account. HTML formatting allowed.), or comment on Twitter with a link to this post and/or me @tmccormick; or email me at tmccormick (at) gmail.com.

Suggested hashtag to follow/add to discussion: #oaindex.

 

6 thoughts on “Proposal: experiment to compare open vs paywall publishing”

  1. Pingback: Social media anonymity encourages and nurtures the herd mentality | The k2p blog

  3. Pingback: @lupicinio

  4. Pingback: Tim McCormick (@tmccormick)

  5. Pingback: @Brarian

Comments are closed.