Experimental versus transparent

comment posted on “Crowdsourcing Best Practices for Experimental Journals: Transparency” by Adeline Koh, August 31, 2013, regarding controversy with Journal of Digital Humanities.

Addendum: followup discussion on Twitter, and blocked by Koh.

PDF version: Experimental-versus-transparent_2013-09-10.pdf

Skunk Works (Lockheed Martin Advanced Development Programs), Palmdale, California

Skunk Works (Lockheed Martin Advanced Development Programs), Palmdale, California

excellent ideas & topic, thanks for writing this. I like the Slashdot/Reddit angle.. you may know there’ve been several interesting Reddit+scholarly projects already, like r/scholar for requesting papers, and the Arxaliv Reddit front-end to arXiv repository (now defunct I think).

There’s an good study, or series of, to be done examining such new moderation structures across popular & academic peer-review systems, especially as new ones keep emerging all the time (like the just-announced Libre platform, new features in 3rd-party commenting networks, etc).

But more specifically about the experimental / transparency point, and without intending any judgment about any part of the JDH case, I was thinking about what general patterns may be seen here — or in the projects/orgs I’ve worked on, which have been all over the map in their processes.

It seems to me, and seems to be observed by others here, that ‘experimental’ and ‘transparent’ are really separate axes, and it’s helpful not to assume they will coincide or one lead to the other.

For example, well-established & stable processes can be highly transparent (legislative process in Denmark or Finland, say) or highly opaque (how to get a city contract in Naples?).

Experimental (or relatedly, in engineering, ‘agile‘) processes can be transparent — open participation, all docs public, etc. — or they might be opaque, or opaque to some points of view — such as an agile team or experimental/skunkworks lab that deliberately operates under less supervision, doesn’t work to pre-approved specifications, etc. In the latter type of case, experimentalism may well be opposed to transparency.

There are valid reasons various combinations of experimental or not, transparent or not might be chosen; but better they be chosen deliberately, and that one knows the arrangement.

Perhaps among the best ways to combine experimental & transparent is by iterative process. So a journal issue, or a software development cycle, or a year, might be set up with well-understood procedures and goals (as suggested by Scott, et al) but be very revisable for the next iteration. So this is, in a sense, transparent within the cycle, possibly opaque/negotiable across cycles.

Another idea one hears in software/project land is that a mixture of formal and informal guidances is needed, the latter being based more on factors like shared values, social behaviors, joint experience, & shared environment. These latter can’t be all spelled out, but evolve organically, and are in a sense non-transparent.

That’s a reminder of perhaps the first rule of process: do not talk too much about process.


Tim McCormick
@tmccormick  tjm.org  Palo Alto

.


Addendum: followup discussion on Twitter, and I’m blocked by Koh.

https://twitter.com/roopikarisam/status/377409865280655360

https://twitter.com/adelinekoh/status/377414431824543744