User:Daniel Mietchen/Sandbox/Open Knowledge Conference 2010

Background

This document shall help the drafting of a "full paper" (meaning 15 min of talk on the basis of "5-10 pages describing novel strategies, tools, services or best-practices related to open knowledge") for the Open Knowledge Conference (OKCon) 2010. Submission deadline for the abstract: Jan 31, 2010. Our abstract has been accepted. Feel free to edit as you see fit. For discussion, please use this forum thread.

Deadline for submission of full paper: March 31, 2010.

The presentation will be given by Tom Morris, but everyone is invited to chime in on the drafting. It is intended to reuse much of this material for improving our Citizendium entry.

Key issues

This section is auxiliary to the drafting process and will be deleted when the draft is nearing completion. Let's concentrate on the following areas from the call for proposals:

  •   Platforms, methods and tools for creating, sharing and curating open knowledge
  •   Open educational tools and resources
  •   Supporting scientific workflows with open knowledge models

hence:

• Citizendium as

  • a platform for creating, sharing, curating and navigating open knowledge
  • an Open educational tool and resource
  • an open knowledge model supporting professional workflows
  • a democratic and meritocratic community

Technical details

All authors of OKCon papers:

• Can submit a camera ready version of the paper in LNCS style [1] for publication in the online proceedings e.g. till March 31st, to the proceedings editor: Claudia Muller-Birn <clmb@cs.cmu.edu>

[1] http://www.springer.com/computer/lncs?SGWID=0-164-7-72376-0%20LNCS%20Style

Please also note there is no page restriction in place on your papers for this process.

The final typesetting in LaTeX will be done by Daniel Mietchen.

Title

Public and experts working together: an experiment in structuring (disseminating?) knowledge

Alternative 1: Structuring knowledge for and with both the public and experts

Alternative 2: Collaborative structuring of knowledge by experts and the public

Abstract

The abstract should consist of no more than 200 words.

There is much debate on how public participation and expertise can be brought together in collaborative knowledge environments. One of the experiments addressing the issue directly is the Citizendium. In seeking to harvest the strengths (and avoiding the major pitfalls) of both user-generated wiki projects and traditional expert-approved reference works, it is a wiki to which anybody can contribute using their real names, while those with specific expertise are given a special role in assessing the quality of content. Upon fulfillment of a set of criteria like factual and linguistic accuracy, lack of bias, and readability by non-specialists, these entries are forked into two versions: a stable (and thus citable) approved "cluster" (an article with subpages providing supplementary information) and a draft version, the latter to allow for further development and updates. We provide an overview of how the Citizendium is structured and what it offers to the open knowledge communities, particularly to those engaged in education and research. Special attention will be paid to the structures and processes put in place to provide for transparent governance, to encourage collaboration, to resolve disputes in a civil manner and by taking into account expert opinions, and to facilitate navigation of the site and contextualization of its contents.

Introduction: The tension between expertise and public participation in the structuring of knowledge

Science is already a wiki if you look at it a certain way. It's just a highly inefficient one -- the incremental edits are made in papers instead of wikispace, and significant effort is expended to recapitulate existing knowledge in a paper in order to support the one to three new assertions made in any one paper. (John Wilbanks)

There are many ways to structure knowledge. One is via coordinated cellular activity in your brain. Others may involve spatial arrangements of sheets of paper or numeric arrangements of digital documents. Here, we will focus on the latter.

Traditionally, given the scope of a particular journal, knowledge about specialist terms (which may describe completely non-congruent concepts in different fields), methodologies, notations, mainstream opinions, trends, or major controversies could reasonably be expected to be widespread amongst the audience, which reduced the need to redundantly say and then repeat the same things all over again and again (in cross-disciplinary environments, there is a higher demand for proper disambiguation of the various meanings of a term). Nonetheless, redundancy is still quite visible in journal articles, especially in the introduction, methods, and discussion sections and the abstracts, often in a way characteristic of the authors (such that services like eTBLAST and JANE can make qualified guesses on authors of a particular piece of text, with good results if some of the authors have a lot of papers in the respective database, mainly PubMed, and if they have not changed their individual research scope too often in between).

Of course, there would be side effects: A manuscript well-adapted to the scope of one particular journal is often not very intelligible to someone outside its intended audience, which hampers cross-fertilization with other research fields (we will get back to this below). When using paper as the sole medium of communication there is not much to be done about this limitation. Indeed, we have become so used to it that some do not perceive it as a limitation at all. Similar thoughts apply to manuscript formatting. However, the times when paper alone reigned over scholarly communication have certainly passed, as discussed in part I. The relative merits of paper-based and wiki-based scholarly communication are covered in more detail at a dedicated Wikiversity page.

Cross-field fertilization is crucial with respect to interdisciplinary research projects, digital libraries and multi-journal (or indeed cross-disciplinary) bibliographic search engines (e.g. Google Scholar), since these dramatically increase the likelihood of, say, a biologist stumbling upon a not primarily biological source relevant to her research (think shape quantification or growth curves, for instance). What options do we have to systematically integrate such cross-disciplinary hidden treasures with the traditional intra-disciplinary background knowledge and with new insights resulting from research?

Wikis as an example of public knowledge environments online

Groupware comes to mind in this regard, and wikis in particular (another example would be collaborativey edited mindmaps, like the one embedded above that represents the topics covered by this blog post series): They allow us to aggregate and inter-link diverse sets of knowledge in an online-accessible manner, basically for free. The by now classical example is Wikipedia, and one scientific journal — RNA biology — has already announced that it requires an introductory Wikipedia article for papers it is to publish on RNA families, an idea that recently spurred a debate on the merits of such an initiative and of doing it with Wikipedia where basically anyone can edit any page, regardless of subject matter expertise.

An investigation (video lecture by Bill Wedemeyer here, a brief annotation here) of the quality of a set of science articles in the English Wikipedia is currently being written up for classical paper-style publication but the preliminary results indicate that "[t]here is a subset of reliably helpful science articles on the English Wikipedia for outreach, teacher training, and general science education" (slide shown at 29:35min in the video). However, the distribution of the set of articles was skewed towards the Good Article and Featured Article classes, which constituted only 2% of the English Wikipedia at the time of investigation, and it did not include articles in the humanities (scheduled to come next). Further information on academic studies about Wikipedia is available via these two Wikipedia pages.

The larger Wikipedias have a serious problem with vandalism: take an article of your choice and look in its history page for reverts - most of them will be about changes like this or worse. This is less of an issue with more popular topics for which large numbers of volunteers may be available to correct "spammy" entries but it is probably fair to assume that most researchers value their time too much to spend it on repeatedly correcting such information if it had already been correctly entered once. Other problems with covering scientific topics at Wikipedia include the notability criteria which have to be fulfilled to avoid an article being deleted, and the rejection of "original research" in the sense of not having been peer reviewed before publication. Peer review is indeed an important aspect of scholarly communication, as it paves the way towards the reproducibility that forms one of the foundations of modern science. Yet we know of no compelling reason to believe that it works better before than after publication (doing it beforehand was just a practical decision in times when journal space was measured in paper pages).

Fortunately, the Wikipedias are not the only wikis around, and amongst the more scholarly inclined alternatives, there are even a number of wiki-based journals, though usually with a very narrow scope and/or a low number of articles. On the contrary, Scholarpedia (which has classical peer review and an ISSN and may thus be counted as a wiki journal, too), OpenWetWare, Citizendium and the Wikiversities are cross-disciplinary and structured (and of a size, for the moment) such that vandalism and notability are not really a problem. With minor exceptions, real names are required at the first three, and anybody can contribute to entries about anything, particularly in their fields of expertise. None of these is even close to providing the vast amount of context existing in the English Wikipedia but the difference is much less dramatic if the latter were broken down to scholarly useful content, as discussed above. Out of these four wikis, only OpenWetWare and some Wikiversities (here counted as one) currently allow for original research to be published on their site — in the case of OpenWetWare, this is indeed the main purpose. Furthermore, a number of more specialized scholarly wikis exist (e.g. WikiGenes, the Encyclopedia of Earth, the Encyclopedia of the Cosmos, or the Dispersive PDE Wiki) which can teach us about the usefulness of wikis within specific academic fields.

The Citizendium model: Real names, stable versions, contextualization and open governance

Basic overview about the main differentiators to other Open Knowledge projects, and why they were introduced.

Many of those involved with the Citizendium believe that it is possible to go much further than existing open knowledge projects have gone - to be a bridge between existing scientific and academic communities and the new online communities. The founder of the Citizendium, Larry Sanger, was the co-founder of Wikipedia. The Citizendium differs in a number of important ways from Wikipedia and other similar projects.

The first of these is an insistence on real names. Every participant in the Citizendium project must be signed up using their real name - that is, the name they have on their passport or driving licence or their gas bill. This will exclude a number of legitimate contributors who prefer to remain anonymous. The real name policy gives participants accountability.

The second is the involvement of experts. Some critics of the project have come up with names for this like "credentialism", but it rests on a common sense belief that some people do know more than others: it is sometimes the case that the thirteen-year-old kid in Nebraska does know more than the physics professor. But most of the time it is not the case. The role the experts have is not that of a supreme leader, allowed to exercise his will on the populace - it is much more of a guiding role. We use the analogy of a village elder wandering around the busy marketplace who can resolve disputes and to whom people pay some deference due to his maturity, wisdom and expertise. Wikipedia rejects "credentialism" in much the same way that the Internet Engineering Task Force (IETF) does. David Clark summarised the IETF process thusly: "We reject kings, presidents and voting. We believe in rough consensus and running code." Wikipedia obviously doesn't have running code to worry about: in an open source project, or an IETF standardisation project, one can decide a great many of the disputes with reference to the compiler. If the code doesn't compile, think again. For rough consensus to happen, one needs to get the people together who have some clear aim in mind: getting two different servers to talk to one another. The rough consensus required for producing an encyclopedia article is different - it should attempt to put forward the truth, and people disagree on that. It is difficult to get "rough consensus, running code" when two parties are working on completely different epistemological standards. At this point, one needs the wisdom of Solomon, a grown-up who will come and clear up the mess and sort it all out. In other words, you need a government of sorts created by something like a Hobbesian social contract.

Open governance

Citizendium has an evolving, but hopefully soon-to-be clearly defined governance process - currently, a Charter is being drafted by an elected group of writers that will allow for democratic governance and oversight. The broad outline is this: we will have a democratically elected Editorial Council which will deal with content policy and resolving disputes regarding content, and we will also have a Management Committee, responsible for anything not related to content. The Management Committee appoint Constables who uphold community policy regarding behaviour. Disputes with the Constables can be brought to an Ombudsman selected by the Editorial Council and Management Committee. The charter is still to be ratified by the community. One of the reasons we have this is that although there is a cost to having bureaucracy and democracy, the benefits of having an open governance process outweigh the costs. We have a real problem when governments of real-life communities are controlled by shadowy cabals who invoke byzantine legal codes - all the same problems would seem to apply to online communities. With a Wikipedia article, the debate seems to never be the content itself, but a reified argument about acronyms (AfDs, NPOV, CSD, ArbCom, OR etc.). There is always a challenge in any knowledge-based community in attempting to reconcile a fair and democratic process with a meritocratic respect for expertise. There are no easy answers - if we go too far towards bureaucracy, we risk creating a system where management is separated from the actual day-to-day writing of the site, while if we attempt to let the site 'manage itself', we risk creating a rather conservative mob rule that doesn't afford due process to interested outsiders. A more traditional management structure, combined with real names and civility, should help those outside of the online community - the many experts in real life who work in universities, in business and in public life - be able to participate. Hopefully, if we get the governance decisions right, we can also not get in the way of the people who engage on hobbyist terms with the Citizendium.

Open education

An important part of the governance process is collaboration with those outside of the Citizendium. We have a long-standing project called Eduzendium, which allows for educators in higher education to work on articles as part of a course. We have most recently had politics students from the Illinois State University work on articles on pressure groups in American public life, as well as medical students from Edinburgh, biologists from City University of New York and the University of Colorado at Boulder, finance students from Temple University and others. These courses reserve a batch of articles usually for one semester, assign each article to one or more students. The course leader can either reserve the articles for just the group to work on, or they can work on them alongside editors on the site.

Open knowledge

The differences described so far have been around governance, but the policy the Citizendium uses for governance is only part of what makes it different from other collaborative knowledge projects: the content policy differences are more significant. Citizendium attempts to structure knowledge in a different way. Each article on Citizendium can make comprehensive use of Subpages. All pages are encouraged to have a short definition (around 30 words or 150 characters) which define or describe the subject of the page. They are also encouraged to have a comprehensive Related Articles subpage, which pulls in the definitions from the pages that it links to. If one looks at the article on 'Biology', one can see the parent topics of biology (science), the subtopics - subdisciplines of biology like zoology, genetics and biochemistry, articles on the history of biology and techniques used by biologists - and finally other related topics, including material on the life cycle, the various biochemical substances like DNA and proteins, the components of the cell, and other specialised language. This Related Articles page gives a pretty comprehensive contextual introduction to what biology is all about, and is structured by the authors of the article in a way that the category system used on Wikipedia isn't. Wikipedia has something similar with some of the sub-boxes you find at the end of articles, but we feel that the Related Articles subpage system allows for more detail, and is comprehensive rather than ad-hoc.

Each article can also have a comprehensive, categorised bibliography with annotations, pointing to books and journal articles on the subject. For scientific and academic topics, we attempt to write the list in a way so as to be useful both to a novice to the subject and to students and experts: much like a good academic reading list (e.g. London Philosophy Study Guide). Here, the expert guidance is important. Critics of Citizendium seem to think that the respect for experts is solely because of their knowledge of facts - as if whether or not the articles are good is dependent on whether or not the facts are good. This is only part of the reasoning: the experts correct factual mistakes, but they also help to guide how the structure of the subpages goes. The experts bring facts, but we also hope they bring with them wisdom gathered from extended in-depth knowledge of their subject matter.

The External Links subpage provides something similar to the Bibliography page but for web resources - pointing to relevant organizations, educational resources and other resources. For articles about authors, we allow for a 'Works' subpage to list the person's written works. For musicians and musical groups, a discography subpage can be added; for actors, producers and directors of film and television, a filmography subpage can be added. For some topics, we allow for a Timelines subpage to be added where a comprehensive timeline or multiple timelines about the topic can be presented (we have one, for instance, on the article on Tony Blair). The Catalogs Subpages exist to contain lists or tables of information much like that found in almanacs. Articles on food and cuisine can have a Recipes subpage.

We have Subpages to contain extra images, as well as audio and video resources, and for programming-related articles, to contain code samples and tutorials (for example, the article on Perl - also, some mathematical articles will contain a machine-readable implementation on the Code Subpage). The Subpages system can also contain alternative presentations of the topic at a variety of different levels - 'Advanced' Subpages can contain a version more suited to experts in the field, while 'Student Level' provides one more suited to children. A 'Debate Guide' Subpage can be created to provide an outline of all the arguments on a disputed topic. The Citizendium can also host 'Signed Articles', which are placed in a subpage alongside the main article. A Signed Article is an article on the topic described by a recognised expert in the field, but can express opinions and biases in a way that the main article ought not to.

Subpages are one way that the Citizendium is attempting to go beyond simply what is provided in an encyclopaedia and to engage with related forms of knowledge. Marx wrote that "Philosophers have hitherto only interpreted the world in various ways; the point is to change it". Traditional encyclopaedias attempt to reflect the world, but we are attempting to go further. The open science movement - which has formed around the combination of providing open access to journal articles, making scientific data more openly available in raw forms, using and sharing open source software and experimenting with some of the new techniques appearing from the community that is formed under the 'Web 2.0' banner - is exploring the edge of what is now possible for scientists to do to create new knowledge. Some of the electronic engagements by academics has been for actual research benefit, some has just been PR for universities - doing podcasts to sound 'relevant'. The Citizendium model, while a little bit more traditional than some of the open science community, is willing to try a variety of new things. Wikipedia has produced a pretty good encyclopedia - the challenge is to see if we can go further and produce a citizens' compendium of structured and comprehensive knowledge

There are still a number of challenges and opportunities:

Open questions

• how to motivate registered users to contribute
• how to motivate more users to register
• how to allow feedback by non-registered users
• how to codify the policies (and especially the subpages system) into a MediaWiki extension
• financial perspectives

Open perspectives

• contextualization
• potential for mutually beneficial partnerships with projects at similar wavelengths, e.g. AcaWiki for references, OpenWetWare for primary research, Open Access publishers as possible content providers

Just imagine you had a time slider and could watch the history of knowledge on tool making, cooking, clothing, learning, general relativity, plate tectonics, self-replication, or cell division unfold from the earliest ideas of their earliest proponents (and opponents) onwards up to now.

Figures

We should probably add a few illustrations of some core aspects (e.g. screenshots of Related Articles, Approval, Charter, Eduzendium, plus possibly some of the diagrams from RationalWiki). Please put them here.

References

Just links will do fine for the time being - Wiki and LaTeX are not compatible in this regard.

• Shirky and Sanger, or the costs of crowdsourcing

Contains a number of critical remarks on CZ that may be worth addressing.

Summary in the Wikipedia signpost

• RationalWiki article on Citizendium

Acknowledgements

The authors wish to thank Russell D. Jones and Howard C. Berkowitz for critical comments on earlier versions of this draft.

Further notes

• Speaker bio:

Tom Morris is a postgraduate philosophy student and programmer and has been actively involved as an author in the Citizendium project since October 2007.

• Just some suggestions for consideration from Aleta, not intended to be cast in stone. Question: what do you mean by an 'organically-growing wiki' (fifth line in abstract)? Doesn't resonate with me, can you say it in a different way?

• There is a talk given by Chris Day on Citizendium in mid-2008 at this workshop: video. His slides may serve as the basis for Tom's on this occasion (anyone can edit them)

• In preparation for CZ:Biology Week, we had already drafted a primer and turned it into a blog post that may serve as a seed for this draft.

Open knowledge

• a collaborative platform for creating, sharing, curating and navigating open knowledge

Open education

• an Open educational tool and resource

Taking these educational considerations into practice, Citizendium, in collaboration with teachers and lecturers, has launched Eduzendium [18], a project that allows students to write their course assignments online on the Citizendium. Students work for course credits, and their teachers grade the finished work based on the quality of the article drafts produced from each student's input. But by writing their assignments under this scheme, students not only get to earn grade credits, they can see their work online and add to the global store of knowledge. By collaborating with the rapidly growing Citizendium community of expert and non-expert authors, they stand good chances that their essays eventually develop into a lasting encyclopedic article. Finally, perhaps best of all, students get to learn in a highly collaborative real-time way, and rumours have it that they might actually have fun doing so. Not surprisingly, educators who opted for Eduzendium noticed a higher degree of enthusiasm amongst their students. The educational potential of CZ is enhanced by the use of subpages which provide for an easy integration with other free educational materials like videos, e.g. the non-profit, K-12 educational video contest WatchKnow [19] or, at undergraduate level, the non-profit world lecture project (wlp)° [20].

Open science

1. an open knowledge model supporting professional workflows
2. For scholarly uses, a detailed outline can be found in this blog post.
3. Demo at Research:In Vivo Assessment of Cold Adaptation in Insect Larvae by Magnetic Resonance Imaging and Magnetic Resonance Spectroscopy

CZ covers many fields, both academic and beyond, which are organized in workgroups whose main responsibility is to identifiy a set of core articles around which the field’s knowledge is structured, and to oversee the approval process (editorship in the sense discussed above is defined in terms of these workgroups).

As in traditional encyclopedias and Wikipedia, original research will not be allowed in the main namespace of CZ. Discussions are afloat for including original research into the subpages (e.g. as „signed articles“, similar to contributions to Scholarpedia) or other namespaces. Ways to take academic credit for contributions to CZ are also being discussed [21], whereas bot assistance for fact picking (as in [7]) can be made available on a case-by-case basis to facilitate data-intensive contributions.

Cross-disciplinary links are achieved in a variety of ways: First, several workgroups can collaborate on individual articles. Second, each article features a „related articles“ subpage where parent topics, subtopics and related topics are linked independent of their respective workgroups. Third, a coherent disambiguation strategy avoids page name disputes for articles on topics associated with different meanings in different fields, while allowing for a synopsis of what the different uses may have in common. Fourth, Citizendium organizes monthly Write-a-thons on broad topics to which anybody can contribute. Fifth, every user can nominate drafts as „Article of the Week“ or „New Draft of the Week“, and the winning entries are featured on the Welcome page, from where they usually receive lots of edits from specialists and non-specialists alike. Finally, as is typical for wikis, all contributions are immediately visible by anyone, and so the potential of frequent visits to the „recent changes“ page to initiate cross-disciplinary interactions should not be underestimated.

Activities in the biomedical fields have been especially visible: Biology is second to history in terms of number of articles (followed by health sciences), second to computers in terms of number of authors (followed by history) and fourth (after computers, engineering and health sciences) in number of editors (for details, see the CZ statistics [22].

Open governance

• a democratic and meritocratic community

Comments

• Initial reactions:

1. I suppose an inorganically growing wiki would use silicon...and it does. Some things cast in stone, indeed, would have high silicon content. It is, perhaps, a metaphor to avoid.
2. Not knowing much about Rationalwiki, yes, the article raises legitimate questions about some areas, such as healing arts. I would hope, however, that the observations on herding cats display some inability or unwillingness to recognize humor, and, indeed, informal analysis of idioms. It does seem needlessly antagonistic.
3. Real name policy needs, I think, to be in a broader perspective than comparison to Wikipedia. Netnews/USENET, for example, started declining in utility once AOL made public, anonymous access possible (1988 or thereabouts). Previously, while there was no real name policy, to get access, one had to be affiliated with a research or academic group, or have someone there willing to give you an account: a reputation factor.
4. Reputation factors, also called karma systems, have been used in blogs, and, I believe, are quite appropriate for Wikis. I believe the first widespread use was at the Well. There is a tension between name verification and reputation.
5. Contextualization is, I believe, one of the great opportunities, but it's fair to say that it's experimental. Nevertheless, it would be worth showing a decent Related Articles page, and perhaps compare-and-contrast it with Semantic Web techniques. Our subpages have a steep learning curve, but I wonder if any structured knowledge does not.

Howard C. Berkowitz 22:25, 24 March 2010 (UTC)

For Linked Data, see this discussion, for example. --Daniel Mietchen 23:17, 24 March 2010 (UTC)

Since the duties of a Managing Editor are still being discussed, I'd hesitate to publish it.

Reified argument? --Howard C. Berkowitz 01:00, 29 March 2010 (UTC)

Are we going to have slides? If so, I'm using a Mac so it'd be useful if they could be in Keynote format. –Tom Morris 05:36, 29 March 2010 (UTC)

@Howard, I am aware of the problems related to the draft not being finished, but think the role should be mentioned somehow. Any suggestions on the phrasing?

@Tom, whether to use slides or not is up to you (I tend to avoid them), but some old slides from a similar presentation by Chris are at #Further notes. Keynote is not very compatible with the rest of the world, so I suggest drafting them in the Google Doc I set up for the purpose, from where they can be exported in various formats. If we don't go for sophisticated formatting, this should work fine. Still four weeks for figuring it out, though. --Daniel Mietchen 08:41, 29 March 2010 (UTC)

It was more just so that if we need to show any images. I generally don't do "PowerPoint" - I just use slides to show images. Probably what is easiest is if we need images, just have a list and I'll download them (or screenshot them) and load them into Keynote. –Tom Morris 11:02, 29 March 2010 (UTC)

Given that the drafting committee has not made its discussions on the role public even to the CZ community, and there remains substantial discussion, I do not feel that any roles should be published, outside, that have not even reached the Citizen draft comment page. As far as suggestions on the phrasing, that is a matter of discussion in the committee; if I had better wording, I would have offered it there. I am certainly not going to offer it first for this presentation.

I would appreciate responses to the specific points I did make here. Howard C. Berkowitz 13:05, 29 March 2010 (UTC)

I numbered your points from above, so as to facilitate direct responses:

1. rephrased
2. While I am considering to use some of their diagrams, I do not think there was any thought on bringing in some of their language.
3. Yes — how to phrase it?
4. Yes — how to phrase it?
5. Yes — how to phrase it?

As for the role, I am waiting for feedback from the committee, and in the absence thereof, it will not be mentioned herein.

--Daniel Mietchen 14:07, 29 March 2010 (UTC)