Where should we collaborate on open questions?

From QuestionWiki
Jump to navigation Jump to search

Currently there are projects where people collaborate on research. The most successful one seems to be the polymath project. Timothy Gowers posted an open question in mathematics on his blog and opened it up to collaboration. People had discussions in comments to the blog post, and kept track of stuff on a wiki. It was wildly successful and they went on to do 15 more projects. Gowers wrote:

First, let me say that for me personally this has been one of the most exciting six weeks of my mathematical life. That is partly because it is always exciting to solve a problem, but a much more important reason is the way this problem was solved, with people chipping in with their thoughts, provoking other people to have other thoughts (sometimes almost accidentally, and sometimes more logically), and ideas gradually emerging as a result.

That’s great. But the polymath project is still essentially in its own silo albeit a larger one than normal. What if one of those math questions has bearing on a question in chemistry or physics? Where should those questions be documented? Should we create separate wikis for open questions in chemistry and physics?

There are some problems that come from not having all of these collaborative documents on the same platform:

  1. Wikipedia has a process for merging articles but there’s no clean way to do that if everyone just starts their own project.
  2. It becomes harder to find the relevant collaborative documents if they are spread out across platforms.
  3. You cannot standardize policies and workflow across collaborations.

The frontier of knowledge is really a tightly interconnected web of open questions.  It seems that the appropriate way to work on these questions would reflect this interdependence. There should be one completely generic platform that handles any sort of open question.

What infrastructure do we need to collaborate on open questions?[edit | edit source]

(1) A web of interlinking collaborative documents, with a different document per question where the questions link to  other relevant questions.[edit | edit source]

So you might start with the question “How could we reduce the number of homeless people in California?” That’s related to the question “what are the main causes of homelessness?”. And that in turn is somewhat dependent on the answer to the question “how should we measure the number of homeless people in a population?” You get the idea.

On each of these question pages you’d track relevant sources, related questions, possible/proposed answers, possible experiments to decide between the answers etc. Basically the page would distill where the frontier of knowledge is on that question, and possible ways to push that frontier forward.

This wiki (openquestions.wiki) is the first attempt at this part of the infrastructure.

(2) A place to have free-form discussion about the questions being worked on.[edit | edit source]

Ideally this discussion would happen on the same platform as the wiki (for example, the talk page here). But it should also be fine to host discussions on other forums and simply post links to those discussions on the wiki. And the post in each of those off-wiki discussions should link back to the wiki page. That way the people on the various platforms can easily find the other places where the question is being discussed.

For the polymath project most of their discussion happened in blog post comments that were linked to from the wiki. Though the format isn’t ideal, it benefited from having a pre-existing audience.

(3) A place to coordinate tasks.[edit | edit source]

In answering open questions, there can be a lot of work to be done beyond thinking and discussing. Maybe someone should visit some museum, do some interviews, gather data etc. You need to coordinate all of that with some sort of task management system like trello or GitHub issues/projects. Wikipedia uses https://phabricator.wikimedia.org/.

Again, ideally this would be on the same platform as the collaborative documents. But it could be hosted somewhere else as long as the relevant tasks for each question were linked to in that question’s wiki page.

(4) A set of norms and rules to govern it all.[edit | edit source]

Think of Wikipedia’s Policies and Guidelines. At the core of wikipedia is governance by consensus, and it’s policies are simply a representation of that consensus. I imagine you could use many of the same policies as Wikipedia (obviously not the no original research policy).

The polymath project also has a set of rules that could be adapted to a more general platform for open questions.

You wouldn’t need to come up with all of the rules ahead of time.  One of the open questions on the platform could be something like “What rules and norms should we have on this platform?”.

What are the places online where people are already trying to work like this?[edit | edit source]

There are various specialized projects like the polymath project. But there are only 2 platforms that seem to be interested in a more general approach to answering open questions.

Wikiversity.org's research section[edit | edit source]

Wikiversity has a research section where you can collaborate on research projects. So far that seems to be the closest to QuestionWiki. But more will have to be done to determine what are the similarities and differences between Wikiversity and this wiki. It's possible that the efforts of this wiki should simply be moved over to Wikiversity's research pages.

Some differences between this wiki and Wikiversity research that may or may not matter:

  1. On Wikiversity you can have your own research projects where you can let only certain people edit them. That may lead to a certain kind of fracturing that QuestionWiki is trying to avoid.
  2. QuestionWiki is structured around on questions, whereas Wikiversity research is much more free-form. I'm not sure how much that matters. And if it does matter maybe Wikiversity could just host the question based wiki?

Here is a discussion on some of the downside of wikiversity: https://www.quora.com/Why-did-Wikiversity-fail[edit | edit source]

Asone.ai[edit | edit source]

They currently host the polymath wiki and are aiming to have the discussion for the next polymath project take place on their platform (instead of blog post comments). So far they are essentially a reddit clone where the idea is that each subreddit type thing is a different question. The questions can be organized into fields (like mathematics, psychology etc).

I appreciate their vision, but so far it’s mostly just a place to have discussion, which is just section (2) of the requirements listed above. So far the only robust wiki functionality they have on that site is limited to the polymath projects.

If they could make the wiki completely general, and allow the polymath projects to continue as a subset of the wiki, then they would satisfy criteria (1) above.

LessWrong.com[edit | edit source]

It looks like LessWrong is interested in collaborating on open questions as well. Here is a post describing some of the goals for the questions section in lesswrong (Upcoming: Open Questions - LessWrong):

Ultimately, we’d like [the] Q&A section to not just be useful for answering one-off questions but for laying out actual research agendas. (Possibly with a higher-level organization that groups questions together, similar to how sequences group posts).

Or from here:

The vision for Q&A is scaling up intellectual progress on important problems. Doing real research. People taking their large questions, carving off pieces, people going off and making their own contributions to research (without hiring and all that overhead).

So far the questions section of lesswrong fulfills the requirement (2) from above. But there aren’t any collaborative documents associated with these questions yet, let alone an interlinked centralized set of collaborative documents. It does look like lesswrong is interested in going in that direction though.

Where should the wiki part of the open question platform be?[edit | edit source]

Self hosted Mediawiki[edit | edit source]

The natural starting point seems to be some sort of wiki. Mediawiki is the most used open source wiki software, but there are other options as well. This wiki (openquestions.wiki) is using mediawiki for a few reasons:

  1. Because it's used so much it has a rich feature set. In particular it has a lot of features to help control spam and vandalism.
  2. There is a possibility that the Wikimedia Foundation may want to include the open question wiki as a sister project, which would be easier to do if it was hosted on the a mediawiki wiki.

openquestions.wiki has taken this approach. But there are other possible solutions that may or may not be better.

A Wikimedia Sister Project[edit | edit source]

If this wiki couldn't be incorporated into Wikiversity, maybe they would include it as it's own standalone project.

This may be the best option, as the project would then have the full financial and technical support of the Wikimedia Foundation. They have a process for proposing new sister projects. There even exists a proposal for a question based wiki.

The main problem with this option is that very few projects get accepted as a sister project. The last project to be created was Wikidata in 2013. Because of this it seems useful to start the wiki before waiting for approval.

Gitlab or Github[edit | edit source]

You could have each question be a separate repo on GitHub or Gitlab. You’d just have the collaborative document(s) be in markdown. Discussion would happen in pull request comments and issues.

Alternatively, gitlab is open source, so you could potentially just use that, rip out the stuff that’s not relevant, and host it on your own site. Then you could use your own domain and be in control of everything. That seems better to me intuitively, but there are potentially downsides there compared to just using gitlab.com or github.com (like having to pay for your own servers and stuff).

One huge potential problem has to do with permissions. Gitlab and GitHub understandably allow for fine grained control of who has permission to do what. Part of wikipedia’s success has been due to its policy of allowing anyone to edit. I’m worried about the person who initially creates a question having ultimate power over permissions of that question.

But maybe it wouldn’t take too much work to change the open source gitlab to allow for a more open permissions policy to be required for each repo?

Google Docs[edit | edit source]

Google docs are probably the best collaborative documents. But I’m not really sure you can scale them up to have large networks of interlinked google docs. And you still have the permissions issue, where the owner of the doc could just decide to delete it at any time, or lock people out of it. But I’m open to suggestions on how to overcome those issues.

Remaining questions[edit | edit source]

  1. Should we have a centralized location for the interlinking collaborative documents? Or does it make more sense to have a more distributed model with different open research initiatives happening on different platforms?
  2. Are there any other online platforms that are trying to do open research besides Wikiversity, LessWrong, and Asone?
  3. Should this wiki be merged with one of those projects (assuming they would be willing)?
  4. Assuming we do want a centralized wiki, what kind of software should be used and where should it be hosted (either in conjunction with asone/lesswrong or independent from them)?