Results of the February 2013 community evaluation

Question

As you might know, we had a community evaluation review last week. The votes have been tabulated and below are the sample questions organized by score (Excellent - Needs Improvement):

1. Why is the blind man in John 9 sent to the pool that means “sent”? 7/1/0
2. Who does Jesus refer to by the phrase “another who testifies about me” in John 5:32? 6/2/0
3. Should the title in Isaiah 9 be translated? 6/4/0
4. Does Luke put the Twelve in a negative light in Acts 6? 3/5/0
5. Is Satan one of the “sons of God”? 3/5/1
6. Tribe of Benjamin Elite Corps of Slingers 3/5/2
7. How can Psalms 68:18 be translated as in Ephesians 4:8? 1/4/1
8. Paul's knowledge of Jannes and Jambres the magicians of Egypt 2/4/4
9. Romans 16:7 : “fellowprisoners” in the book of Romans?! 1/5/4
10. Why are the Psalms broken into five books? 1/3/4

The numbers after each question are the count of votes recorded for Excellent/Satisfactory/Needs Improvement. So question #1 had seven Excellent votes, one Satisfactory and no Needs Improvement. By my count we had 13 participants, but for various reasons, not everyone voted on every question. (I wasn't allowed to vote on questions 3 or 5, since I wrote those questions myself, for instance.)

I'd like to open the answers to any comments y'all might have on the results or the process of reviewing the questions. I'm particularly interested in analysis of the 3 questions at the bottom that clearly need improvement. What is lacking in those questions? What can we do to improve them? What lessons can we learn to avoid those pitfalls in the future?

I'd also like to know what problems people found with the higher-scoring questions. What did the reviewers miss?

Feel free to be critical of our site and don't hold back because:

Next eval begins on May 3 at 3:00.

(Obviously, don't get personal, however. Let's try to avoid naming names if possible and just focus on the content of the questions and answers.)

Answer 1

Overall, I'm pleased with the results. 4 of the 10 questions are at least satisfactory to our reviewers and only 3 had a negative score. As it happens, I voted "Needs Improvement" on those questions myself:

Why are the Psalms broken into five books?

The question is fine; it might be overly complicated since it throws out several theories and probes for implications of the divisions. But that's really not a huge issue since the main point of the question is clear: why are the Psalms broken up the way they are?

The answer is a bit short and mostly consists of a quotation that doesn't really answer the question so much as raise more. It's a good start, but not anywhere near what I would consider a comprehensive answer. Surely there is more information out there. Therefore, I've added a bounty to the question. Let's see if we can improve on this.

As a side note, I didn't vote either way on this answer, but it hits two of my reasons a correct answer might be downvoted. I'm not saying that the answer should be downvoted, but I don't think it should have gotten three upvotes. One of my screens for questions I might answer or place a bounty on is the Unanswered Questions queue. This question looks answered to the system since it's got an upvoted answer. In the future, let's try to hold back on upvoting answers that don't really answer the question.

Romans 16:7 : “fellowprisoners” in the book of Romans?!

The problem this time is the question itself. Part of the problem seems to be a language barrier: s/scientists/scholars/g. But the other part is that the question falls on the wrong side of our guideline: it springs from Wikipedia and not the text. As a result, the answer is speculative and a bit undercooked. Personally, I voted on neither the question nor the answer, but both probably should have been downvoted.

To fix up this particular question, someone needs to edit to reorient the focus on the text somehow. I've left a comment to see if we can decipher the OP's intentions.

Paul's knowledge of Jannes and Jambres the magicians of Egypt

This has almost an identical problem to the previous question: it seems to come from somewhere besides the text. This time, I don't know what we can do to fix it. Paul used sources other than the Tanakh: so what? The answer got a downvote from me largely because of the tangent about Simon the magician and the suggestion that J&J were "apostate believers". As far as I'm concerned, this is the worst question of the bunch and I'm curious where the 2 Excellent votes came from.

Looking back on the questions, I think I ought to have been more critical of:

How can Psalms 68:18 be translated as in Ephesians 4:8?

This is a self-answered question and so the risk is that the question will be underwhelming. And that's exactly how I feel about it. I can't put my finger on what the issue is, but the question is more like a butter knife than an arrow. Both answers are well argued, so that's not really the issue. The self-accepted answer is odd, but understandable. A the moment, I don't know how to edit the question to improve it, however.