Wednesday, March 07, 2007

Peer review and the tomb

I am not yet convinced by one element in the scholarly reaction to the "Jesus Family Tomb", the claim that the case should have been made through the normal channels of academic peer reviewed journals. The claim has been made on several occasions and I don't need to illustrate. A large part of this scholarly reaction is, I think, understandable. It proceeds from the frustration of having to make a speedy judgement, when a paper calls you for a quick comment, on a case that one has not had the time to begin to digest. Even those who have already written more extensively on the case have often confused the filmmakers' case over the "Yose" tomb, which is that this is Jesus' brother Joses, not Jesus' father Joseph, something that comes from having to rush to comment while the topic is hot, while the wider world is interested. Scholarship is rarely about snap judgements, and it is not easy to provide a quick and easy answer to the claims when approached to do so by the papers. Few took the time last week to familiarise themselves with the basics of the case , but even if one does have the data at one's fingertips, one can hardly begin to get into any kind of detail when approached for a quick comment by the media.

All those things having been said, I think it is unreasonable to expect Jacobovici to have published his case in academic peer-reviewed journals. As he has repeatedly insisted, he is a journalist and a filmmaker and not an academic. It is notoriously difficult for non-professional academics to make it into a peer reviewed journal, all the more so if the case one has might be seen as imaginative or speculative. To turn it around, how often do we academics first publish our results by means of television documentary? If a filmmaker were to complain about our not having gone first to television, we would rightly point out that our access to that medium is limited and that we do not have the requisite expertise and experience to go through that portal. I realize that for some this might be seen to be begging the question. Their complaint is that the case for the identification of the Talpiot tomb needs to be properly set out, fully sourced and carefully argued. I agree that this is ultimately necessary if the claim is going to be properly explored, and James Tabor mentions that he and Shimon Gibson are working on such a piece (see The Jesus Dynasty blog, bottom of that entry). But I wouldn't expect a filmmaker like Jacobovici, without such academic expertise or experience, to be able to prepare such a piece. Under such circumstances, and if one were convinced one had chanced upon something massive, surely one would pursue it using the avenues available to one as a filmmaker.

I remain unpersuaded by the claims made in the film, and there are points where I wish Jacobovici had run his scripts through (at least) some fact-checking academic consultants, but I am not sure that it is right to criticize him for doing something that would have been outside his remit as a filmmaker. After all, if one were to ask the question about whether he has done what he can to get the data out to the public, one would have to say that he has done a good job. The reason that so many academics have been able to weigh in so quickly is that the Discovery website and the Jesus tomb website have featured so much of the necessary data, including even PDFs of earlier works (like Kloner's article, the pages from the Rahmani book), and then interviews and more.

12 comments:

Anonymous said...

Fair enough that Jacobovic is not an academic and shouldn't be expected to follow academic procedures, but the film was wrapped in a whole lot of academic garb, and if you're going to try to give the public the impression that your research is academic in nature, I think it's fair that they should at least have consulted with and presented the views of academics that questioned their thesis. Or maybe the real problem is not that Jacobovivi didn't follow academic protocols, but that he didn't follow proper filmmaking/journalism protocols, such as the basic principle that a journalist should try to be fair to all sides of an issue, not sensationalize matters, and not egregiously overstate the evidence that he's discovered.

yuckabuck said...

Mark,
I hate to say it, but I must agree with you on this. Darrel Bock said something on his blog about how this is the way peer review will occur from now on- through blogs or some kind of wiki-type thing like wikipedia.

Actually, I think Simsha J's film makes it more likely to move in that direction. The whole James Cameron/ film thing was so over the top, that it may lead people to find a more humble source to air their theories- one that is faster and more open to outsiders than peer journals, but something not so gaudy as a Hollywood movie premiering on a national television network.

BTW, I really enjoyed reading this site during the last week.

Anonymous said...

Thanks for this Mark and I completely agree. I plan to post something on this topic myself in a few days and have it already drafted. As I understand the proceess, that I observed from the inside the past two years, Jacobovici did in fact, in every case, seek those that are considered among the best in the various areas: epigraphy, paleo DNA testing, statistics, onomastics, archaeologists involved in the excavation, curators, etc. And aw shucks, even a biblical scholar or two, even if not the best. And as to whether he overstated his case, well I suppose since no one had ever made any case at all that this tomb might be significant for understanding Jesus of Nazareth in 26 years, it seems like there was a pretty open field and now is the time for that peer review. In other words, since everyone had dismissed the tomb as utterly common and insignificant in this regard, there was not a lot upon which Simcha could build, other than his own pioneering insight as in investigative journalist that there was more to be said. His Emmy award winning documentary roster, including the coveted Columbia Broadcasting award, do say something about his knowledge of his trade and the respect of his peers. I plan to say more about this at my own site later, as well as on the SBL site and the BAS site, particularly in response to my UNC colleage down the road, Jodi Magness. In the meantime, thanks for this Mark.

James Tabor
jesusdynasty.com/blog

ricb said...

I work with Simcha and I see all this talk about how he is not an academic. With due respect, he probably has as many degrees as a lot of your academics. We've just learned that his book, The Jesus Family Tomb, that he co-wrote with Dr. Charles Pellegrino, is going to appear at #6 on the New York Times Best Sellers List next Sunday. It may be written in a popular way but it's academically sound. Maybe it's time that a lot of academics get used to the fact that their little insular world will be invaded from time to time by smart people from the outside. This is the internet generation. And even academics will have to get used to it.

Ricb

Chris Weimer said...

Ricb - Sorry to but in here, but if Simcha J. was an academic, why did he so vehemently deny it on the Koppel special? He stated over and over again that "he's a journalist, not a scientist" (or something to that extent, the point being the same).

What are his credentials in? Hebrew epigraphy? Palaeontology? DNA forensics? New Testament chronology?

Anonymous said...

From: johnfp@earthlink.net
Subject: Peer Review in science and ethics
Date: March 8, 2007 8:54:51 AM EST
To: blog@ntgateway.com

Mark,

Wow, a great Blog! One of the good things that will come out of this is that I have found the NT Gateway!

Thanks too for your thoughtful post on the issue of peer review and "The Jesus Family Tomb." You make some good points as do a number of the responders.

As you know, the storm over this will probably get worse before it gets better. Here are a couple observations from a small town pastor (liberal).

1. Many, if not most of the people I interact with, don't believe what they see on TV anyway. This all falls into and reinforces the "people-will-say-anything" approach to information.

2. For me the curiosity at the lack of peer review comes from the fact that these people are making spectacular claims here about things that are sacred to to a large portion of the world. Knowing--as all must have--that those claims wouldl inflame, confuse or mislead might lead some people to set even higher standards for peer review. Is there an ethical obligation here?

2.1 The lack of concern for what is sacred for others is only confounded when the academics making the claims are teaching in departments of religion. I would think that being sensitive to the emotions surrounding things held sacred is an important part of teaching religion--even if it is history of religion It's a little like medical students dealing with a cadaver. In some ways the body is just a "thing", but medical schools expect the students to behave in a dignified way in the presence of the dead. Are there students who are disrespectful over cadavers? Of course, and they might make great scientists, but they do not make great physicians.

3. The insensitive seems to have something to do with the profit motive. Simple question of ethics and decor: Why make these claims and release the show and book at the beginning of Lent? The marketing department tells us that controversy is good for ratings and book sales. The ethic is "who cares if it is right in the end"; the goal appears to have been to create buzz. Perhaps the producers could redeem themselves by donating the hefty profits this book and documentary will produce to a non-profit society for the exploration of antiquities.

As in so many things there are multiple issues and cross currents at play here. There are academic and journalistic egos, there is the need of Discovery to turn a profit, and there are all the social issues that surround the role of faith in our individual lives and in our society. The archaeological issues, the math issues and all the scientific issues will be settled--some say resettled--in the peer journals. The other issues will live on. You can't un-ring a bell. The bottom line for me is that the ringing of this bell without serious peer review--not just in science but in ethics also--reflects poorly on the maturity of all the individuals and institutions involved in the production and the promotion of "The Jesus Family Tomb".

I want to leave the bolg on a lighter note. Here is the funniest comment that I heard so far: After a long conversation about how the scholars of Jesus Family Tomb reached their conclusions, an old Mainer rubbed his chin, scratched his head and slowly asked us all: "What's the probability that a show like this released at the beginning of lent is a hoax?" Sounds like a legit question but we had no idea of how to calculate....

Thanks again for your excellent blog!

paulf said...

Good post. Of all the strange things about this issue, the most bizarre may be dismissal of the theory by academics like William Dever (on the Koppel program) because the information was presented via a television show.

First, facts are facts. Whether they are true or not has nothing to do with who presents them or where.

Also, there is an irrational snobbery that academics have some sort of monopoly on truth. Now, I wholeheartedly agree that the vast majority of Christians, even ones who read the Bible every day, know very little about its history in any depth, and academics are often unfairly stigmatized. But academics should not unfairly stigmatize others who have done research and made an effort to uncover truth.

Third, Dever and others obviously know nothing about how journalism works. The point that Jacobivi should have waited until the investigation is complete until publishing anything is breathtakingly ignorant. Have these critics ever read a newspaper? It's like saying one shouldn't cover a trial until after the verdict. (BTW, I'm a 23-year veteran print journalist who has won three national awards.) Journalism is about getting facts out sooner rather than later and following up when more facts are known. Good journalism raises questions as much as provides answers.

paul fiorillarzdnod

Anonymous said...

You make a good point, but yet I still disagree.

Jacobovivi is a filmmaker. Sure. He also calls himself a journalist. And he called the film a documentary. Does anyone see a problem already? Those are conflicting titles, especially in the sense of objectivity.

My point here is, if you are going to utilize academia and label your work as a documentary, it needs to go through peer review because it becomes deceptive to the general public.

The purpose of peer review is not to simply sit around and enjoy each other's work. We should be inviting to others when they bring findings such as this to validate and remove bias / tunnel vision.

Who would you rather educate the untrained masses?

geoffhudson.blogspot.com said...

What does one decide when the academic peers disagree with each other? The ITV Channel 4 documentary on global warming last night was an absolute classic of disagreement among peers. I had to agree with the program makers who implied that global warming is not caused by increased carbon dioxide levels but by increased levels of cosmic rays from the sun during sun spot activity. The cosmic rays literally blow clouds away exposing the earth to the sun. Thus the earths's historical temperature variations can be correlated with varying sunspot activity. And how many academic peers have jumped on the bandwagon of global warming being caused by carbon dioxide?

Instead of mutual congratulation, a little more humility would perhaps be appropriate.

Ken Carl said...

Most of the problems with the information in this theory is that it is completely based on wild guesses. This is not the way that science is conducted. Here are some examples:

1) Since we don't know who this Mariamne is, she must be Mary Magdalene - There is no evidence in any writing anywhere that Mary Magdalene was called Mariamne. Simcha sites the Acts of Philip but the Mariamne contained therein is only called Philip's sister, no reference to Mary Magdalene whatsoever. BTW if you want a good chuckle, read the acts of Philip- it won't take long to be obvious why it is not in cannon.

2) If the DNA in the Jesus Ossuary is not a match with the Mariamne Ossuary they must be married. Again, nonsense. First of all, the practice of putting multiple people in one ossuary is not uncommon so they are most likely contaminated beyond any reasonable certainty. But anyway, There are plenty of other familial relationships that could exist between these two individual other than husband and wife but the film draws a pretty firm conclusion that they must be married.

3) The names on the ossuaries for which there is no historical record should be used to disprove the theory rather than bolster it. Instead Simcha chose to devise a story that makes his theory fit. He makes huge assumptions that cannot be called academic. There is no credible evidence that Jesus was married nor had a son. The Jose ossuary further confirms that this is not the family of Jesus.

4) If there had been proper peer review, as Mr. Tabor states above, why are these experts backing away from this. They are dropping like flies and it appears it is because they were misquoted, or their quotes did not represent what they really meant. Even the statistics guy has written a letter to his peers saying the numbers may not be accurate.

I am convinced by the evidence in this case that it is simply a media circus designed to make a fast buck! The irresponsible behavior on the part of everyone involved is shameful and this will fade away into the near future and those involved will have lost any respect in their communities that they may have enjoyed in the past.

Anonymous said...

Had this tomb contained the names Josephus son of Matthais,Vespasian and Justus I don't think we would be seeing all this nonsence. I think there would be quick scientific consensus that this is an intruiging find. By no means conclusive but suggestive, at least, and clearly in need of further investigation. As for the admittedly sensational way in which it has come to world attention, I dont see where it is at all important at this point. It would be different had the film makers rushed out their documentary ahead of some more traditional accademic research that was in the works, however in this case, traditional archaeology seems to have dropped the ball. In the end what the scientific community has before it is a very common situation. Evidence exists that is suggestive of an intruiging possibility. The evidence is far from conclusive but there is also nothing in the evidence that excludes said posibility. From this point there exists only one thing for science to do. Follow the evidence and stop debating how it came to light. In short open the tomb, open the disscusion and open our minds to the facts as they present themselves

Denis Lynch said...

It's not the film that necessarily should have been peer reviewed. The problem is that various academics let themselves be quoted saying things that they should have known were taken out of context, or should have known were just plain wrong.

James Tabor is astonishing: most people don't know about Luke 3? And Luke 3 presents a genealogy of Mary? If this were the 16th Century, those remarks wouldn't be outrageous. But in the 21st Century they are scandalous.

If every discussion has to rehash hundreds of years of settled arguments, scientific discourse is surely doomed. Peer review keeps that sort of time-wasting drivel out of the public conversation.