Fact-finding Mission to China Provides Key Insights
Fact-finding Mission to China Provides Key Insights
By Tracey Brown Posted on 18 September 2011
"In China, the research community is gaining year on year in resources and ability. That is very exciting to be around.” — Tracey Brown, Managing Director, Sense About Science
Tracey Brown believes peer review is vital to good science and the society that uses it.
And it’s a conviction the Managing Director of Sense About Science shares with members of the Chinese Academy of Sciences, as she discovered during a trip to the research-rich country in March this year.
Brown embarked on the fact-finding mission with two key aims in mind; she was keen to test out views advanced about the integration of Chinese authors and reviewers into international STM publishing, and to explore future collaborations to help researchers, policy makers and journalists identify the best science.
During the two-week visit, which was supported by Elsevier, she met not only the Chinese Academy of Sciences (CAS), but Science.net, journalists, post docs and publishers.
Brown says: “It was clear that CAS is keen to discuss the best ways to evaluate research and to explore their concerns about what peer-reviewed publishing can - and can't - deliver. In an effort to avoid cronyism and subjective assessment in China, there has been a shift towards using flatter measurements; for example, the Impact Factor. There is a feeling, however, that these do not reveal enough about individual papers or the research output of an institution. Most people, including CAS, are coming to the conclusion that what we really need is a mix of the two.”
Gaining new understanding
Asked to highlight some of her key learnings during the trip, Brown says:
“People raised many interesting points and some quite contradictory ones. The early career researchers I spoke with viewed international journals as motivated by quality and fairness, and in some cases compared them favourably with Chinese journals, which can be seen as wedded to the relationships and prestige of individuals and institutions.
“On the other hand, some of the more editorially-experienced people had stories of less than positive attitudes among international editors to Chinese papers. They were concerned about a head-in-the-sand approach to such a major research base and that valuable new insights could be missed.”
Other key take-aways for Brown include:
- General agreement that reviewing is an important part of the role of a researcher. However, involvement in it varies enormously.
- As in many other countries, a researcher's day is structured in a way that makes it difficult to find time to review and their career progresses in response to grants and publications, not time spent reviewing. Views differed widely about the problems this posed and whether it inhibited the reviewing effort.
- A strong interest in training, both for authors and reviewers.
- Interest in other metrics for evaluating research output, the respective contributions of regions/countries and the performance of individual institutions.
There were also a few eye-opening moments for Brown.
She explains: “I had not expected people’s personal experiences to differ so widely. For example, I was speaking to two post doc students at Shanghai Jiao Tong University. Both had published very successfully early in their careers in some of the top journals - the elite of the elite. One was receiving almost weekly requests to review while the other had received only one request in a year. That may reflect the different nature of their papers but I heard their stories repeated elsewhere. It is perhaps to be expected that peer-review requests from international journals are still a bit of a hit and miss process in China.”
She adds: “Each time I was about to draw a conclusion about anything I would meet someone who took me in a different direction – a symptom, I imagine, of things being a work in progress there.
“Another surprising thing for me was the high level of confidence in the research community in contrast to the UK, and perhaps the US, where universities face straitened circumstances. In China, the research community is gaining year on year in resources and ability. That is very exciting to be around.”
The pressure to publish
Commenting on the quality – and quantity – of papers submitted by Chinese researchers, Brown says: “There is some concern, internationally, about filtering the sheer weight of papers produced by China. A big sea of papers makes it difficult to pick out the best.
“The thing is, there is a large pressure to publish in China and doing so in international journals brings career breaks and prestige. While lead institutions no longer pay incentives for this, some second-tier universities still appear to, which may contribute to journals being overwhelmed by unsuitable papers.
“We discovered that inappropriate submissions also stem from a lack of local knowledge about international journals, with younger researchers copying where their supervisors have published. Library services can play a very important role in widening the pool of journals considered.”
She adds: “Since returning I have been in touch with members of the Publishing Research Consortium to discuss the prospect of looking at how these new regions, such as China and India, are being integrated. Do editors now need something different from publishers with regard to support and advice? These are questions I know publishers are asking too. There is clearly some opportunity for international publishers to improve the availability of information about how to publish and where to publish, probably via librarians in those institutions where library services are developing and pro-active.”
Looking to the future
And what does Brown think the next five years will hold for the Chinese research community?
“Because of the volume of research and population size, even minority behaviors in China are likely to have a significant effect. If just a proportion of the new generation of researchers are trained and engaged with reviewing, it could have a big impact on sharing the reviewing burden. I know that there are already programs underway, such as Elsevier’s Reviewer Workshops and Reviewer Mentorship Program. The value of their contribution to the research output cannot be overstated – just like so many other things in China at the moment!”
What is Sense About Science?
Sense About Science is a UK charitable trust that equips people to make sense of science and evidence on issues that matter to society. With a network of more than 4,000 scientists, the organization works with scientific bodies, research publishers, policy makers, the public and the media, to lead public discussions about science and evidence. Through award-winning public campaigns, it shares the tools of scientific thinking and the peer-review process. Sense About Science’s growing Voice of Young Science network engages hundreds of early career researchers in public debates about science. Sense About Science will be publishing a Chinese edition of its public guide to peer review I Don’t Know What to Believe early in 2012 in collaboration with learned societies, patient groups and journalists.
MANAGING DIRECTOR OF SENSE ABOUT SCIENCE
Tracey has been the Director of Sense About Science since shortly after it was established in 2002. Tracey is a trustee of
David Sampson says: September 22, 2011 at 11:27 am
In Elsevier’s opinion, what impact will the decision by GAPP to improve the quality of Chinese journals have on STM publishing? Are we seeing weaker journals in China folding and stronger ones transforming?
Philippe Terheggen, Elsevier Senior Vice President Physical Sciences says: September 23, 2011 at 12:49 pm
While the quality of Chinese research is improving, Chinese academics and their funders increase their aspiration levels for Chinese-based research titles. STM publishers, also those outside China, welcome this direction, because it encourages Chinese authors to strive for quality levels that compete with international standards.
Sandy Shumway says: September 22, 2011 at 2:32 pm
I enjoyed your article and am pleased to see this sort of thing taking place. As an Editor of 4 international journals, I can attest to the flood of papers from China – and the quality is certainly variable, but mostly they are weak papers both in terms of science and writing. I just returned from a trip to China and read a fascinating editorial/story entitled ‘Honest Research Needed’. It was written in response to the fact that none of the research works shortlisted for the 2011 National Natural Science Awards qualified for the top prize and that this was the 8th time this has occurred since the award was introduced in 2000. Given the quality of the research that crosses my desk in these fields, I was not surprised at the numbers – I contend that in addition to help with publishing, there needs to be a concerted effort to advise on the science itself. Much of what is being done in the natural sciences is mundane and only of local interest – if that. Moreover, it is done poorly, bad experimental design, execution, etc. It isn’t all about publishing – there needs to be quality data first!
David C. Baker says: September 22, 2011 at 3:18 pm
My experience as an editor is that Chinese scientists, in particular, no matter how well qualified, are very reluctant to participate in the review process. This is disappointing, given the fact that we editors spend a lot of time getting their material reviewed and ready for publication. It is my feeling that they should assist us in sorting out the huge volume of papers coming from their country.
Kuan-Teh Jeang says: September 27, 2011 at 4:36 pm
As a journal editor of Chinese extraction (albeit fully educated and raised in the United States), I do not experience the same difficulty as outlined by David C. Baker. Could the issue simply be (as commented upon elsewhere by Philippe Terheggen and David Clark) that “the percentage of Elsevier editors from China is 3.3% while nearly 13% of published articles originate there. Some countries are also significantly over-represented, for example, 40% of our editors come from the US while only 18% of published articles originate there”. In science, each person is linked to his/her network. If Elsevier journals wish to build a better network of reviewers in China, why not start by recruiting good Editors and Editorial board members from China. If one walks that path, I bet that the disappointment of not finding enough well-qualified reviewers from China will quickly vanish.