David E. Meltzer

Working Group Members: Brad Ambrose, Ian Beatty, Lillian McDermott, Sam McKagan, David Meltzer, Bill Reay, Miriam Reiner, Jing Wang

This working group had the task of arriving at a position on the value of non-traditional publications such as online published volumes, arxiv.org, white papers, blogs, and others. I will present the outcomes of the group’s discussion by enumerating the principal topics that arose.

arxiv.org [http://www.arXiv.org]

arXiv.org is an on-line e-print service in a variety of technical fields including physics. It is owned, operated and funded by Cornell University. Publishing on arxiv.org ensures open access to authors’ work; however, posted items can never be removed. (Modified versions may be added, making earlier versions less easily accessible.) As a consequence, after a paper is published in final form in a journal or elsewhere, an obsolete copy of the paper will continue to exist permanently on the web and, presumably, be visible to search engines. Some authors would see that as undesirable.

A potentially significant advantage is that items which may not easily be accessible in any other form (such as papers in obscure conference proceedings) may be posted on arxiv.org. Papers that are not yet accepted—which the authors may not wish to revise until some time in the future—may be posted on arxiv.org to ensure some degree of dissemination.

PER-Central [http://www.compadre.org/per/]

This site is a collection of hundreds of citations and links to articles and dissertations, research groups, PER-based curricular materials, news and events, and many other things of interest to the PER community. Recently, this site has published a small number of invited review papers that provide extensive discussions of research-based curriculum and instruction projects. Our group wondered whether, at some time in the future, PER-Central might serve as a venue for posting preprints, preliminary versions of instructional materials and/or resources, and similar materials. As of now, it provides links to these materials when they have already been posted on another web site.

ERIC (Education Resources Information Center) [http://www.eric.ed.gov/]

ERIC is sponsored by the U.S. Department of Education; it calls itself “the world’s largest digital library of educational literature.” It provides free access to more than 1.2 million bibliographic records of journal articles and other education-related materials and, if available, includes links to full text. Many PER journals and conference proceedings are included in the index. ERIC microfiche archives are widely available in research libraries nationwide, but the current thrust is for digital publication. According to their site, “ERIC is actively seeking individual submissions of high-quality education-related materials for inclusion in the ERIC database. Types of materials appropriate for individual submission include research reports, conference papers and presentations, and dissertations and theses. ERIC does not accept lesson plans, blogs, or individual Web pages.” Our group felt that ERIC might well be a resource that is underutilized by members of the PER community.

Review Papers and Collections in Book Form

The group felt that it would be useful to have more review articles including, perhaps, full-length “review books” (similar to the lengthy invited papers posted at PER Central). These could include guides to the PER literature, which might have special value for graduate students. Examples that were proposed were an expanded and updated version of the McDermott/Redish Resource Letter (Am. J. Phys. 67, 755-767, 1999), and an annotated version of the tabulation of PER papers published in AJP which is posted at http://www.physicseducation.net/current/index.html. Another possibility would be to have book-length collections of overview papers or papers focused on a single theme. As an example it was noted that the APS and AAPT, as part of the PhysTEC project, together plan joint publication of a book of scholarly papers focused on the topic of physics teacher preparation (http://www.ptec.org/features/newsDetail.cfm?id=139).

Research-Group Web Sites

Some research groups post on their own web sites a wide variety of items that are unavailable elsewhere, for example:

• “white papers” and opinion pieces
• reports
• very short articles
• teachers’ guides
• meeting notes
• validation studies of curricular methods and materials, etc.

There may be other viable publication venues for these types of materials, for example, newsletters (such as the APS Forum on Education), the new periodical AAPT Interactions, and the ERIC digital library.

Issues of Quality Control and Peer Review

The group addressed the question of how curricular materials might be subjected to some form of peer review or quality control by the PER community, apart from authors publishing articles in refereed journals that discussed the development of the materials. This was considered to be an important issue since peer review is given paramount importance in the physics community generally, and a large portion of PER work is related to the creation of curricular materials.
It was suggested that developers should report enough background information regarding self-testing so that, in principle, the testing would be reproducible by other groups. For instance, developers could provide specific diagnostic questions that others could use in their own assessments. Reports of this type of validation study should be published and disseminated in some fashion so they might be evaluated by peers. It might be possible to publish the validation studies by themselves, without extensive additional commentary.

There was discussion as to whether it might be possible to have a “validation stamp” of some type for curricular materials, provided by AAPT or some related group. There was skepticism about the practicality of this approach and the discussion was inconclusive. A separate question arose as to whether potential users actually cared much about peer review of curricular materials they might be considering. It is not clear that either peer review or formal validation studies play a significant role in convincing instructors to test or use new materials.

Printed Curricular Materials in Book, On-line, or other Formats

Research-based curricular materials are becoming available in increasing numbers of formats, both printed and electronic. The issue of peer review is obviously a key concern. It was proposed that on-line reviews of curricular materials (a single review or perhaps multiple reviews by users) might be posted on PER-Central or other sites.

A key issue is to assess the advantages and disadvantages of on-line accessibility to curricular materials. Among the advantages are wide availability to users, and relative ease for developers to update and modify the materials. If materials are made available only on CD, for example, they can be relatively hard for the developer to modify or update. On the other hand, some developers would have a concern that, in some cases, it might be too easy for potential users to “misuse” on-line materials in ways not intended by developers (e.g., leave out important parts, modify files, etc.) and disseminate the altered materials. However, this is strongly dependent on the dissemination format since, for some materials such as computer animations, modifications by users may be very difficult to carry out. Examples of such materials are the Colorado PhET animations. These have been disseminated on CD to some extent; however, they are primarily intended for on-line use and thus they are easy for the developers to update. Another option is for open-source-style dissemination done online via a “hidden” website, for which access is given mainly to non-PER instructors who attend workshops or directly contact the curriculum developers. An example of this is the Intermediate Mechanics Tutorials [http://www.compadre.org/per/items/detail.cfm?ID=5522].

Graduate Students and Post-Docs

The group discussed the various incentives and disincentives for graduate students and post-docs to publish. One issue dealt with appropriate publication venues, e.g., are they the same as or different from those for faculty? It was felt that students can start by writing proceedings papers and similar short items for publication, but that post-docs should also be encouraged to publish in major journals.

Dissemination of Grant Proposals

It is possible that voluntary posting of funded or unfunded grant proposals, perhaps after some delay (e.g., 1-2 years), could benefit both the poster via dissemination of their work, and the PER community by providing a model of fundable work presented in a successful proposal. The Group wondered whether, in addition to individual researchers’ web sites, PER-Central might be used for this purpose.

Additional Issues for Discussion

The group discussed a number of other issues without reaching consensus on appropriate recommendations. A number of questions were raised. Among the issues discussed were these:

1. Obtaining tenure removes one of the significant incentives to publish, and after obtaining tenure faculty members may focus on other activities determined by their own or their institution’s interests. This could be seen as a problem for the community as a whole, since significant work may not get adequately disseminated. What changes might be made to alter this situation—or should it be changed?
2. There is a distinction between “research-based” materials (which employ results of research) and “research-validated” materials (which have gone through a testing and validation process employing research methods and techniques). These distinctions can be important; should they be emphasized more strongly than is commonly done?
3. PRST (and numerous other journals used in PER) are not yet indexed on the Web of Science (Science Citation Index). What impact does this have, and what are the prospects for it to change? (Recently, Google Scholar may be growing in importance due to more comprehensive scope.)

David E. Meltzer is a Senior Research Scientist with the PhysTEC project and a Research Scientist in the Department of Physics at the University of Washington. He is also the 8th-grade science teacher at Seattle Country Day School.