Would researchers scrawl notes, critiques and comments across online research papers if software made the annotation easy for them? Dan Whaley, founder of the non-profit organization Hypothes.is, certainly thinks so.
Whaley's start-up company has built an open-source software platform for web annotations that allows users to highlight text or to comment on any web page or PDF file. And on 1 December, Hypothes.is announced partnerships with more than 40 publishers, technology firms and scholarly websites, including Wiley, CrossRef, PLOS, Project Jupyter, HighWire and arXiv.
Whaley hopes that the partnerships will encourage researchers to start annotating the world's online scholarship. Scientists could scribble comments on research papers and share them publicly or privately, and educators could use annotation to build interactive classroom lessons, he says. If the idea takes off, some enthusiasts suggest that the ability to annotate research papers online might even change the way that papers are written, peer reviewed and published.
Hypothes.is, which was founded in 2011 in San Francisco, California, and is supported by philanthropic grants, has a bold mission: “To enable conversations over the world's knowledge.” But the concept it implements, online annotation, is as old as the web itself. The idea of permitting readers of web pages to annotate them dates back to 1993; an early version of the Mosaic web browser had this functionality. Yet the feature was ultimately discarded. A few websites today have inserted code that allows annotations to be made on their pages by default, including the blog platform Medium, the scholarly reference-management system F1000 Workspace and the news site Quartz. However, annotations are visible only to users on those sites. Other annotation services, such as A.nnotate or Google Docs, require users to upload documents to cloud-computing servers to make shared annotations and comments on them.
Hypothes.is is not the only service that wants to make it easy for users to leave annotations across the entire web. A competing offering is a web annotation service from Genius, a start-up firm that began as a site for annotating rap lyrics. In April, it launched services such as browser plugins to help users to annotate any web page. But unlike Hypothes.is, the Genius code is not open-source, its service doesn't work on PDFs, and it is not working with the scholarly community. On the scholarly side, the reference-management tool ReadCube makes it possible for users to annotate PDFs of papers viewed on a ReadCube web reader — but that software is proprietary. (ReadCube is owned by Digital Science, a firm operated by the Holtzbrinck Publishing Group, which also has a share in Nature's publisher.)
By contrast, the open-source nature of the Hypothes.is platform means that anyone could use it to create their own annotation reader or writer — just as anyone can create their own web browser using standards-based technology. The company is also a member of a working group within the World Wide Web Consortium, the standards body for the web, which is developing a universal standard for annotations and how they are communicated. The hope is that web pages that allow annotations would all adopt the same underlying code and protocols (as they do with hyperlinks, for example), making the function easier to use and interact with. The working group has released a draft version of its standard, which is expected to be finalized by the end of 2016.
How it works
For now, Hypothes.is users have several options for creating and viewing annotations. These include bookmarklets (a simple program within a browser bookmark), browser plugins or adding 'via.hypothes.is/' to the start of any URL. [To see public annotations on this article in hypothes.is, for example, visit https://via.hypothes.is/http://www.nature.com/news/annotating-the-scholarly-web-1.18900.]
When a Hypothes.is user opens a page — a scholarly article, for instance — the web browser shows any annotations to which the user has access. These appear as highlighted words and comments on top of the text, like an overlaid transparency. Users can then add their own comments, similar to a student marking up a textbook. These are public by default but can be made private, and, following an update added on 3 November, annotations can be shared with private groups. That should enable the tool to be used for journal clubs, classroom exercises and even peer review.
If a page has been altered since an annotation was made, the software uses 'fuzzy' logic to map annotations to their approximate original location. The system can also map annotations from HTML to PDF and back again (for instance, if a user annotates the web version of an article and subsequently views a PDF of the same document).
Annotations are stored on a dedicated Hypothes.is server, which Whaley says looks set to log around 250,000 comments from some 10,000 users in 2015. For instance, after Hurricane Patricia in October, climate scientists left comments and highlighted text on a widely shared mashable.com article (see go.nature.com/rcsesf). But publishers that wish to host annotations for their own content, or companies that want to annotate corporate documents behind a firewall, could run their own server using the same software platform, Whaley adds.
A Hypothes.is user can already annotate any web page — including research papers and pay-to-view articles to which they have access. But the formal partnership announced this week sees some publishers working harder to encourage annotation, including tackling content that annotation systems stumble over, such as page frames and embedded page readers.
The digital library JSTOR, for example, is developing a custom Hypothes.is tool for its educational project with the Poetry Foundation, a literary organization and publisher in Chicago, Illinois. Alex Humphreys, who is director of JSTOR Labs in New York City, says that teachers will be able to use the tool to annotate poems with their classes. An instructor selects the poem to be annotated, sets up a dedicated page with a copy of it, and restricts access to their class only. Students can then create personal notes or share them with the group; an extra annotation layer finds the scholarly resources in JSTOR that quote each line of poetry and provides links out to those resources. The tool is slated to launch in mid-December, Humphreys says.
The scientific publisher eLife in Cambridge, UK, has been testing the feasibility of using Hypothes.is to replace its peer-review commenting system, says Ian Mulvany, who heads technology at the firm. The publisher plans to incorporate the annotation platform in a site redesign instead of its current commenting system, Disqus. At a minimum, says Mulvany, Hypothes.is provides a mechanism for more-targeted commentary — the equivalent of moving comments up from the bottom of a web page into the main body of the article itself.
Another partner, the arXiv preprint service run by Cornell University Library in Ithaca, New York, has been working on making annotations flow across multiple article versions, says information scientist Simeon Warner, who leads technology development for arXiv. To jump-start interest in the annotation program, arXiv has been converting mentions of its articles in external blog posts (called trackbacks) into annotations that are visible on an article's abstract page when using Hypothes.is.
Not just graffiti
Hypothes.is plans improvements to its platform that include a way to validate the identities of commenters, by incorporating researchers' unique ORCID digital profiles. That could go a long way towards improving adoption of the system among scholars, by facilitating expert commentary on published works and filtering out unwanted marginalia, says Paul Ginsparg, the founder of arXiv and a physicist at Cornell University. “If people start looking at articles and they see the equivalent of graffiti, then people will turn off the comments and the experiment will fail,” he says.
If it takes off, online annotation could represent a fundamental shift in the way scholarly communication is done, adds Cameron Neylon, part of the research team at the Centre for Culture and Technology at Curtin University in Perth, Australia, who formerly worked at PLOS.
At the moment, Neylon explains, the scholarly publishing process involves ferrying a document from place to place. Researchers prepare manuscripts, share them with colleagues, fold in comments and submit them to journals. Journal editors send copies to peer reviewers, returning their comments to the author, who goes back and forth with the editor to finalize the text. After publication, readers weigh in with commentary of their own.
With an open-source annotation platform, Neylon says, the document is the centre of attention. Different contributors act on the content simply by changing who has access to it and its comments, with the document becoming richer over time. “You can think of this as a fabric that allows those comments to move freely both in time and [across] versions in a way that we've never been able to do before,” he says.
But as Ginsparg points out, it is not clear that researchers — who have proved reluctant in repeated trials to comment on published articles — will take to annotation, even if they can share their comments privately. “There's no incentive structure for people to comment extensively, because it can take time to write a thoughtful comment, and one currently doesn't get credit for it,” he says. “But it's an experiment that needs to be done.”
Illustration by The Project Twins
“You can think of this as a fabric that allows those comments to move freely in time and [across] versions.”
- Journal name:
- Date published:
- Descriptive: This annotation describes the source without summarizing the actual argument, hypothesis, or message in the content. Like an abstract, it describes what the source addresses, what issues are being investigated, and any special features, such as appendices or bibliographies, that are used to supplement the main text. What it does not include is any evaluation or criticism of the content. This type of annotation seeks to answer the question: Does this source cover or address the topic I am researching?
- Informative/Summative: This type of annotation summarizes what the content, message, or argument of the source is. It generally contains the hypothesis, methodology, and conclusion or findings, but like the descriptive type, you are not offering your own evaluative comments about such content. This type of annotation seeks to answer these types of questions: What are the author's main arguments? What conclusions did the author draw?
- Evaluative/Critical/Analytical: This annotation includes your evaluative statements about the content of a source. It is the most common type of annotation your professor will ask you to write. Your critique may focus on describing a study's strengths and weaknesses or it may describe the applicability of the conclusions to the research problem you are studying. This type of annotation seeks to answer these types of questions: Is the reasoning sound? Is the methodology sound? Does this source address all the relevant issues? How does this source compare to other sources on this topic?
NOTE: Strategies about how to critically evaluate a source can be found here.
II. Choosing Sources for Your Bibliography
There are two good strategies you should use to begin identifying possible sources for your bibliography--one that looks back into the literature and one that looks forward.
- The first strategy is to identify several recent scholarly books or journal articles on the topic of your annotated bibliography and review the sources cited by the author(s). Often, the items cited by an author will effectively lead you to related sources about the topic.
- The second strategy is to identify one or more important books, book chapters, journal articles, or other documents on your topic and paste the title of the item in Google Scholar [e.g., from Negotiation Journal, entering the article, "Civic Fusion: Moving from Certainty through Not Knowing to Curiosity"], placing quotation marks around the title so Google Scholar searches as a phrase rather than a combination of individual words. Below the citation may be a "Cited by" reference followed by a linked number. This link will direct you to a list of other study's that have cited that particular item after it was published.
Your method for selecting which sources to annotate depends on the purpose of the assignment and the research problem you are investigating. For example, if the research problem is to compare the social factors that led to protests in Egypt with the social factors that led to protests against the government of the Phillippines in the 1980's, you will have to consider including non-U.S., historical, and, if possible, foreign language sources in your bibliography.
NOTE: Appropriate sources to include can be anything that has value in understanding the research problem. Be creative in thinking about possible sources, including non-textual items, such as, films, maps, photographs, and audio recordings, or archival documents and primary source materials, such as, diaries, government documents, collections of personal correspondence, meeting minutes, and official memorandums. Consult with a librarian if you're not sure how to locate these types of materials for your bibliography.
III. Strategies to Define the Scope of your Bibliography
It is important that the sources cited and described in your bibliography are well-defined and sufficiently narrow in coverage to ensure that you're not overwhelmed by the number of potential items to consider including. Many of the general strategies used to narrow a topic for a research paper are the same that you can use to define the scope of your bibliography. These are:
- Aspect -- choose one lens through which to view the research problem, or look at just one facet of your topic [e.g., rather than a bibliography of sources about the role of food in religious rituals, create a bibliography on the role of food in Hindu ceremonies].
- Time -- the shorter the time period to be covered, the more narrow the focus [e.g., rather than political scandals of the 20th century, cite literature on political scandals during the 1930s and the 1990s].
- Geography -- the smaller the region of analysis, the fewer items there are to consider including in your bibliography [e.g., rather than cite sources about trade relations in West Africa, include only sources that examine trade relations between Niger and Cameroon].
- Type -- focus your bibliography on a specific type or class of people, places, or things [e.g., rather than health care provision in Japan, cite research on health care provided to elderly men in Japan].
- Source -- your bibliography includes specific types of materials [e.g., only books, only scholarly journal articles, only films, etc.]. However, be sure to describe why only one type of source is appropriate.
- Combination -- use two or more of the above strategies to focus your bibliography very narrowly or to broaden coverage of a very specific research problem [e.g., cite literature only about political scandals during the 1930s and the 1990s and that have only taken place in Great Britain].
IV. Assessing the Relevance and Value of Sources
All the items you include in your bibliography should reflect the source's contribution to understanding the research problem or the overall issue being addressed. In order to determine how you will use the source or define its contribution, you will need to assess the quality of the central argument within the source. Specific elements to assess include an item’s overall value in relation to other sources on the topic, its limitations, its effectiveness in defining the research problem, the methodology used, the quality of the evidence, and the author’s conclusions and/or recommendations.
With this in mind, determining whether a source should be included in your bibliography depends on how you think about and answer the following questions related to its content:
- Are you interested in the way the author frames the research questions or in the way the author goes about answering it [the method]?
- Does the research findings make new connections or promote new ways of understanding a problem?
- Are you interested in the way the author uses a theoretical framework or a key concept?
- Does the source refer to and analyze a particular body of evidence that you want to cite?
- How are the author's conclusions relevant to your overall investigation of the topic?
V. Format and Content
The format of an annotated bibliography can differ depending on its purpose and the nature of the assignment. Contents may be listed alphabetically by author or arranged chronologically by publication date. If the bibliography includes a lot of sources, items may also be subdivided thematically or by type. If you are unsure, ask your professor for specific guidelines in terms of length, focus, and the type of annotation you are to write.
Your bibliography should include a brief introductory paragraph that explains the method used to identify possible sources [including what sources, such as databases, you searched], the rationale for selecting the sources, and a statement, if appropriate, regarding what sources were deliberately excluded and the reasons why.
This first part of your entry contains the bibliographic information written in a standard documentation style, such as, MLA, Chicago, or APA. Ask your professor what style is most appropriate and be consistent!
The second part should summarize, in paragraph form, the content of the source. What you say about the source is dictated by the type of annotation you are asked to write. In most cases, however, your annotation should provide critical commentary that examines the source and its relationship to the topic. Things to think critically about when writing the annotation include: Does the source offer a good introduction on the issue? Does the source effectively address the issue? Would novices find the work accessible or is it intended for an audience already familiar with the topic? What limitations does the source have [reading level, timeliness, reliability, etc.]? Are any special features, such as, appendices or non-textual elements effectively presented? What is your overall reaction to the source? If it's a website or online resource, is it up-to-date, well-organized, and easy to read, use, and navigate?
Annotations can vary significantly in length, from a couple of sentences to a couple of pages. However, they are normally about 300 words. The length will depend on the purpose. If you're just writing summaries of your sources, the annotations may not be very long. However, if you are writing an extensive analysis of each source, you'll need to devote more space.
Annotated Bibliographies. The Writing Center. University of North Carolina; Annotated Bibliographies. The Writing Lab and The OWL. Purdue University; Annotated Bibliography. The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Annotated Bibliography. Writing Center. Walden University; Engle, Michael et al. How to Prepare an Annotated Bibliography. Olin Reference, Research and Learning Services. Cornell University Library; Guidelines for Preparing an Annotated Bibliography. Writing Center at Campus Library. University of Washington, Bothell; Harner, James L. On Compiling an Annotated Bibliography. 2nd edition. New York: Modern Language Association, 2000; How to Write an Annotated Bibliography. Information and Library Services. University of Maryland; Knott, Deborah. Writing an Annotated Bibliography. The Lab Report. University College Writing Centre. University of Toronto; Norton, Donna. Top 32 Effective Tips for Writing an Annotated Bibliography Top-notch study tips for A+ students blog; Writing from Sources: Writing an Annotated Bibliography. The Reading/Writing Center. Hunter College.