Instructions for Using the Claim Scope Annotator App

Welcome to the Claim Scope Annotator web app. This tool is designed for patent specialists to compare pairs of patent claims and assess their relative scope. The following instructions will guide you through using the app effectively.

Main Interface

Upon launching the app, you'll see the main interface as shown in the image above. This includes two claim text boxes, a slider for scope assessment, checkboxes for clarity and compatibility issues, and control buttons.

Claim Display

At the top of the interface, you'll find two text boxes labeled "Claim A" and "Claim B". These display the patent claims you'll be comparing. Read both claims carefully before making your assessment.

Timer

In the top right corner, you'll see a timer. This timer is for reference only and does not impose a time limit on your evaluation. It's designed to help you track your pace. You can pause or resume the timer using the adjacent play/pause button if you need to step away. Note that when paused, the claim text will be blurred to maintain consistency in timing data.

Scope Assessment Slider

Below the claim boxes, you'll find a horizontal slider. This is your primary tool for indicating which claim has a broader scope. Move the slider towards the claim you consider broader. The further you move it from the center, the greater your confidence in the scope difference.

Above the slider, two colored bars visually represent your scope assessment. As you move the slider, these bars will adjust in size. The larger bar corresponds to the claim you've indicated as having a broader scope. Additionally, the bars include a text overlay showing a corresponding categorisation of confidence in your current scope assessment.

Clarity and Compatibility Checkboxes

Below the slider, you'll find three checkboxes:

• Two "clarity" checkboxes labeled "Claim A is unclear or otherwise unsuitable for comparison" and "Claim B is unclear or otherwise unsuitable for comparison"
• One "compatibility" checkbox labeled "Claims A and B are individually clear, but cannot meaningfully be compared"

Use the "unclear" checkboxes when, for example, you cannot make sufficient sense of one, or both, claims to form an opinion on the relative scope of the two claims. Use the "compatibility" checkbox when the claims are incomparable for another reason, for example if the system has erroneously presented claims that are in different categories, or that relate to different technologies.

Note: if more than one participant selects the "compatibility" checkbox for a given claim pair, the system will automatically exclude that pair from the dataset for all future participants. Similarly, if more than one participant selects an "unclear" checkbox for a given claim, the system will automatically exclude all claim pairs which include that claim from the dataset for all future participants. If you are unsure about clarity or compatibility, you also have the option to pass on the claim pair (see below).

Control Buttons

At the bottom right of the interface, you'll see three buttons:

• "Submit": Use this to send your evaluation and move to the next claim pair.
• "Pass": Click this to skip the current pair without submitting an evaluation and move on to the next claim pair. You can use this if you're unsure about the clarity or compatibility of the claims, if you are not sufficiently comfortable with the technology they cover, if the example is too difficult to evaluate, or for any other reason that you feel unable to evaluate the claim pair.
• "Exit": Use this to leave the annotation session. You can return any time, for as long as the evaluation project is active.

If you choose to pass on evaluating a claim pair, you will be asked to provide a reason for passing. This information will help us to improve the dataset and the app for future users.

Help Icons

Throughout the interface, you'll see question mark icons (?) next to various controls. Click on these for specific guidance about that particular feature.

Evaluation Summary

At the bottom of the page, you'll find statistics about your progress, including the number of evaluations you've completed, the number of claim pairs remaining that match the technical fields identified in your profile, and average evaluation times for you and all users.

Important Notes for Patent Specialists

We acknowledge that this is not an exercise in formal claim construction, particularly because we are not providing the full patent specification and other context for each claim. In this sense, it may be regarded as an atypical task for patent specialists.

While quality of data is important when training AI models, the quantity of data is also crucial. We are relying on your expertise and experience to make relatively rapid assessments that are of sufficient quality as a matter of first impression.

Most pairs of claims that are presented to you will relate to a common invention, i.e. they are drawn from the same patent or from two related patents. However, some pairs relate to different inventions in the same field of technology. While evaluating such pairs may be more challenging, we ask you to consider the comparative scope of the claims in each pair relative to the relevant field(s) of technology, based on your knowledge and experience. As a last resort, if you are unable to make a meaningful comparison, you can use the "compatibility" checkbox to indicate this.

Please keep in mind:

• The app relies on your expertise as a patent specialist to make informed assessments efficiently.
• While there's no time limit, the app is designed for reasonably quick evaluations based on your professional judgment.
• Use the checkboxes as a last resort, when claims are irredeemably unclear or incomparable.
• Submit your evaluation once you've made your assessment; avoid second-guessing your initial expert judgment.

Your expertise in patent law is crucial for this task. We appreciate your valuable contributions to this annotation project. If you encounter any issues or have questions, please contact contact Dr Mark Summerfield at m.summerfield@student.unimelb.edu.au.