user manual aqa examiner online standardisation system v … · user manual aqa examiner online...

User manual AQA examiner online standardisation system V 4.1

June 2013

User Manual AQA examiner online standardisation V 4.1 release June 13

Amendment history Version Date Author Status

V 4.1.0 18/10/12 Jasmin Heyer Review AQA

Change register 4.1.0 Change to the CD of AQA.


Table of contents

1. General requirements ................................................................................................................. 4

2. Login ............................................................................................................................................ 5

3. Wrong Password ......................................................................................................................... 6

4. Examiner Technical Contact Centre ........................................................................................... 6

5. Choose component ...................................................................................................................... 7

6. System personalisation ............................................................................................................... 7

7. Component and end date ........................................................................................................... 7

8. Status ........................................................................................................................................... 7

9. Task list ........................................................................................................................................ 8

10. Logout .......................................................................................................................................... 9

11. Change component ..................................................................................................................... 9

12. Important notice .......................................................................................................................... 9

13. Messaging - examiner .............................................................................................................. 10

14. Messaging - Team Leader, Principal Examiner, Assistant Principal Examiner, Subject Manager ..................................................................................................................................... 12

15. Examiner welcome .................................................................................................................... 14

16. Supporting information ............................................................................................................ 15

17. Admin briefing ........................................................................................................................... 15

18. Question paper and mark scheme ........................................................................................... 17

19. Read model marked script (MMS) ............................................................................................ 19

20. Standardisation scripts - overview .......................................................................................... 23

21. Standardisation scripts - marking ........................................................................................... 23

22. Annotations ................................................................................................................................ 29

23. First phase sample - overview ................................................................................................. 31

24. First phase sample - marking .................................................................................................. 33

25. Tutorial ...................................................................................................................................... 35

26. Frequently asked questions (FAQ) ............................................................................................ 35


4

1. General requirements

Screen resolution:

The application is optimised for a screen resolution of 1024 x 768 minimum.

Plug-ins:

In order to use the system, Flash 9 plug-in is required.

Connection: Broadband internet access is required.

Windows Macintosh Linux

CPU Intel Pentium III 1GHz or faster processor (or equivalent)

PowerPC G3 500MHz or faster processor Intel Core Duo 1.33GHz or faster processor

Modern processor (1GHz or faster)

Memory 1 GB of RAM

Internet connection Access to a stable Broadband connection

Resolution Minimum 1024x768

Browser MS Internet Explorer (6.0 or later), Mozilla Firefox (2.0 or later)

With Flash Player Plugin (9 or later)


5

2. Login

• Type the following into your address bar: Uhttps://www.onlinestandardisation.co.uk/U

• The following window appears:

• Type in your UserID (user name) and the Password given to you in advance.

• Click on the Proceed button.


6

3. Wrong Password

• If you enter three times an incorrect password, the following window appears:

• You have two more chances to type in the correct password and solve the captcha; otherwise, your account will be blocked and you will receive an email with the contact details to receive a new password.

4. Examiner Technical Contact Centre Our Contact Centre (see Login screen for details) helps with system operating problems, e.g. a page not loading properly or missing, previously viewed, information.

Our email address and telephone number are also available at the top right of most pages:


7

5. Choose component You can switch between different OLSs by selecting the respective tab.

Make sure the correct tab is selected.

On the welcome screen you will find some welcome text and several buttons.

• Click on a component’s play button to select it and you will be taken to its Welcome page.

6. System personalisation Your personal details are shown at the top of the screen.

7. Component and end date Your chosen component and the end date (by which online standardisation must be finished) are shown at the top of the screen.

8. Status The progress bar shows your progress through the application.


8

The status breakdown is as follows:

• Welcome 5%

• Admin briefing 11%

• Question paper & mark scheme 11%

• Read model marked script 11%

• Standardisation scripts 29%

• First phase sample batch 1(if in use) 11%

• First phase sample batch 2(if in use) 11%

• First phase sample batch 3 (if in use) 11%

9. Task list

• The list of online standardisation tasks is at the top left of the page.

• The tasks have to be worked through in order.

• Each task becomes active as you move through the process.

• The date and time alongside each task tell you when it will be accessible.

• The task list can be closed by clicking any menu item.

• The task list can be opened again by clicking the arrow in the top right corner of the drop-down list.


9

10. Logout • You can click Logout at any time.

’

11. Change component You can switch between the components you are appointed to without logging out; simply click on Change Component.

The ‘Choose Component’ screen with your former selection will appear. Click on the play button of another component to access it

12. Important notice If a new version of an important document - e.g. the Mark Scheme - is published during standardisation you will receive an update alert (see below) when you next log in.

You have to acknowledge this notice (by clicking OK) before you can do anything else. The notice will then disappear.


10

Before continuing, check the new version of the mark scheme. There may be changes you need to be aware of before marking the standardisation or first phase sample scripts.

13. Messaging - examiner The Messages box is in the top right corner of the screen.

The button shows the number of new messages.

• Click this button, the screen turns grey and the Messaging window comes up.

Messaging works as follows:

• the left column lists all threads

• the right column shows all the messages


11

• Click on the horizontal triangle to extend the description and get a chronological list of all messages about this thread.

• To read a message click on the author. The message comes up in the right box.

• To reply click on Reply to Team Leader. The subject is inserted automatically, type your message in the box and click Send.


12

• To compose a new message relating to content in the system, click on the button with the thread in it, type your message in the box and click Send.

• To start a new thread, click on General. You must type in the name of the subject so that readers can find your message. When you have typed your message, click Send. A new thread will come up in your and your Team Leader’s thread box showing this message.

14. Messaging - Team Leader, Principal Examiner, Assistant Principal Examiner, Subject Manager

The Messages box is in the top right corner of the screen.

The button shows the number of new messages.

Click this button, the screen turns grey and the Messaging window comes up.

Messaging works as follows:

• the left column lists all users you can communicate with

• you can send messages to all examiners in your team, your Principal Examiner if you are a Team Leader, or the Subject Manager if you are a Principal Examiner

• the middle column has a list of all the threads

• the right hand column shows the messages

• click on a username to see all corresponding threads in the middle column

• click on the triangle to the left of a thread to extend the description and get a [Wort bzw. Woerter fehlen?]


13

• To read a message, click the author’s name. The message is displayed in the box on the right.

• To reply directly click on Reply, type your message and click Send

• To compose a new message about content in the system, choose one or more of the shown users, click on the button with the thread in it, type your message in the right column and click Send.

• To send your message to all examiners click the square next to all Examiners.

• To start a new thread, click on General. You must type in the name of the subject so that readers can find your message. When you have typed your message, click Send.

• To create emails/system messages within the message box, click on the via email checkbox at the top. Messages will be marked with special symbols.

System message e-‐mail


14

15. Examiner welcome

The Principal Examiner for the component welcomes you to online standardisation.

After this page has been displayed on your first login, the progress bar at the top of the page changes to 5%.

• Read through the Principal Examiner Welcome and any supporting information (see below) before proceeding to the next stage.

The Welcome page will also contain:

• Team Leader contact details

• general information about examiner OLS

• a news section with the latest information

• the option to download the tutorial


15

16. Supporting information

The Supporting Information box on the welcome page contains additional material for examiners. To view a document, simply click on it.

Supporting Information is also accessible from the task list.

17. Admin briefing

• Click on Admin Briefing in the task list to see the following page.


16

• Click on the word here to open the Admin briefing. You will need Adobe Acrobat Reader on your computer to read it.

• Confirm that you have read the briefing by ticking the I have studied Admin Briefing checkbox and clicking submit. You must do this before you can move on to the next task.

Your progress status will change to 11% and the next task will be available to you.


17

18. Question paper and mark scheme

Click on Question Paper & Mark Scheme to display the following page which shows the breakdown of question paper and mark scheme by question or item.

• Click on the squares alongside the question number to go directly to the relevant question or sub-question.

• Click on the Question Paper tab to bring up either the first or last viewed question.

• To print the whole question paper, click the top right printer icon. This will open a PDF document that you can save or print immediately.

• To switch from the standard top-bottom layout to one of the other two available, click the appropriate screen layout icon in the top right corner.


18

• Use the scrollbar to scroll up and down the page of any question which is too long to be displayed on the screen.

• Use the zoom slider on the left to enlarge or reduce the question resolution.

• The left hand corner question numbers take you to the other questions.

• Click on the Mark Scheme tab to go to the relevant part of the mark scheme.

• Click the top right printer icon to open the whole mark scheme. You can then save or print the document.

• Use the zoom function to enlarge/reduce the text you are viewing.

Multi-page feature

Long clips from the question paper, mark scheme or candidate responses can be displayed over several pages.

Click the arrows to go through the extract page by page.

Click a page number in the white box to go directly to that page.

The double arrows lead you to the top or bottom of the document.

Having read the question paper and mark scheme, click the Overview tab.

Tick the checkbox and click on submit. The next task will now be available.


19

19. Read model marked script (MMS) This task is available for non CMI+ components only.

Clicking on Read MMS brings up the following page:

The page shows the question paper by question/item.


20

• Click on the squares alongside the question numbers to go to the relevant question in the model marked script and the corresponding part of the mark scheme. You can also click on the Model Marked Script tab to go to the first or last viewed question of the MMS.

• Use the right hand scroll bar to scroll up and down questions that are too long to be displayed on the screen.

• A paging function is available for these multi-page questions.

• Use the zoom sliders to enlarge or reduce the size of the question and the mark scheme.

• To move the border between the two sections of the page, grab the slider with the mouse and move it until the layout fits your needs.

• To switch from the standard top-bottom layout to one of the other two available, click the appropriate screen layout icon in the top right corner.

This is the side-by-side display option:


21

• To move the border between the two sections of the page in this layout, grab the slider with the mouse and move it until the layout fits your needs.

This is the model script-only display option. Where you have a hard copy mark scheme, you may prefer this option.


22

• You can switch from one question to another on all display options by clicking on the next button.

• You can also use the left hand question numbers to navigate to any other question.

• Click on the Overview tab to acknowledge you have read the MMS.

• Tick the checkbox and click submit. The next task will now be available.


23

20. Standardisation scripts - overview The overview screen shows your progress through the standardisation scripts:

• white fields are not yet marked

• green fields show that your marks for this question are correct

• amber and red fields show your marks and the correct marks

Where the box is red, the mark difference exceeds the tolerance set. Amber boxes indicate a mark difference within tolerance.

• After you have marked all scripts, acknowledge the Standardisation Scripts stage and click the submit button.

21. Standardisation scripts - marking

When you have marked all standardisation scripts you will be cleared for:

• the first phase sample (non CMI+ only)

• to mark (CMI+ only)


24

The system gives instant, item by item feedback on marking. Use the messaging system to contact your Team Leader and discuss this feedback.

• You can choose to work through the process by question or by script. Change your preference by checking the button in the top right menu.

• To print the whole script with your marks and the agreed marks, click the top right printer icon. This will open a PDF document that you can save or print immediately.

The numbers of the questions or items in the current script are shown on the left hand side.

• Switch to another question/item in the script by clicking on the question or item number. Clips with multiple pages have a label at the bottom. You can browse through the corresponding pages for one clip.

The number of the current scripts and the status of the different scripts are shown at the top of the page. Possible script statuses are:

• not worked through (black font on bright blue background)

• currently in process (black font on yellow background)

• worked through (black font on blue background, checked off)

• navigate to another script by clicking on the script number


25

• To position a mark, click in the question window. A marking box with the available mark types and symbols will appear.

• Click on the mark symbol and a value box comes up where you click on the chosen value. You do not have to close the box or save. The mark is set by clicking the value. Alternatively you can use the arrow left/right/up/down keys or a number on your keyboard to select a mark. Press enter on your keyboard to confirm the selected mark. The tabs button on your keyboard allows you to jump between mark type or symbol.

• To annotate a mark, set your annotation beforehand in the text field above the mark type box.


26

You can award a symbol to the question/item rather than a mark:

• click on a symbol in the bottom of the marking box, the box will be closed and your symbol set

• to annotate the symbol do the annotation before choosing a symbol


27

Entered marks show up in red in a box at the bottom of the question.

• You can award more than one mark for the same question by clicking in the script again. Multiple marks for the same question are totalled up in the yellow submit field at the bottom of the screen.

• To change or delete a mark, click on it and make your changes either using the user interface or your keyboard. If you use the keyboard, use the arrow left/right/up/down buttons or any number on your keyboard and press enter to confirm. The tabs button on your keyboard allows you to jump between mark type or symbol.


28

• Click ‘submit’ to send your marks to the system. The submit button is greyed out until at least one mark, including zero, has been assigned to the question.

• The system gives immediate feedback. The system marks are shown in green on the script and in the right hand summary box.

• Review the correct marks and proceed by clicking the next button.

• Once a mark is submitted, it cannot be changed, so be sure that you are satisfied with the mark/symbols for that item before you click submit.


29

22. Annotations For each mark you can add a comment for yourself and your Team Leader.

• Add an annotation by typing your comment in the text field on top of the marking box.

• Write your comment in the text field before entering a mark.

• To edit your existing annotations, click on the mark/symbol, make your text changes and save.

Please note: if you use the Delete symbol (bottom left), you also delete the mark.


30

Your annotation appears as a red notes symbol next to the mark.

System feedback annotation appears as a green notes symbol next to the green marks.

• To view the annotation, place the mouse over the red or green notes symbol.


31

You can underline by using the Line tool below the symbol box.

• Lines can be moved by Drag and Drop.

• Lines can be made shorter or longer

• Lines can be deleted

• Lines are displayed in red

23. First phase sample - overview

After standardisation scripts you need to mark first phase sample scripts (if the component is not being marked by CMI+). A successful first phase sample clears you to mark candidates’ live scripts.

• Mark first phase sample scripts in the same way as standardisation scripts.

The overview screen shows your progress through the first phase sample scripts:

• white boxes show questions which have not been marked

• ticked white boxes show marked questions.

The first phase sample is available as batch 1 and batch 2.


32

The first phase sample batch cannot be submitted until all questions have been marked.

• To submit the first phase sample batch, acknowledge it and click the submit button.

The system provides immediate feedback:

If the component has tolerances which have been set, the system shows the following

• green/green fields show that your marks are correct

• amber/green and red/green fields show your marks above the correct marks

• red shows the mark difference has exceeded tolerance

• amber shows a mark difference within tolerance

It may be that the senior examiners have chosen not to set any tolerance and in this case the marks will still show, but they will all be green.


33

24. First phase sample - marking The first phase sample is the final process qualifying examiners for marking. To qualify, all scripts from batch one and batch two have to be marked completely.

• You can contact your Team Leader for feedback using the internal messaging system.

Once you have submitted a batch the system will give you immediate feedback. Final clearance does not come from the system - this is confirmed by the Team Leader by telephone. Otherwise you will get another batch.

First phase sample scripts are displayed by question or by script, depending on your preference. The numbers of the scripts and questions are displayed.

• Click the script/question numbers to change them.

First phase marking is identical to marking in standardisation (see Section 21).

• When you have marked a question, save your marks in the system by clicking the submit button.

Sample Comparison Report (SCR) and Team Leader comment

When you have started the first phase sample batch 1, the Team Leader can add a comment for you in the overview screen.

This comment will also appear on the Sample Comparison Report which can be opened as a PDF or an Excel file.


34

Example of SCR Excel file


35

25. Tutorial

• Use the task Tutorial to go to the online standardisation tutorials on the AQA Examiner Extranet.

Tutorial button

The Tutorial button is at the top of the Choose Component section.

This will take you to the AQA website with several online standardisation tutorials.

26. Frequently asked questions (FAQ) • You can always click on the FAQ option in the task list to see typical questions asked by

system users.


36

• If you have questions not covered by the FAQ section, contact the AQA Examiner Technical Contact Centre on the telephone number or email address at the top right of the page.

FAQ button

The FAQ (Frequently Asked Questions) button is at the top of the Choose Component section.

If you have a question not covered in the FAQ section, contact the AQA Examiner Technical Contact Centre using the number at the top right of the page.

user manual aqa examiner online standardisation system v … · user manual aqa examiner online...

Documents