reporting tool user guide

Reporting tool user guide

Contact

The surveys team [email protected]

General Medical Council

National training survey reporting tool user guide 2

Contents Page

Part 1 Getting started 6

Navigation | home button | legends 6

How to use filters | filter hints | using filters 7

Benchmark groups 10

Part 2 Outlier analysis 11

Menu page 11

Building charts 12

Outlier charts explained 13

Navigating to the all results view 14

All results view functions 15

All results view explained 16

Master class | creating a sorted report 17

Part 3 Indicator analysis 19

Menu page 19

Navigating the charts 20

Post specialty and programme group reports | controlling the horizontal axis 21

Post specialty and programme group reports | controlling the chart bars 22

Post specialty and programme group reports | additional reports 23

Other reports | controlling the horizontal axis 25

Other reports | additional reports 26

Question item reports 28

Master class | replicating your post specialty by trust outlier report 31



Contents Page

Part 4 Trend analysis 34

Menu page 34

Building charts 35


Trend analysis charts explained 37

Master class | creating a trend report for multiple specialties and a single indicator 38

Part 5 Programme specific reports 40

Menu page 40


Part 6 Aggregation reports 43

Menu page 43

Building charts 44

Additional reports 46

Question items 48

Part 7 Trainer reports 50

Menu page 50

Building charts 51

Viewing your charts 52

Navigating to the all results view 53

All results view functions 54

All results view explained 55



Contents Page

Part 8 Legends 56

Outlier and trend analysis legend 56

Indicator analysis legend 57

Part 9 How we calculate indicator outcomes 58

How answers to questions become a single score 58

Benchmark groups 59

Quartiles 60

Benchmark mean and confidence intervals 61

Report group mean 62

Outlier summary grid 63

Example outcomes explained 64

Part 10 Printing and downloading 65

Annexes Annex A – benchmark groups 66

Annex B1 – post specialty report groups and their benchmark groups 67

Annex B2 – post specialty benchmark groups 72

Annex C1 – programme group reporting and benchmark groups 74

Annex C2 – programme group benchmark groups 80

Annex D1 – programme type reporting and benchmark groups 82

Annex D2 – programme type benchmark groups 86

Annex E1 – GP scheme reporting and benchmark groups 87

Annex E2 – GP scheme benchmark groups 88

Annex F – trainee indicator questions 89

Annex G – trainer benchmark groups 102

Annex H – trainer indicator questions 103



Annex I – column heading definitions 112

Annex J - glossary 113



Part 1 | Getting started Navigation

The reporting tool is divided into five areas where you can find different types of report. Each area has its own menu.

After navigating to a report, you can return to the menu of a report area at any time by clicking the home button.

When in a report you can also find a legend explaining the colours and markers on the charts. See the section on legends below for more details.



Part 1 | Getting started How to use filters

After you’ve navigated to a report, you’ll need to apply filters to build your chart.



Part 1 | Getting started Filter hints

Searching for your choice is easier than scrolling if your category is not at the top of the list. Using the multi-select box as shown in the page before, change from a ‘begins with’ search to a ‘contains’ search to ensure you find what you’re looking for.



Part 1 | Getting started Using filters

You can clear filters easily.

Look in the Applied filters box to remind you what is included in the report you’ve built. Some filters are applied by default (in the red boxes in the image). Others show the filters you have applied (in the green box in the image).



Part 1 | Getting started Benchmark groups

All the results you will see in the reporting tool are derived from a comparison between a score generated by a smaller group of respondents (a report group) and a score generated by a larger group of respondents (a benchmark group). The benchmark group will always include the scores of the report group. To ensure survey results are shown in context, the benchmark group changes depending on the report. For example, the post specialty by trust/board report uses benchmark groups comprised of related specialties. Cardiology report groups are compared to a benchmark group comprising all UK medicine posts. Otolaryngology report groups are compared to a benchmark comprising surgical posts and so on. You can check the benchmark group for the report you’re looking at in the applied filters box (see previous page for details). You can read about how we calculate results in part 9 of this guide. By excluding less relevant groups from the benchmark calculations, the differences between scores are more likely to reflect differences between the trusts/boards, than the differences found between different specialties. All the benchmark groups are listed in Annex A.



Part 2 | Outlier analysis Menu page

Use outlier analysis to find the outcomes (including outliers, also known as red and green flags) for each indicator in a colour-coded grid for your chosen report. In the menu, the most popular reports are towards the top.



Part 2 | Outlier analysis Building charts

When you navigate to the report you want, you need to apply filters to build your report. In most cases you should start with the filter on the left, prompted by ‘Please select’. In some reports, there may be no chart displayed by default. If this happens, just apply the filters as normal. The guidance in part 1 explains how to use filters.



Part 2 | Outlier analysis Outlier charts explained

When you’ve applied your filters, your outlier chart will load. The legend explains what each colour means. From any of the by trust/board reports, you can navigate to by site reports by clicking the link on the right hand side, under Additional Reports.



Part 2 | Outlier analysis Navigating to the all results view

For a more detailed report, from your outlier chart, click on All Results. In the all results view you can filter on outcomes and indicators and rank your results.



Part 2 | Outlier analysis All results view functions

The all results view expands your outlier chart so that every coloured cell is now a row of numbers. If you had filters applied in the previous view, they will carry through to this chart. You can sort on each column now by hovering over the column header to prompt an ascending and descending arrow indicator to appear then select which direction you want.



Part 2 | Outlier analysis All results view explained

Each outcome is calculated using the figures shown in the data table. The light blue columns in the middle show the ‘local’ scores and their associated figures. The dark blue columns to the right show the ‘national’ scores and their associated figures. The outcome is calculated by comparing the local mean to the national mean. See part 9 for a full explanation. Question items show the individual questions that form each indicator result. To see the question items for a particular indicator, click on ‘local’ mean scores.



Part 2 | Outlier analysis Master class | creating a sorted report

In this example, I’ve decided to create a sorted report to see how my deanery/LETB compares to others for overall satisfaction in the programme type of cardiology. You can do this for any report. For example, you could compare survey results for handover in the different post specialties at St George’s Hospital, or clinical supervision for general surgery across hospitals in Wales. In this example, I’ve used the programme type by deanery/LETB report from the outlier analysis menu.



Part 2 | Outlier analysis Master class | creating a sorted report (cont)

Following step 5, you will see a chart showing overall satisfaction in cardiology sorted by deanery/LETB. All the columns in the all data view are sortable. So you could re-sort this report to see which deanery/LETB has the most cardiology trainees (for example). You can also use the Outcome filter to remove n<3 results, or only show outliers.



Part 3 | Indicator analysis Menu page

Use indicator analysis to compare training in different locations or different specialties using bar charts for each indicator. You can also navigate to question items from here.



Part 3 | Indicator analysis Navigating the charts

Whichever report you choose, the default chart will show all the available categories in the data series across the horizontal axis. We have also implemented a ‘slider’ function on large indicator charts so the values along the X axis are always readable. To reveal a chart for every indicator, click on the View all indicators link.



Part 3 | Indicator analysis Post specialty and programme group reports | controlling the horizontal axis

The post specialty report and programme group report are different to the other indicator reports because you can control the information that is being displayed in each bar of the graphs. For example, you can show the scores for each specialty at your site. The filter that controls the horizontal axis will always be to the left. This controls how many categories or bars are displayed on your chart.



Part 3 | Indicator analysis Post specialty and programme group reports | controlling chart bars

The filters to the right allow you to control the chart bars. This means you can adjust which scores contribute to your chart. Therefore, you can create a report for your trust/board or site which shows all your post specialties or programme groups along the horizontal axis.



Part 3 | Indicator analysis Post specialty and programme group reports | additional reports

You can also navigate to additional reports by clicking on the bar for any category. For example, if you click on the bar for cardiology and choose Trusts for this post, in the new chart, all the trusts providing cardiology training will appear along the horizontal axis.



Part 3 | Indicator analysis Post specialty and programme group reports | additional reports (cont)

The new chart will show the scores only for whichever specialty category or bar you clicked on in the previous chart. There will now be a category or bar along the horizontal axis for each of the locations (trusts/boards or sites) where training in that specialty is provided.



Part 3 | Indicator analysis Other reports | controlling the x-axis

All other reports such as the programme type report have only a horizontal axis filter. You can use the filter to reduce the number of categories or bars displayed along the horizontal axis and therefore compare a smaller number of categories.



Part 3 | Indicator analysis Other reports | additional reports

You can also navigate to additional reports by clicking on the bar for any category. For example, in the programme type report, if you click on the bar for anaesthetics and choose Programme by deanery/LETB, in the new chart, all the deaneries will appear along the horizontal axis.



Part 3 | Indicator analysis Other reports | additional reports (cont)

The new chart will show the scores only for whichever category or bar you clicked on in the previous chart. There will now be a bar along the horizontal axis for each of the sub-categories within the category you chose. In this example the new chart shows the scores for overall satisfaction for anaesthetics in each deanery/LETB.



Part 3 | Indicator analysis Question item reports

For any indicator result, you can view the question responses that produced the mean score. To do this, click on the category or bar you want to see the questions for and choose Question items.



Part 3 | Indicator analysis Question item reports (cont)

In the question item reports, there are charts showing the responses to the questions that make up the indicator. Scroll down to see all the charts. Only the questions for the indicator you have chosen will be displayed in the report. To find the questions for another indicator go back to the previous report and choose a different indicator.



Part 3 | Indicator analysis Question item reports (cont)

The chart shows what percentage of the respondents in your report chose each available response option to the question (displayed above the chart).



Part 3 | Indicator analysis Master class | replicating your post specialty by trust outlier report

You may want to look at your post specialty by trust/board outlier report in indicator analysis. The reports are labelled slightly different in each area. Follow these steps to create an indicator chart by the same parameters as your outlier chart.



Part 3 | Indicator analysis Master class | replicating your post specialty by trust/board report

Next, I navigate to the Indicator analysis area and choose the post specialty report.



Part 3 | Indicator analysis Master class | replicating your post specialty by trust/board report

Then I choose my trust in the trust/board filter. When my chart loads I can see the name of my trust in the applied filters box, and all the post specialties within my trust named along the horizontal axis. If a specialty is missing it means that there were not enough respondents in that specialty to create a report. We only report where n is equal to or greater than 3. From here you can click on the bar to see the question items (see page 32 for more instructions).



Part 4 | Trend analysis Menu page

Trend analysis provides year-on-year trend reports showing the outlier results (also known as red and green flags) for your location or specialty over three years. Using these reports you will be able to see how results have changed across time.



Part 4 | Trend analysis Building charts

When you navigate to the report you want, there will be no chart displayed. You need to apply filters to build your report.



Part 4 | Trend analysis Navigating the charts

When you’ve applied your filters, your trend analysis chart will load. The chart shows outcomes for three years side by side. The legend explains what each colour means. From any of the by trust/board reports, you can navigate to by site reports by clicking the link on the right hand side.



Part 4 | Trend analysis Trend analysis charts explained

In trend analysis charts the results are displayed in two ways. The outcome column shows the outcomes for each year side-by-side as coloured boxes, as you would see a single year in outlier analysis. The mean column simply shows the mean score for each year. If the mean score seems to contradict the outcome (for example if the mean goes up, but the outcome seems worse), this is because the benchmark has changed.



Part 4 | Trend analysis Master class | creating a trend report for multiple specialties and a single indicator

The previous instructions show how to see a trend report for all indicators for all specialties in your department. However, you may want to compare a single indicator for all specialties in your department. In this example I’m going to compare clinical supervision for all specialties in my trust by using the post specialty by trust/board report.



Part 4 | Trend analysis Master class | creating a trend report for multiple specialties and a single indicator (cont)

I can now see how well each of the specialties in my department are doing for clinical supervision over a three year period.



Part 5 | Programme specific reports Menu page

Programme specific reports show the results from programme specific questions set by Royal Colleges, faculties and the UK Foundation Programme Office. Colleges and faculties might set these questions to help them evaluate curricula and assessment systems. There is a separate report for each programme and you can use the menu page to choose the report for the programme that you’re interested in. Not all Colleges and Faculties choose to ask programme specific questions for all their programmes.



Part 5 | Programme specific reports Navigating the charts

When you’ve selected the programme that you want to see, a default report will load. This will be the question item reports for all the respondents in the latest survey year. You can change the report year using the drop down menu on the left. Because programme specific questions are submitted by Royal Colleges, Faculties and the UKFPO every year, the questions might not be the same year-on year. Beneath the question text, the n range (or n number for users of the Royal College reporting tool) tells you how many respondents answered each question.



Part 5 | Programme specific reports Navigating the charts (cont)

You can also view the programme specific reports by LETB/deanery or by training level. Above the default reports, you will see the option to see the results by LETB/deanery or by training level. Clicking on one of these options will create a new set of charts showing the results for each programme broken down by LETB/deanery or training levels.



Part 6 | Aggregation Menu page Aggregation allows you to add two or three years of indicator reports together. Before using this area of the tool please make sure you’re familiar with indicator reports (part 3 of this guide). Aggregation is particularly beneficial for small specialties or locations with lower numbers of trainees. Throughout the reporting tool no results are displayed where n (number of respondents) is less than 3. Aggregation means you can combine to increase n to 3 or more, which means you’ll be able to view a report in locations or specialties that have n of less than 3 in any given year.



Part 6 | Aggregation Building charts

When you navigate to the report you want, there will be no chart displayed. You need to apply filters to build your report. The guidance in part 1 explains how to use filters. In aggregation the year filter will combine reports from the years you select.



Part 6 | Aggregation Building charts (cont)

When your report loads, by default, only the chart for overall satisfaction will be displayed (except for users with a login account, who will automatically see all indicator charts). There will be a bar for each post or programme you’ve chosen in the filters (depending on the report you chose). Each bar shows a combined result for that post or programme for the years you’ve chosen to combine.



Part 6 | Aggregation Additional reports

In the post specialty report you can compare results for any specialty at different locations (trusts or sites). Click on the bar for the specialty you want to compare and choose trusts or sites from the pop-up menu.



Part 6 | Aggregation Additional reports (cont)

In your new report there will be a bar for every location* where there are more than 2 responses (n) in the specialty you’ve chosen, after the years have been combined. We’ve implemented a “slider” function on large indicator charts so the values along the X axis are always readable.

*This will be the trust or site depending which you chose from the pop-up menu in the previous report.



Part 6 | Aggregation Question items

To see the results of the questions that contribute to a particular score, click on the bar for that score. Choose question items from the pop-up, and a question item report will appear.



Part 6 | Aggregation Question items (cont)

In the aggregation question item reports, there are charts showing the results of the questions that make up the indicator, including the scores for each year you have combined. Scroll down to see all the charts. Only the questions for the indicator you have clicked on will be displayed in the report.



Part 7 | Trainer reports Menu page

Use the trainer results tab to find the outcomes (including outliers, known as red and green flags) for each indicator in a colour-coded grid for your chosen report.



Part 7 | Trainer reports Building charts

When you navigate to the report you want, you need to apply filters to build your report. In most cases you should start with the filter on the left. In some reports there may be no chart displayed by default. If this happens just apply the filters as normal and click ‘apply’. The guidance in part 1 explains how to use filters.



Part 7 | Trainer reports Viewing your charts

When you’ve applied your filters, the chart will load. The legend explains what each colour means. From the main reports you can navigate to additional reports by clicking the links to the right of the filters.



Part 7 | Trainer reports Navigating to the all results view

For a more detailed report, from your outlier chart, click on all results. In the all results view you can filter on outcomes and indicators and rank your results.



Part 7 | Trainer reports All results view functions

The all results view expands your outlier chart so that every coloured cell is now a row of numbers. If you applied filters in the previous view they will be carried through to this chart. In the all results view you can also filter by outcome. You can sort each column by hovering over the column header to prompt an up/down row indicator.



Part 7 | Trainer reports All results view explained

Each outcome is calculated using the figure shown in the data table. The light blue columns in the middle show the ‘local’ scores and their associated figures. The dark blue columns to the right show the ‘national’ scores and their corresponding figures. The outcome is calculated by comparing the local mean to the national mean. See part 9 for a full explanation. To see the questions that form each indicator result click on the ‘local’ mean score. For more detail see question item reports in part 3.



Part 8 | Legends Outlier and trend analysis legend In outlier analysis and trend analysis, the charts are presented in a grid format with coloured cells representing each outcome. The table to the left explains what these outcomes are. For an explanation of how outliers are calculated, please see part 9. You can find a full explanation of all terms in the glossary in Annex J.



Part 8 | Legends Indicator analysis legend

In indicator analysis (see part 3) each bar in your chart represents the score for the report group (labelled along the horizontal axis) set against the benchmark group. The diagram to the right explains what each section of the bar represents. For an explanation of how outliers are calculated, please see part 9. You can find a full explanation of all terms in the glossary in Annex J.



Part 9 | Calculations How answers to questions become a single score

An indicator is a score or result (called an outcome) made by grouping together the responses to questions or similar themes. Your report group (for example, your specialty in your trust) has a result that is based on the average scores of this group compared to a benchmark. The process starts with your answers to the questions for the indicator. Your scores for each question are averaged to give an overall indicator score.



Part 9 | Calculations Benchmark groups

Now your score contributes to two different calculations for each outcome. In this example, you are a general surgery trainee and we are looking at the post specialty by trust/board report. Firstly we calculate the benchmark group. For a general surgery trainee in this report, the benchmark group is all surgery trainees. You’ll see the benchmark scores in the reporting tool displayed as ‘national’ scores.



Part 9 | Calculations Quartiles Next, we take all the scores within the benchmark group and sort them into ascending order. Then we split it into quarters to find the quartiles. Now we transpose quartiles onto a 0-100 scale (similar to the vertical bars you see in indicator analysis). If the lowest score given (national min) is more than zero, you’ll see a gap between zero and the lowest score. The same applies to national max at the other end of the scale.



Part 9 | Calculations Benchmark mean and confidence intervals

Next, we calculate the average for the benchmark group (national mean). This is the sum of the scores divided by the number of contributing scores (N). Then, we calculate the confidence intervals* for the national mean. This gives us the comparator. The outcome depends on where the report group score falls on this scale.

*Confidence intervals show how statistically confident we can be that our mean score represents a true average. The smaller the confidence intervals, the more confident we can be. You might see confidence intervals represented as a line through the mean (black square) like in the picture above. In indicator analysis, we show the upper and lower confidence limits with a black cross (+).



Part 9 | Calculations Report group mean Now we calculate the report group mean. In our example, you are a General Surgery trainee at AN Other NHS Foundation Trust. The report group includes the scores for all the General Surgery trainees in your trust. The report group mean (mean) is an average of all the indicator scores for the individual respondents comprising the report group. Next we compare the report group score to the benchmark we calculated earlier. The red dot is your score. It’s above the national mean, which is good, but it’s within the interquartile range, so it’s not exceptional.



Part 9 | Calculations Outlier summary grid Now we have a report group score and a benchmark, and we know that our outcome is ‘within IQR’, we can show it as a white coloured cell on an outlier summary grid like this. Each cell is calculated by the method described above. There are examples of all the different outcomes on the next page.



Part 9 | Calculations Example outcomes explained This is above outlier (dark green). The mean is in the top quartile and the confidence intervals do not overlap. This is light green. The mean is in the top quartile, but it’s not an above outlier because the confidence intervals overlap. Like our example outcome, this is white. The mean is within the interquartile range (IQR). This would be pink. It’s within the bottom quartile but not a below outlier because the confidence intervals overlap. This is a below outlier (red). The mean is in the bottom quartile and the confidence intervals do not overlap with those of the national mean.



Part 10 Printing and downloading

Below every chart you will find an export link. Click on the link and choose a format/program to open the data in. Use the print function to print the data. Choose PDF to convert the chart to an Adobe Acrobat PDF file. Use Acrobat’s print function to print the chart. Click on the Excel link to export to Excel. Graphs will download as images in Excel. Tabular data will download as tables in Excel with row and column headings.



Annex A – trainee benchmark groups Report Benchmark group Country All UK trainees

Deanery All UK trainees

Trust/Board All UK trainees

Site All UK trainees

Foundation School All Foundation Programme trainees

Training Level All UK trainees

Training Group All UK trainees

Post Specialty Trainees in related post specialty groups (see Annexes B1 and B2)

Programme Specialty All UK trainees

Programme Group Trainees within related programme groups (see Annex B2)

Programme Type Trainees within related programme types (see Annex B3)

GP Scheme Trainees within related GP scheme groups (see Annex B4)

Place of Qualification All Foundation Year 1 trainees



Annex B1 – post specialty report groups and their benchmark groups

Use this table to find a post specialty (A-Z) and which benchmark group it is compared to in the post specialty, post specialty by trust/board and post specialty by site reports. Use the table in Annex B2 to see which post specialties are in each benchmark group.

A-Z Post specialty Benchmark group A Acute Internal Medicine All Medicine Posts Acute Medicine All Medicine Posts Allergy All Medicine Posts Anaesthetics All Anaesthetics Posts Audio Vestibular Medicine All Medicine Posts C Cardiology All Medicine Posts Cardio-Thoracic Surgery All Surgery Posts Chemical Pathology All Pathology Posts Child and Adolescent Psychiatry All Psychiatry Posts Child Mental Health All Paediatric and Child Health Posts Clinical Genetics All Medicine Posts Clinical Neurophysiology All Medicine Posts Clinical Oncology All Radiology Posts Clinical Pharmacology and Therapeutics All Medicine Posts Clinical Radiology All Radiology Posts Community Child Health All Paediatric and Child Health Posts Community Sexual and Reproductive Health All Obstetrics and Gynaecology Posts Congenital Cardiac Surgery All Surgery Posts Cytopathology All Pathology Posts



A-Z Post specialty Benchmark group D Dermatology All Medicine Posts Diagnostic neuropathology All Pathology Posts E Emergency Medicine All Emergency Medicine Posts

Endocrinology and Diabetes Mellitus All Medicine Posts

F Forensic Histopathology All Pathology Posts Forensic Pathology All Pathology Posts Forensic Psychiatry All Psychiatry Posts G Gastroenterology All Medicine Posts General (internal) Medicine All Medicine Posts General Practice All General Practice Posts General Psychiatry All Psychiatry Posts General Surgery All Surgery Posts Genito-Urinary Medicine All Medicine Posts Geriatric Medicine All Medicine Posts Gynaecological Oncology All Obstetrics and Gynaecology Posts H Haematology All Medicine Posts Hepatology All Medicine Posts Histopathology All Pathology Posts I Immunology All Medicine Posts Infectious Disease All Medicine Posts Intensive Care Medicine All Anaesthetics Posts Intervention Radiology All Radiology Posts



A-Z Post specialty Benchmark group L Liaison Psychiatry All Psychiatry Posts M Maternal and Foetal Medicine All Obstetrics and Gynaecology Posts Medical Microbiology All Pathology Posts Medical Microbiology and Virology All Pathology Posts Medical Oncology All Medicine Posts Medical Ophthalmology All Medicine Posts Medical Psychotherapy All Psychiatry Posts Medical Virology All Pathology Posts Metabolic Medicine All Pathology Posts N Neonatal Medicine All Paediatric and Child Health Posts Neurology All Medicine Posts Neuropathology All Pathology Posts Neurosurgery All Surgery Posts Nuclear Medicine All Medicine Posts O Obstetrics and Gynaecology All Obstetrics and Gynaecology Posts Occupational Medicine All Occupational Medicine Posts Old Age Psychiatry All Psychiatry Posts Ophthalmology All Ophthalmology Posts Oral and Maxillo-Facial Surgery All Surgery Posts Otolaryngology All Surgery Posts P Paediatric and Perinatal Pathology All Pathology Posts Paediatric Cardiology All Medicine Posts



A-Z Post specialty Benchmark group Paediatric Clinical Pharmacology and Therapeutics All Paediatric and Child Health Posts Paediatric Diabetes and Endocrinology All Paediatric and Child Health Posts Paediatric Emergency Medicine All Paediatric and Child Health Posts Paediatric Gastroenterology, Hepatology and Nutrition All Paediatric and Child Health Posts Paediatric Immunology, Infectious Diseases and Allergy All Paediatric and Child Health Posts Paediatric Inherited Metabolic Medicine All Paediatric and Child Health Posts Paediatric Intensive Care Medicine All Paediatric and Child Health Posts Paediatric Nephrology All Paediatric and Child Health Posts Paediatric Neurodisability All Paediatric and Child Health Posts Paediatric Neurology All Paediatric and Child Health Posts Paediatric Oncology All Paediatric and Child Health Posts Paediatric Palliative Medicine All Paediatric and Child Health Posts Paediatric Pathology All Pathology Posts Paediatric Respiratory Medicine All Paediatric and Child Health Posts Paediatric Rheumatology All Paediatric and Child Health Posts Paediatric Surgery All Surgery Posts Paediatrics All Paediatric and Child Health Posts Palliative Medicine All Medicine Posts Pharmaceutical Medicine All Medicine Posts Plastic Surgery All Surgery Posts Pre-Hospital Emergency Medicine All Emergency Medicine Posts Psychiatry of Learning Disability All Psychiatry Posts



A-Z Post specialty Benchmark group Public Health Medicine All Public Health Posts R Rehabilitation Medicine All Medicine Posts Rehabilitation Psychiatry All Psychiatry Posts Renal Medicine All Medicine Posts Reproductive Medicine All Obstetrics and Gynaecology Posts Respiratory Medicine All Medicine Posts Rheumatology All Medicine Posts S Sexual and Reproductive Health All Obstetrics and Gynaecology Posts Sport and Exercise Medicine All Medicine Posts Stroke Medicine All Medicine Posts Substance Misuse Psychiatry All Psychiatry Posts T Trauma and Orthopaedic Surgery All Surgery Posts Tropical Medicine All Medicine Posts U Urogynaecology All Obstetrics and Gynaecology Posts Urology All Surgery Posts V Vascular Surgery All Surgery Posts



Annex B2 - post specialty benchmark groups Use this table to see which post specialties are in each benchmark group.

Use the table in Annex B1 to find a post specialty (A-Z) and which benchmark group it is compared to in the post specialty, post specialty by trust/board and post specialty by site reports.

Benchmark group Post specialties included in benchmark group All Anaesthetics Posts Anaesthetics

All Anaesthetics Posts Intensive Care Medicine

All Emergency Medicine Posts Emergency Medicine

All Emergency Medicine Posts Pre-Hospital Emergency Medicine

All General Practice Posts General Practice

All Medicine Posts Acute Internal Medicine, Acute Medicine, Allergy, Audio Vestibular Medicine, Cardiology, Clinical Genetics, Clinical Neurophysiology, Clinical Pharmacology and Therapeutics, Dermatology, Endocrinology and Diabetes Mellitus, Gastroenterology, General (internal) Medicine, Genito-Urinary Medicine, Geriatric Medicine, Haematology, Hepatology, Immunology, Infectious Disease, Medical Oncology, Medical Ophthalmology, Neurology, Nuclear Medicine, Paediatric Cardiology, Palliative Medicine, Pharmaceutical Medicine, Rehabilitation Medicine, Renal Medicine, Respiratory Medicine, Rheumatology, Sport and Exercise Medicine, Stroke Medicine, Tropical Medicine

All Obstetrics and Gynaecology Posts

Community Sexual and Reproductive Health, Gynaecological Oncology, Maternal and Foetal Medicine, Obstetrics and Gynaecology, Reproductive Medicine, Sexual and Reproductive Health, Urogynaecology

All Occupational Medicine Posts Occupational Medicine

All Ophthalmology Posts Ophthalmology

All Paediatric and Child Health Posts

Child Mental Health, Community Child Health, Neonatal Medicine, Paediatric Clinical Pharmacology and Therapeutics, Paediatric Diabetes and Endocrinology, Paediatric Emergency Medicine, Paediatric Gastroenterology, Hepatology and Nutrition, Paediatric Immunology, Infectious Diseases and Allergy, Paediatric Inherited Metabolic Medicine,



Benchmark group Post specialties included in benchmark group Paediatric Intensive Care Medicine, Paediatric Nephrology, Paediatric Neurodisability, Paediatric Neurology,

Paediatric Oncology, Paediatric Palliative Medicine, Paediatric Respiratory Medicine, Paediatric Rheumatology, Paediatrics

All Pathology Posts Chemical Pathology, Cytopathology, Diagnostic neuropathology, Forensic Histopathology, Forensic Pathology, Histopathology, Medical Microbiology, Medical Microbiology and Virology, Medical Virology, Metabolic Medicine, Neuropathology, Paediatric and perinatal pathology, Paediatric Pathology

All Psychiatry Posts Child and Adolescent Psychiatry, Forensic Psychiatry, General Psychiatry, Liaison Psychiatry, Medical Psychotherapy, Old Age Psychiatry, Psychiatry of Learning Disability, Rehabilitation Psychiatry, Substance Misuse Psychiatry

All Public Health Posts Public Health Medicine

All Radiology Posts Clinical Oncology, Clinical Radiology, Intervention Radiology

All Surgery Posts Cardio-Thoracic Surgery, Congenital Cardiac Surgery, General Surgery, Neurosurgery, Oral and Maxillo-Facial Surgery, Otolaryngology, Paediatric Surgery, Plastic Surgery, Trauma and Orthopaedic Surgery, Urology, Vascular Surgery



Annex C1 - programme group reporting and benchmark groups Use this table find a programme group (A-Z) and which benchmark group it is compared to in the programme group, programme group by trust/board and programme group by site reports.

Use the table in Annex C2 to see which programme groups are in each benchmark group.

A-Z Programme group Benchmark group A ACCS All ACCS Trainees Acute Internal Medicine All Medicine Programmes (ST3+) Allergy All Medicine Programmes (ST3+) Anaesthetics All Anaesthetists Programmes (ST3+) Anaesthetics F1 All F1 Trainees Anaesthetics F2 All F2 Trainees Audio Vestibular Medicine All Medicine Programmes (ST3+) B Broad Based Training All Broad Based Training Trainees C Cardiology All Medicine Programmes (ST3+) Cardio-thoracic Surgery All Surgery Programmes Chemical Pathology All Pathology Programmes Child and Adolescent Psychiatry All Psychiatry Programmes (ST4+) Clinical Genetics All Medicine Programmes (ST3+) Clinical Neurophysiology All Medicine Programmes (ST3+) Clinical Oncology All Radiology Programmes Clinical Pharmacology and Therapeutics All Medicine Programmes (ST3+) Clinical Radiology All Radiology Programmes CMT All CMT Trainees



A-Z Programme group Benchmark group Community Sexual and Reproductive Health All Obstetrics and Gynaecology Programmes Core Anaesthetics All CAT Trainees CPT All CPT Trainees CST All CST Trainees D Dermatology All Medicine Programmes (ST3+) Diagnostic neuropathology All Pathology Programmes E Emergency Medicine All Emergency Medicine Programmes Emergency Medicine F1 All F1 Trainees Emergency Medicine F2 All F2 Trainees Endocrinology and Diabetes Mellitus All Medicine Programmes (ST3+) F Forensic Psychiatry All Psychiatry Programmes (ST4+) Forensic histopathology All Pathology Programmes G Gastroenterology All Medicine Programmes (ST3+) General (Internal) Medicine All Medicine Programmes (ST3+) General Practice F1 All F1 Trainees General Practice F2 All F2 Trainees General Psychiatry All Psychiatry Programmes (ST4+) General Surgery All Surgery Programmes Genito-Urinary Medicine All Medicine Programmes (ST3+) Geriatric Medicine All Medicine Programmes (ST3+) GP Prog - Anaesthetics All GPs in a Secondary Care Post (excluding F2) GP Prog - Emergency Medicine All GPs in a Secondary Care Post (excluding F2)



A-Z Programme group Benchmark group GP Prog - Medicine All GPs in a Secondary Care Post (excluding F2) GP Prog - Obstetrics and Gynaecology All GPs in a Secondary Care Post (excluding F2) GP Prog - Occupational Medicine All GPs in a Secondary Care Post (excluding F2) GP Prog - Ophthalmology All GPs in a Secondary Care Post (excluding F2) GP Prog - Paediatrics and Child Health All GPs in a Secondary Care Post (excluding F2) GP Prog - Pathology All GPs in a Secondary Care Post (excluding F2) GP Prog - Psychiatry All GPs in a Secondary Care Post (excluding F2) GP Prog - Public Health All GPs in a Secondary Care Post (excluding F2) GP Prog - Radiology All GPs in a Secondary Care Post (excluding F2) GP Prog - Surgery All GPs in a Secondary Care Post (excluding F2) H Haematology All Medicine Programmes (ST3+) Histopathology All Pathology Programmes I Immunology All Medicine Programmes (ST3+) Infectious Diseases All Medicine Programmes (ST3+) Intensive Care Medicine All Anaesthetists Programmes (ST3+) M Medical Microbiology All Pathology Programmes Medical Microbiology and Virology All Pathology Programmes Medical Oncology All Medicine Programmes (ST3+) Medical Ophthalmology All Medicine Programmes (ST3+) Medical Psychotherapy All Psychiatry Programmes (ST4+) Medical Virology All Pathology Programmes Medicine F1 All F1 Trainees



A-Z Programme group Benchmark group Medicine F2 All F2 Trainees N Neurology All Medicine Programmes (ST3+) Neurosurgery All Surgery Programmes Nuclear Medicine All Medicine Programmes (ST3+) O Obstetrics and Gynaecology All Obstetrics and Gynaecology Programmes Obstetrics and Gynaecology F1 All F1 Trainees Obstetrics and Gynaecology F2 All F2 Trainees Occupational Medicine All Occupational Medicine Programmes Occupational Medicine F1 All F1 Trainees Occupational Medicine F2 All F2 Trainees Old Age Psychiatry All Psychiatry Programmes (ST4+) Ophthalmology All Ophthalmology Programmes Ophthalmology F1 All F1 Trainees Ophthalmology F2 All F2 Trainees Oral and Maxillo-Facial Surgery All Surgery Programmes Otolaryngology All Surgery Programmes P Paediatric and Perinatal Pathology All Pathology Programmes Paediatric Cardiology All Medicine Programmes (ST3+) Paediatric Surgery All Surgery Programmes Paediatrics All Paediatrics and Child Health Programmes Paediatrics and Child Health F1 All F1 Trainees Paediatrics and Child Health F2 All F2 Trainees



A-Z Programme group Benchmark group Palliative Medicine All Medicine Programmes (ST3+) Pathology F1 All F1 Trainees Pathology F2 All F2 Trainees Pharmaceutical Medicine All Medicine Programmes (ST3+) Plastic Surgery All Surgery Programmes Psychiatry F1 All F1 Trainees Psychiatry F2 All F2 Trainees Psychiatry of Learning Disability All Psychiatry Programmes (ST4+) Public Health F1 All F1 Trainees Public Health F2 All F2 Trainees Public Health Medicine All Public Health Programmes R Radiology F1 All F1 Trainees Radiology F2 All F2 Trainees Rehabilitation Medicine All Medicine Programmes (ST3+) Renal Medicine All Medicine Programmes (ST3+) Respiratory Medicine All Medicine Programmes (ST3+) Rheumatology All Medicine Programmes (ST3+) S Sport and Exercise Medicine All Medicine Programmes (ST3+) Surgery F1 All F1 Trainees Surgery F2 All F2 Trainees T Trauma and Orthopaedic Surgery All Surgery Programmes Tropical Medicine All Medicine Programmes (ST3+)



A-Z Programme group Benchmark group U Urology All Surgery Programmes V Vascular Surgery All Surgery Programmes



Annex C2 - programme group benchmark groups

Use this table to see which programme groups are in each benchmark group.

Use the table in Annex C1 find a programme group (A-Z) and which benchmark group it is compared to in the programme group, programme group by trust/board and programme group by site reports.

Benchmark group Programme groups included in the benchmark group All ACCS Trainees ACCS

All Anaesthetists Programmes (ST3+) Anaesthetics

All Anaesthetists Programmes (ST3+) Intensive Care Medicine

All Broad Based Training Trainees Broad Based Training

All CAT Trainees Core Anaesthetics

All CMT Trainees CMT

All CPT Trainees CPT

All CST Trainees CST

All Emergency Medicine Programmes Emergency Medicine

All F1 Trainees Anaesthetics F1, Emergency Medicine F1, General Practice F1, Medicine F1, Obstetrics and Gynaecology F1, Occupational Medicine F1, Ophthalmology F1, Paediatrics and Child Health F1, Pathology F1, Psychiatry F1, Public Health F1, Radiology F1, Surgery F1

All F2 Trainees Anaesthetics F2, Emergency Medicine F2, General Practice F2, Medicine F2, Obstetrics and Gynaecology F2, Occupational Medicine F2, Ophthalmology F2, Paediatrics and Child Health F2, Pathology F2, Psychiatry F2, Public Health F2, Radiology F2, Surgery F2

All GPs in a Secondary Care Post (excluding F2)

GP Prog - Anaesthetics, GP Prog - Emergency Medicine, GP Prog - Medicine, GP Prog - Obstetrics and Gynaecology, GP Prog - Occupational Medicine, GP Prog - Ophthalmology, GP Prog - Paediatrics and Child Health, GP Prog - Pathology, GP Prog - Psychiatry, GP Prog - Public Health, GP Prog - Radiology, GP Prog - Surgery



Benchmark group Programme groups included in the benchmark group All Medicine Programmes (ST3+) Acute Internal Medicine, Allergy, Audio Vestibular Medicine, Cardiology, Clinical Genetics, Clinical

Neurophysiology, Clinical Pharmacology and Therapeutics, Dermatology, Endocrinology and Diabetes Mellitus, Gastroenterology, General (Internal) Medicine, Genito-Urinary Medicine, Geriatric Medicine, Haematology, Immunology, Infectious Diseases, Medical Oncology, Medical Ophthalmology, Neurology, Nuclear Medicine, Paediatric Cardiology, Palliative Medicine, Pharmaceutical Medicine, Rehabilitation Medicine, Renal Medicine, Respiratory Medicine, Rheumatology, Sport and Exercise Medicine, Tropical Medicine

All Obstetrics and Gynaecology Programmes

Community Sexual and Reproductive Health

All Obstetrics and Gynaecology Programmes

Obstetrics and Gynaecology

All Occupational Medicine Programmes

Occupational Medicine

All Ophthalmology Programmes Ophthalmology

All Paediatrics and Child Health Programmes

Paediatrics

All Pathology Programmes Chemical Pathology, Diagnostic neuropathology, Forensic Histopathology, Histopathology, Medical Microbiology, Medical Microbiology and Virology, Medical Virology, Paediatric histopathology

All Psychiatry Programmes (ST4+) Child and Adolescent Psychiatry, Forensic Psychiatry, General Psychiatry, Medical Psychotherapy, Old Age Psychiatry, Psychiatry of Learning Disability

All Public Health Programmes Public Health Medicine

All Radiology Programmes Clinical Oncology

All Radiology Programmes Clinical Radiology

All Surgery Programmes Cardio-thoracic Surgery, General Surgery, Neurosurgery, Oral and Maxillo-Facial Surgery, Otolaryngology, Paediatric Surgery, Plastic Surgery, Trauma and Orthopaedic Surgery, Urology, Vascular Surgery



Annex D1 - programme type reporting and benchmark groups Use this table to find a programme group (A-Z) and which benchmark group it is compared to in the programme type and the programme type by deanery reports.

Use the table in Annex D2 to see which programme types are in each benchmark group.

A-Z Programme type Benchmark group A Acute Care Common Stem All Core Programmes Acute Internal Medicine All higher (ST3+) Specialty Programmes Allergy All higher (ST3+) Specialty Programmes Anaesthetics All higher (ST3+) Specialty Programmes Audio Vestibular Medicine All higher (ST3+) Specialty Programmes B Broad Based Training All Core Programmes C Cardiology All higher (ST3+) Specialty Programmes Cardio-Thoracic Surgery All higher (ST3+) Specialty Programmes Chemical Pathology All higher (ST3+) Specialty Programmes Child and Adolescent Psychiatry All higher (ST3+) Specialty Programmes Clinical Genetics All higher (ST3+) Specialty Programmes Clinical Neurophysiology All higher (ST3+) Specialty Programmes Clinical Oncology All higher (ST3+) Specialty Programmes Clinical Pharmacology and Therapeutics All higher (ST3+) Specialty Programmes Clinical Radiology All higher (ST3+) Specialty Programmes Community Sexual and Reproductive Health All higher (ST3+) Specialty Programmes Core Anaesthetics Training All Core Programmes Core Medical Training All Core Programmes



A-Z Programme type Benchmark group Core Psychiatry Training All Core Programmes Core Surgical Training All Core Programmes D Dermatology All higher (ST3+) Specialty Programmes Diagnostic Neuropathology All higher (ST3+) Specialty Programmes E Emergency Medicine All higher (ST3+) Specialty Programmes Endocrinology and Diabetes Mellitus All higher (ST3+) Specialty Programmes F F1 All F1 Trainees F2 All F2 Trainees Forensic Histopathology All higher (ST3+) Specialty Programmes Forensic Psychiatry All higher (ST3+) Specialty Programmes G Gastroenterology All higher (ST3+) Specialty Programmes General (internal) Medicine All higher (ST3+) Specialty Programmes General Psychiatry All higher (ST3+) Specialty Programmes General Surgery All higher (ST3+) Specialty Programmes Genito-Urinary Medicine All higher (ST3+) Specialty Programmes Geriatric Medicine All higher (ST3+) Specialty Programmes GP in Secondary Care All GPs in a Secondary Care Post GP Prog - in a GP Practice All GPs in a General Practice Post H Haematology All higher (ST3+) Specialty Programmes Histopathology All higher (ST3+) Specialty Programmes I Immunology All higher (ST3+) Specialty Programmes Infectious Diseases All higher (ST3+) Specialty Programmes



A-Z Programme type Benchmark group Intensive Care Medicine All higher (ST3+) Specialty Programmes M Medical Microbiology All higher (ST3+) Specialty Programmes Medical Microbiology and Virology All higher (ST3+) Specialty Programmes Medical Oncology All higher (ST3+) Specialty Programmes Medical Ophthalmology All higher (ST3+) Specialty Programmes Medical Psychotherapy All higher (ST3+) Specialty Programmes Medical Virology All higher (ST3+) Specialty Programmes N Neurology All higher (ST3+) Specialty Programmes Neurosurgery All higher (ST3+) Specialty Programmes Nuclear Medicine All higher (ST3+) Specialty Programmes O Obstetrics and Gynaecology All higher (ST3+) Specialty Programmes Occupational Medicine All higher (ST3+) Specialty Programmes Old Age Psychiatry All higher (ST3+) Specialty Programmes Ophthalmology All higher (ST3+) Specialty Programmes Oral and Maxillo-Facial Surgery All higher (ST3+) Specialty Programmes Otolaryngology All higher (ST3+) Specialty Programmes P Paediatric and Perinatal Pathology All higher (ST3+) Specialty Programmes Paediatric Cardiology All higher (ST3+) Specialty Programmes Paediatric Surgery All higher (ST3+) Specialty Programmes Paediatrics All higher (ST3+) Specialty Programmes Palliative Medicine All higher (ST3+) Specialty Programmes Pharmaceutical Medicine All higher (ST3+) Specialty Programmes



A-Z Programme type Benchmark group Plastic Surgery All higher (ST3+) Specialty Programmes Psychiatry of Learning Disability All higher (ST3+) Specialty Programmes Public Health Medicine All higher (ST3+) Specialty Programmes

R Rehabilitation Medicine All higher (ST3+) Specialty Programmes

Renal Medicine All higher (ST3+) Specialty Programmes

Respiratory Medicine All higher (ST3+) Specialty Programmes

Rheumatology All higher (ST3+) Specialty Programmes S Sport and Exercise Medicine All higher (ST3+) Specialty Programmes T Trauma and Orthopaedic Surgery All higher (ST3+) Specialty Programmes Tropical Medicine All higher (ST3+) Specialty Programmes U Urology All higher (ST3+) Specialty Programmes V Vascular Surgery All higher (ST3+) Specialty Programmes



Annex D2 - programme type benchmark groups

Use this table to see which programme types are in each benchmark group.

Use the table in Annex D1 to find a programme group (A-Z) and which benchmark group it is compared to in the programme type and the programme type by deanery reports.

Benchmark Programme types included in benchmark group All Core Programmes Acute Care Common Stem, Core Anaesthetics Training, Broad Based Training, Core Medical Training, Core

Psychiatry Training, Core Surgical Training

All F1 Trainees F1

All F2 Trainees F2

All GPs in a Secondary Care Post GP in Secondary Care

All GPs in a General Practice Post GP Prog - in a GP Practice

All higher (ST3+) Specialty Programmes

Acute Internal Medicine, Allergy, Anaesthetics, Audio Vestibular Medicine, Cardiology, Cardio-Thoracic Surgery, Chemical Pathology, Child and Adolescent Psychiatry, Clinical Genetics, Clinical Neurophysiology, Clinical Oncology, Clinical Pharmacology and Therapeutics, Clinical Radiology, Community Sexual and Reproductive Health, Dermatology, Diagnostic Neuropathology, Emergency Medicine, Endocrinology and Diabetes Mellitus, Forensic Histopathology, Forensic Psychiatry, Gastroenterology, General (internal) Medicine, General Psychiatry, General Surgery, Genito-Urinary Medicine, Geriatric Medicine, Haematology, Histopathology, Immunology, Infectious Diseases, Intensive Care Medicine, Medical Microbiology, Medical Microbiology and Virology, Medical Oncology, Medical Ophthalmology, Medical Psychotherapy, Medical Virology, Neurology, Neurosurgery, Nuclear Medicine, Obstetrics and Gynaecology, Occupational Medicine, Old Age Psychiatry, Ophthalmology, Oral and Maxillo-Facial Surgery, Otolaryngology, Paediatric and Perinatal Pathology, Paediatric Cardiology, Paediatric Surgery, Paediatrics, Palliative Medicine, Pharmaceutical Medicine, Plastic Surgery, Psychiatry of Learning Disability, Public Health Medicine, Rehabilitation Medicine, Renal Medicine, Respiratory Medicine, Rheumatology, Sport and Exercise Medicine, Trauma and Orthopaedic Surgery, Tropical Medicine, Urology, Vascular Surgery



Annex E1 - GP scheme reporting and benchmark groups Use this table to find a GP scheme (A-Z) and which benchmark group it is compared to in the GP scheme and GP scheme by GP group reports.

Use the table in Annex E2 to see which GP schemes are in each benchmark group.

GP scheme Benchmark group F2 in General Practice All F2s in a General Practice post

GP Prog - Anaesthetics All GPs in a secondary care post

GP Prog - Emergency Medicine All GPs in a secondary care post

GP Prog - General Practice All GPs in a General Practice post

GP Prog - Medicine All GPs in a secondary care post

GP Prog - Obstetrics and Gynaecology All GPs in a secondary care post

GP Prog - Occupational Medicine All GPs in a secondary care post

GP Prog - Ophthalmology All GPs in a secondary care post

GP Prog - Paediatrics and Child Health All GPs in a secondary care post

GP Prog - Pathology All GPs in a secondary care post

GP Prog - Psychiatry All GPs in a secondary care post

GP Prog - Public Health All GPs in a secondary care post

GP Prog - Radiology All GPs in a secondary care post

GP Prog - Surgery All GPs in a secondary care post



Annex E2 - GP scheme benchmark groups Use this table to see which GP schemes are in each benchmark group.

Use the table in Annex E1 to find a GP scheme (A-Z) and which benchmark group it is compared to in the GP scheme and GP scheme by GP group reports.

Benchmark GP schemes included in benchmark group All F2s in a General Practice post F2 in General Practice

All GPs in a General Practice post GP Prog - General Practice

All GPs in a secondary care post GP Prog - Anaesthetics, GP Prog - Emergency Medicine, GP Prog - Medicine, GP Prog - Obstetrics and Gynaecology, GP Prog - Occupational Medicine, GP Prog - Ophthalmology, GP Prog - Paediatrics and Child Health, GP Prog - Pathology, GP Prog - Psychiatry, GP Prog - Public Health, GP Prog - Radiology, GP Prog - Surgery



Annex F – Trainee indicator questions Trainee indicator Questions Overall satisfaction These questions make up the 'Overall satisfaction' indicator.

• How would you rate the quality of teaching (informal and bedside teaching as well as formal and organised sessions) in

this post? • How would you rate the quality of clinical supervision in this post? • How would you rate the quality of experience in this post? • How would you describe this post to a friend who was thinking of applying for it? • How useful do you feel this post will be for your future career?

Why is this score useful? This indicator combines general questions about the quality and usefulness of the training post and provides a global satisfaction score. It does not relate to any particular requirement within ‘Promoting excellence: standards for medical education and training’ (2015).

Clinical supervision These questions make up the 'Clinical supervision' indicator.

• How would you rate the quality of clinical supervision in this post? • In this post did you always know who was providing your clinical supervision when you were working? • In this post how often, if ever, were you supervised by someone who you felt wasn’t competent to do so? • In this post how often did you feel forced to cope with clinical problems beyond your competence or experience? • In this post how often have you been expected to obtain consent for procedures where you feel you do not understand

the proposed interventions and its risks? Please note there are different versions of some questions for Public Health Medicine trainees. Why is this score useful?



Trainee indicator Questions Good training requires practical experience under safe supervision. A low score indicates there may be a problem with clinical

supervision and that patients and trainees could be put at risk. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.7 - Organisations must make sure there are enough staff members who are suitably qualified, so that learners have appropriate clinical supervision, working patterns and workload, for patients to receive care that is safe and of a good standard, while creating the required learning opportunities.

• R1.8 - Organisations must make sure that learners have an appropriate level of clinical supervision at all times by an experienced and competent supervisor, who can advise or attend as needed. The level of supervision must fit the individual learner’s competence, confidence and experience. The support and clinical supervision must be clearly outlined to the learner and the supervisor. Foundation doctors must at all times have on-site access to a senior colleague who is suitably qualified to deal with problems that may arise during the session. Medical students on placement must be supervised, with closer supervision when they are at lower levels of competence.

• R1.10 - Organisations must have a reliable way of identifying learners at different stages of education and training, and make sure all staff members take account of this, so that learners are not expected to work beyond their competence.

• R1.11 - Doctors in training must take consent only for procedures appropriate for their level of competence. Learners must act in accordance with General Medical Council (GMC) guidance on consent. Supervisors must assure themselves that a learner understands any proposed intervention for which they will take consent, its risks and alternative treatment options.

Clinical supervision out of hours

These questions make up the 'Clinical supervision out of hours' indicator.

• How would you rate the quality of clinical supervision, OUT OF HOURS, in this post? • In this post, OUT OF HOURS, did you always know who was providing your clinical supervision when you were working? • In this post, OUT OF HOURS, how often, if ever, were you clinically supervised by someone who you felt wasn't

competent to do so • In this post, OUT OF HOURS, how often did you feel forced to cope with clinical problems beyond your competence or

experience?



Trainee indicator Questions • In this post, OUT OF HOURS, how often have you been expected to obtain consent for procedures where you feel you

do not understand the proposed interventions and its risks? Please note there are different versions of some questions for Public Health Medicine trainees. Why is this score useful? Good training requires practical experience under safe supervision. A low score indicates there may be a problem with clinical supervision out of hours and that patients and trainees could be put at risk. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.


• R1.8 - Organisations must make sure that learners have an appropriate level of clinical supervision at all times by an experienced and competent supervisor, who can advise or attend as needed. The level of supervision must fit the individual learner’s competence, confidence and experience. The support and clinical supervision must be clearly outlined to the learner and the supervisor.

• Foundation doctors must at all times have on-site access to a senior colleague who is suitably qualified to deal with problems that may arise during the session. Medical students on placement must be supervised, with closer supervision when they are at lower levels of competence.


• R1.11 - Doctors in training must take consent only for procedures appropriate for their level of competence. Learners must act in accordance with General Medical Council (GMC) guidance on consent. Supervisors must assure themselves that a learner understands any proposed intervention for which they will take consent, its risks and alternative treatment options.



Trainee indicator Questions Handover These questions make up the 'Handover' indicator.

• Please state whether you agree or disagree with the following statements about your post: • Handover arrangements in this post BETWEEN SHIFTS ensure continuity of care for patients. • Handover of patients BETWEEN DEPARTMENTS in this post ensures continuity of care. • Appropriate members of the multidisciplinary team are included in handover.

Why is this score useful? A good handover ensures that trainees understand what is required to manage the care for patients between shifts and departments during their shift and helps them identify situations they may need assistance with. Low scores indicate that arrangements for handover may provide less continuity of care for patients. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.14 - Handover* of care must be organised and scheduled to provide continuity of care for patients and maximise the learning opportunities for doctors in training in clinical practice.

* Handover at the start and end of periods of day or night duties, every day of the week.

Induction These questions make up the 'Induction' indicator.

• Did you get all the information you needed about your workplace when you started working in this post? • Did someone explain your role and responsibilities in your unit or department at the start of this post? • How would you rate the quality of induction in this post? (This refers to your induction to the organisation in which you

worked.) • Did you sit down with your educational supervisor and discuss your educational objectives for this post?

Why is this score useful?



Trainee indicator Questions A good induction ensures that trainees understand the systems in their local education provider and their department as well

as their personal objectives for the post and how these might be achieved. A good induction helps trainees work safely in their post and make the most of educational opportunities. A low score indicates that trainees did not consider the induction to be thorough. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R5.9c - Postgraduate training programmes must give doctors in training an educational induction to make sure they understand their curriculum and how their post or clinical placement fits within the programme.

• R1.13 - Organisations must make sure learners have an induction in preparation for each placement that clearly sets out:

o a - their duties and supervision arrangements o b - their role in the team

Adequate experience These questions make up the 'Adequate experience' indicator.

• How would you rate the practical experience you were receiving in this post? • How confident are you that this post will help you acquire the competencies you needed at that particular stage of your

training? Why is this score useful? Practical experience is vital to postgraduate training; trainees need appropriate exposure to clinical work to develop their abilities as a doctor. Low scores indicate posts that do not provide adequate experience for trainees for their stage of training. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.12c - Organisations must design rotas to provide learning opportunities that allow doctors in training to meet the requirements of their curriculum and training programme



Trainee indicator Questions • R1.15 - Organisations must make sure that work undertaken by doctors in training provides learning opportunities and

feedback on performance, and gives an appropriate breadth of clinical experience. • R1.19 - Organisations must have the capacity, resources and facilities to deliver safe and relevant learning opportunities,

clinical supervision and practical experiences for learners required by their curriculum or training programme and to provide the required educational supervision and support.

• R5.9 - Postgraduate training programmes must give doctors in training: o a - training posts that deliver the curriculum and assessment requirements set out in the approved curriculum o b - sufficient practical experience to achieve and maintain the clinical or medical competences (or both) required

by their curriculum Supportive environment

These questions make up the ‘Supportive environment’ indicator. Please state whether you agree or disagree with the following statements about your post:

• In general, the working environment is a supportive one. • Staff, including doctors in training, are treated fairly. • Staff, including doctors in training, treat each other with respect. • The working environment is one which helps build the confidence of doctors in training. • If I were to disagree with senior colleagues, they would be open to my opinion.

Why is this score useful? A supportive environment helps doctors in training to learn and develop. Low scores suggest that there may be issues with the training environment. This may also have an impact on patient safety. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.1 - Organisations must demonstrate a culture that allows learners and educators to raise concerns about patient safety, and the standard of care or of education and training, openly and safely without fear of adverse consequences.

• R1.5 - Organisations must demonstrate a culture that both seeks and responds to feedback from learners and educators on compliance with standards of patient safety and care, and education and training.

• R3.3 - Learners must not be subjected to, or subject others to, behaviour that undermines their professional confidence, performance or self-esteem.



Trainee indicator Questions Workload These questions make up the 'Workload' indicator.

• How would you rate the intensity of your work, by day in this post? • How would you rate the intensity of your work, by night in this post? • In this post, how often have you worked beyond your rostered hours? • In this post, how often did your working pattern leave you feeling short of sleep when at work?

Please note there are different versions of these questions for Pharmaceutical Medicine trainees. Why is this score useful? High workloads may lead to fatigue and increased likelihood of error. Low scores are an indicator of a post where work intensity and/or long hours may lead to sleep deprivation or exhaustion. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.


• R1.12e - Organisations must design rotas to minimise the adverse effects of fatigue and workload.

Educational supervision

These questions make up the 'Educational supervision' indicator.

• Did you have a designated educational supervisor (the person responsible for your appraisal) in this post? • In this post did you have a training/learning agreement with your educational supervisor, setting out your respective

responsibilities? • In this post did you use a learning portfolio? • In this post were you told who to talk to in confidence if you had concerns, personal or educational?

Please note there are different versions of these questions for Pharmaceutical Medicine trainees.



Trainee indicator Questions Why is this score useful?

This indicator is about the educational support underpinning the post. Low scores suggest attention should be paid to the management of the training programme and the effectiveness of the educational supervisor role. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.9 - Learners’ responsibilities for patient care must be appropriate for their stage of education and training. Supervisors must determine a learner’s level of competence, confidence and experience and provide an appropriately graded level of clinical supervision.


• R1.12d - Organisations must design rotas to give doctors in training access to educational supervisors • R1.19 - Organisations must have the capacity, resources and facilities to deliver safe and relevant learning opportunities,

clinical supervision and practical experiences for learners required by their curriculum or training programme and to provide the required educational supervision and support.

• R1.21 - Organisations must make sure learners are able to meet with their educational supervisor or, in the case of medical students, their personal advisor as frequently as required by their curriculum or training programme.

• R2.15 - Organisations must make sure that each doctor in training has access to a named educational supervisor who is responsible for the overall supervision and management of a doctor's educational progress during a placement or a series of placements. The educational supervisor regularly meets with the doctor in training to help plan their training, review progress and achieve agreed learning outcomes. The educational supervisor is responsible for the educational agreement, and for bringing together all relevant evidence to form a summative judgement about progression at the end of the placement or a series of placements.

• R5.9f - Postgraduate training programmes must give doctors in training regular, useful meetings with their clinical and educational supervisors

Access to educational resources

These questions make up the 'Access to educational resources' indicator.

• Overall, how would you rate the educational resources available to you in this post?



Trainee indicator Questions • How good or poor was access to each of the following in your post?

o Library o Online journals

• How good or poor was access to each of the following in your post? o E-learning resources o Internet access o Space for private study

• How good or poor was access to the following in your post? o Equipped rooms for group teaching o Simulation facilities

Why is this score useful? It is important that trainees have access to educational resources to support their training. A low score indicates that trainees consider the availability or the quality of educational resources to be poor. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.18 - Organisations must make sure that assessment is valued and that learners and educators are given adequate time and resources to complete the assessments required by the curriculum.

Feedback These questions make up the 'Feedback' indicator.

• How often did you have informal feedback from a senior clinician on how you were doing in this post? • Did you have a formal meeting with your educational supervisor to talk about your progress in this post? • Did you have a formal assessment of your performance in the workplace in this post?

Please note there are different versions of these questions for Public Health and Pharmaceutical Medicine trainees.




Feedback is an important factor in learning. Low scores indicate less informal or formal feedback. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R3.13 - Learners must receive regular, constructive and meaningful feedback on their performance, development and progress at appropriate points in their medical course or training programme, and be encouraged to act on it. Feedback should come from educators, other doctors, health and social care professionals and, where possible, patients, families and carers.

Local teaching These questions make up the 'Local teaching' indicator.

• How would you rate the quality of this local/departmental teaching for this post? • For how many hours per week was the local/departmental basis specialty-specific teaching provided in this post? • When attending these local/departmental sessions, in this post, how often did you have to leave a teaching session to

answer a clinical call? • When attending these local/departmental sessions in this post who covered your service work? • In this post who provided the local/departmental teaching?

Please note there are different versions of some questions for Public Health Medicine trainees. Why is this score useful? On the job training is supplemented by local or departmental teaching. Low scores indicate less frequent or poor quality sessions, or sessions provided by other trainees without senior doctors supervising. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.



Trainee indicator Questions • R2.6 - Medical schools, postgraduate deaneries and LETBs must have agreements with LEPs to provide education and

training to meet the standards. They must have systems and processes to monitor the quality of teaching, support, facilities and learning opportunities on placements, and must respond when standards are not being met.

Regional teaching These questions make up the 'Regional teaching' indicator.

• In this post, was specialty-specific teaching provided on a deanery/regional/school wide basis? • In this post how frequently was this deanery/regional/school specialty-specific teaching provided? • Were you able to attend these whilst in this post? • How would you rate the quality of this deanery/regional/school specialty-specific teaching for this post?

Why is this score useful? Regional teaching sessions supplement on-the-job learning and local teaching. Low scores indicate trainees perceived that regional teaching was less frequent, difficult to attend or poor quality. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R2.6 - Medical schools, postgraduate deaneries and LETBs must have agreements with LEPs to provide education and training to meet the standards. They must have systems and processes to monitor the quality of teaching, support, facilities and learning opportunities on placements, and must respond when standards are not being met.

Study leave These questions make up the 'Study leave' indicator.

• In this post how would you rate the encouragement you had to take study leave? • In this post were you able to access funds to cover the cost of all courses that were recommended for you to complete? • Were any days subtracted from your study leave allowance for compulsory training in this post (in or out of hospital)? • In this post did you have difficulty obtaining study leave for any of the following reasons?

Please note there are different versions of some questions for Public Health Medicine and Pharmaceutical Medicine trainees.




It is important that trainees can take leave to support their studies. A low score indicates that study leave was less accessible, less flexible or more expensive. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.16 - Doctors in training must have protected time for learning while they are doing clinical or medical work, or during academic training, and to attend organised educational sessions, training days, courses and other learning opportunities to meet the requirements of their curriculum. In timetabled educational sessions, doctors in training must not be interrupted for service unless there is an exceptional and unanticipated clinical need to maintain patient safety.

• R3.12 - Doctors in training must be able to take study leave appropriate to their curriculum or training programme, to the maximum time permitted in their terms and conditions of service.

Reporting systems These questions make up the ‘Reporting systems’ indicator. Please state whether you agree or disagree with the following statements about your post:

• I have been made aware of how to report patient safety incidents and near misses. • There is a culture of proactively reporting concerns. • There is a culture of learning lessons from concerns raised. • I am confident that concerns are effectively dealt with. • When concerns are raised, the subsequent actions are fed back appropriately.

Why is this score useful? It is important that trainees know how to report concerns and are comfortable reporting any concerns they might have. A low score indicates that the reporting systems process may be flawed in some way. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following.

• R1.1 - Organisations must demonstrate a culture that allows learners and educators to raise concerns about patient



Trainee indicator Questions safety, and the standard of care or of education and training, openly and safely without fear of adverse consequences.

• R1.2 - Organisations must investigate and take appropriate action locally to make sure concerns are properly dealt with. Concerns affecting the safety of patients or learners must be addressed immediately and effectively.

• R1.3 - Organisations must demonstrate a culture that investigates and learns from mistakes and reflects on incidents and near misses. Learning will be facilitated through effective reporting mechanisms, feedback and local clinical governance activities.


• R1.6 - Organisations must make sure that learners know about the local processes for educational and clinical governance and local protocols for clinical activities. They must make sure learners know what to do if they have concerns about the quality of care, and they should encourage learners to engage with these processes.

• R2.7 - Organisations must have a system for raising concerns about education and training within the organisation. They must investigate and respond when such concerns are raised, and this must involve feedback to the individuals who raised the concerns.



Annex G – Trainer benchmark groups Report Benchmark group Trainer specialty by trust/board All non-GP trainers

Trainer specialty by site All non-GP trainers

Trust/board All non-GP trainers

Site All non-GP trainers

LETB/Deanery All trainers

GP GP trainers only



Annex H – trainer indicator questions Trainer indicator Questions Organisational culture

These questions make up the 'Organisational culture’ indicator. To what extent do you agree or disagree with the following statements?:

• My DEPARTMENT demonstrates that it values the education of trainees. • My TRUST/BOARD demonstrates that it values the education of trainees. • My DEANERY/LETB demonstrates that it values the education of trainees. • Within my DEPARTMENT there is a culture of proactively reporting concerns about education. • Within my TRUST/BOARD there is a culture of proactively reporting concerns about education. • Training in this trust/board is managed consistently between departments. • Training in this deanery/LETB is managed consistently between trusts/boards.

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? The organisational culture should support education and training so that learners are available to demonstrate what is expected in ‘Good medical practice’ (2013) and to achieve the learning outcomes required by their curriculum. Learners will have a good educational experience and educators will be valued where there is an organisational commitment to, and support for, learning. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R1.1 – Organisations must demonstrate a culture that allows learners and educators to raise concerns about patient safety, and the standard of care or of education and training, openly and safely without fear of adverse consequences.

• R1.6 – Organisations must make sure that learners know about the local processes for educational and clinical governance and local protocols for clinical activities. They must make sure learners know what to do if they have



Trainer indicator Questions concerns about the quality of care, and they should encourage learners to engage with these processes.

• R1.7 – Organisations must make sure there are enough staff members who are suitably qualified, so that learners have appropriate clinical supervision, working patterns and workload, for patients to receive care that is safe and of a good standards, while creating the required learning opportunities.

Handover These questions make up the 'Handover’ indicator.

To what extent do you agree or disagree with the following statements?:

• In my department, appropriate members of the multidisciplinary team are included in handover. • Handover arrangements in my department always ensure continuity of care for patients BETWEEN SHIFTS. • Handover arrangements in my department always ensure continuity of care for patients BETWEEN DEPARTMENTS.

Please note there are different versions of some questions for General Practice trainers. Why is this score useful?

A good handover ensures that individuals understand what’s required to manage the care for patients between shifts and departments and helps them identify situations they may need assistance with. Low scores indicate that arrangements for handover may provide less continuity of care for patients. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R1.14 - Handover* of care must be organised and scheduled to provide continuity of care for patients and maximise the learning opportunities for doctors in training in clinical practice.

* Handover at the start and end of periods of day or night duties, every day of the week.



Trainer indicator Questions Supportive environment

These questions make up the 'Supportive environment’ indicator. To what extent do you agree or disagree with the following statements?:

• In general, the working environment in my department is a supportive one. • Staff, including trainee doctors, are treated fairly in my department. • Staff, including trainee doctors, treat each other with respect in my department. • The working environment in my department is one which helps build the confidence of trainee doctors. • If I were to disagree with senior colleagues in my trust/board, they would be open to my opinion.

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? A supportive environment helps doctors to learn and develop. Low scores suggest that there may be issues with the training environment. This may also have an impact on patient safety. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R1.1 - Organisations must demonstrate a culture that allows learners and educators to raise concerns about patient safety, and the standard of care or of education and training, openly and safely without fear of adverse consequences.


• R3.3 - Learners must not be subjected to, or subject others to, behaviour that undermines their professional confidence, performance or self-esteem.

Educational governance

These questions make up the 'Educational governance’ indicator. To what extent do you agree or disagree with the following statements?:

• I'm confident that my DEPARTMENT would act effectively if concerns about education were raised. • I'm confident that my TRUST/BOARD would act effectively if concerns about education were raised.



Trainer indicator Questions • I'm confident that my DEANERY/LETB would act effectively if concerns about education were raised.

• I’m confident that my deanery/LETB works collaboratively with my trust/board to ensure trainees’ educational needs are balanced with service commitments.

• How effective or ineffective is your trust/board in making changes to help improve the provision of education?

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Organisations should have effective systems of educational governance to manage and control the quality of medical education and training. Information should be shared across educational and clinical governance systems to identify risk to patient safety and the quality of education and training, and to ensure transparency and accountability. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R2.7 – Organisations must have a system for raising concerns about education and training within the organisation. They much investigate and respond when such concerns are raised, and this must involve feedback to the individuals who raised the concern.

• R2.8 – Organisations must share and report information about quality management and quality control of education and training with other bodies that have educational governance responsibilities. This is to identify risk, improve quality locally and more widely, and to identify good practice.

• R4.5 - Organisations must support educators to liaise with each other to make sure they have a consistent approach to education and training, both locally and across specialties and professions.

Time for trainers These questions make up the 'Time for trainers’ indicator.

To what extent do you agree or disagree with the following statements?:

• My job plan contains enough designated time for my role as an educator. • I am always able to use the time allocated to me in my role as an educator specifically for that purpose.



Trainer indicator Questions • As an Educational Supervisor, my job plan contains enough time to meet with my trainee(s) as frequently as they

require. Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Those involved in training should have time for learning while they’re doing clinical or medical work and this time shouldn’t be interrupted for service unless there is an exceptional and unanticipated clinical need to maintain patient safety. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R2.10 - Organisations responsible for managing and providing education and training must monitor how educational resources are allocated and used, including ensuring time in trainers’ job plans.

• R4.2 - Trainers must have enough time in job plans to meet their educational responsibilities so that they can carry out their role in a way that promotes safe and effective care and a positive learning experience.

• R4.6 - Trainers in the four specific roles must be developed and supported, as set out in GMC requirements for recognising and approving trainers1.

Appraisals These questions make up the 'Appraisals’ indicator.

To what extent do you agree or disagree with the following statement?:

• The educational responsibilities expected of me in my role are clearly defined.

• If you've had an appraisal within the last 12 months, how effective was it in reviewing your EDUCATIONAL responsibilities?

Please note there are different versions of some questions for General Practice trainers.

1 ‘Recognising and approving trainers: the implementation plan’ (2012)



Trainer indicator Questions

Why is this score useful? The appraisals process can not only help ensure that trainers feel supported and developed in their roles but can act as an opportunity to review and support their performance and identify opportunities for personal development. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R4.1 - Educators must be selected against suitable criteria and receive an appropriate induction to their role, access to appropriately funded professional development and training for their role, and an appraisal against their educational responsibilities.

Support for trainers These questions make up the 'Support for trainers’ indicator.

• How would you rate the support you receive from your DEPARTMENT in your role as an educator? • How would you rate the support you receive from your TRUST/BOARD in your role as an educator? • How would you rate the support you receive from your DEANERY/LETB in your role as an educator?

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Educators should receive the support the need to meet their educational and training responsibilities. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R4.4 - Organisations must support educators by dealing effectively with concerns or difficulties they face as part of their educational responsibilities.

• R4.6 - Trainers in the four specific roles must be developed and supported, as set out in GMC requirements for



Trainer indicator Questions recognising and approving trainers2.

Resources for trainers

These questions make up the 'Resources for trainers’ indicator. To what extent do you agree or disagree with the following statements?:

• Within my trust/board there are enough staff to ensure that patients are always treated by someone with an appropriate level of clinical experience.

• The IT resources I need to perform my role as an educator are available to me in my trust/board. • The resources in my trust/board allow me to cover the parts of the curriculum required by my trainee(s) in their post.

• How would you rate the access to professional development and training opportunities which your TRUST/BOARD

arranges for your role as an educator? • How would you rate the access to professional development and training opportunities which your DEANERY/LETB

arranges for your role as an educator? Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Educators should be provided with the resources required by them to meet their education and training responsibilities and help them deliver effective education and training. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R2.10 - Organisations responsible for managing and providing education and training must monitor how educational resources are allocated and used, including ensuring time in trainers’ job plans.

• R4.2 – Trainers must have enough time in job plans to meet their educational responsibilities so that they can carry out their role in a ways that promotes safe and effective care and a positive learning experience.




Trainer indicator Questions • R4.3 - Educators must have access to appropriately funded resources they need to meet the requirements of the

training programme or curriculum.

Supervisor training These questions make up the 'Supervisor training’ indicator. To what extent do you agree or disagree with the following statements?:

• I am able to access learning and development opportunities for my role as an educator when I need to. • Overall, I am satisfied with the training opportunities offered to me as an educator.

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Educators should be provided with support for their own training and professional development. This can help them to improve the quality of training they deliver, particularly in relation to acting as a positive role model and demonstrating good medical practice, teaching and feedback and assessment decisions. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R4.1 - Educators must be selected against suitable criteria and receive an appropriate induction to their role, access to appropriately funded professional development and training for their role, and an appraisal against their educational responsibilities.

• R4.6 - Trainers in the four specific roles must be developed and supported, as set out in GMC requirements for recognising and approving trainers3.




Trainer indicator Questions Quality of curricula and assessments

These questions make up the 'Quality of curricula and assessments’ indicator. To what extent do you agree or disagree with the following statements?:

• The assessments completed by my trainees are an effective part of their educational development. • The content of my FOUNDATION TRAINEES' curriculum is appropriate to their needs. • The content of my CORE TRAINEES' curriculum is appropriate to their needs. • The content of my SPECIALTY TRAINEES' curriculum is appropriate to their needs. • The content of my GP SPECIALTY TRAINING TRAINEES' curriculum is appropriate to their needs.

Please note there are different versions of some questions for General Practice trainers. Why is this score useful? Postgraduate curricula and assessments should be implemented so that doctors in training are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum. This indicator relates to a number of requirements within ‘Promoting excellence: standards for medical education and training’ (2015) including the following:

• R5.9a - Postgraduate training programmes must give doctors in training posts that deliver the curriculum and assessment requirements set out in the approved curriculum.

• R5.9b - Postgraduate training programmes must give doctors in training sufficient practical experience to achieve and maintain the clinical or medical competences (or both) required by their curriculum.

• R5.10 - Assessments must be mapped to the requirements of the approved curriculum and appropriately sequenced to match doctors’ progression through their education and training.



Annex I – column heading definitions In the reporting tool, you will find data in columns in indicator analysis (tabular data) and in outlier analysis (all results view). The table below explains what the column headings mean.

Column heading Description Lower CI Lower confidence limit for the report group

Mean The mean score of the report group

n range A range, in increments of five, representing the number of respondents in the report group for this report

National Lower CI Lower 95% confidence limit for the benchmark group

National Max The highest score of all respondents in the benchmark group

National Mean The mean score of the benchmark group

National Median The median score of the benchmark group

National Min The lowest score of all respondents in the benchmark group

National N The number of respondents in the benchmark group for this report. This may vary by indicator due to not applicable responses.

National Q1 Upper value of the first quartile or the 25th percentile of the benchmark group

National Q3 The lower value of the first quartile or the 75th percentile of the benchmark group

National Upper CI Upper 95% confidence limit for the benchmark group

Outcome (outlier analysis only) The outcome (including outliers) for the report group

Response rate The percentage of eligible trainers who completed the survey. These figures were calculated once the data had been cleansed after the survey to give improved accuracy. We have calculated an individual response rate for each report group which is displayed in the reporting tool.

SD (outlier analysis only) Standard deviation of the report group mean

Upper CI Upper confidence limit for the report group



Annex J – glossary Term Definition

Benchmark group The benchmark group is the group of respondents whose scores are used to calculate the comparative mean and the quartiles for this report. This is not the score, but the value that the score is compared against to give it meaning. For example a score of 55 out of 100 is only meaningful if you know the average score for the benchmark group is 40. You can then use the quartiles to see how many respondents gave a score within certain ranges.

The benchmark group is the group of respondents whose scores are used to calculate the comparative mean and the quartiles for the report you are looking at. This is not the score, but the value that the score is compared against to give it meaning. For example a score of 55 out of 100 is only meaningful if you know the average score for the benchmark group is 40. You can then use the quartiles to see how many respondents gave a score within certain ranges.

To ensure survey results are shown in context, the benchmark group changes depending on the report.

For example, a report showing the survey scores for a particular post specialty uses the relevant specialty group as the benchmark. So results for Cardiology at a particular trust/board are compared to trainees in only medical posts UK-wide.

By excluding less relevant groups (in this example: surgical trainees etc.) from the benchmark calculation, the differences between scores are more likely to reflect differences between the trusts/boards, than the differences found between different specialties.

You can check the benchmark group for the report you’re looking at in the Applied filters box:

See Annex A for a list of the benchmark groups used.

Confidence limit A confidence limit is the upper or lower value of the confidence interval. In these reports, the confidence interval is calculated against the mean indicator score for the report group.

The smaller the confidence intervals (i.e. the closer the confidence limits are to the mean score), the more confidence you can have that the results you can would be true across a larger sample size.



Term Definition

Indicator An indicator is a measurement of a concept within the survey. These are made up of several question items on the same theme, for example Educational Supervision. To see the question items that make up an indicator and the individual question item scores for the report group you are interested in, click on the relevant bar and choose Question Items from the pop up menu.

Mean The mean is the average score (the sum of all the scores divided by the number of respondents). There are two means for each report. One is for the benchmark group, the other is for the report group.

Median The median is the central value of the benchmark group, or the 50th percentile. This is calculated by identifying the middle value, after sorting numerically all the scores across all responses for the benchmark group for the report you’re looking at.

N (benchmark group population) N is the number of respondents whose score contributed to the benchmark group.

This may not always be the full population of the survey as respondents scores are only included in an indicator score if they answer at least all but one of the questions for that indicator (ie four out of five questions or three out of four questions). This also applies to the report group population (n range).

Answering ‘not applicable’ to a question returns a null score. This means that if a respondent answers ‘not applicable’ to two questions within the same indicator, their responses to the other questions within the indicator are not included in the overall indicator score.

However, if you navigate to the question item analysis, the scores of the respondents excluded from the overall indicator score will be included in individual question item scores where they have answered the question.

n range (report group population) n range is range, in increments of five, representing the number of respondents whose score contributed to the report group. This is affected by the filters you apply in indicator analysis. Please see the entry for N above which explains why N or n range can differ from indicator to indicator.

Outlier An outlier is a score that is distant (positively or negatively) from the average.



Outliers are defined using two criteria (both must apply):

Term Definition

• Confidence interval comparison: A below outlier has an upper 95% confidence limit that is below the lower 95% confidence limit on the benchmarking group’s mean. An above outlier has a lower 95% confidence limit that is above the upper 95% confidence limit on the benchmarking group’s mean.

• The quartile containing the mean: For a below outlier the mean must be in the bottom quartile and for an above outlier the mean must be in the top quartile.

Percentile A percentile is a measurement of how many scores fall into a certain range. So if in a range from 0-100 the 75th percentile is at 55, you know that 25% of the scores were higher than 55.

Quartile Quartiles are the ranges of values based on the percentiles and medians calculated on the full range of scores. Q1 is the range from the lowest score recorded to the 25th percentile. Q2, also known as the interquartile range, is the range from the 25th to the 75th percentile. The median is the central value of Q2. Q3 is the range from the 75th percentile to the highest score recorded.

Report group The report group for any report is the group of respondents whose scores contribute to the bar of the graph (in indicator analysis) or the row of the chart (in outlier analysis) you are looking at.

Response rate The percentage of eligible trainers who completed the survey. These figures were calculated once the data had been cleansed after the survey to give improved accuracy. We have calculated an individual response rate for each report group which is displayed in the reporting tool.

Trainee Doctors in training who are in GMC approved training posts on the census date are included in the survey.

This includes:

• Foundation trainees

• Core trainees

• Higher specialty training, including SpR and GP trainees

• FTSTA trainees, LAT trainees



• Military trainees

• Trainees in Clinical Lecturer and Academic Clinical Fellowship posts

• Out of programme training (OOPT) on an approved programme in the UK

• Post-CCT training towards a sub-specialty

• Trainees working for non-NHS organisations, e.g. occupational medicine, palliative medicine and pharmaceutical medicine

Doctors who are not in training on the census date (eg doctors who are on maternity leave or who have been awarded their CCT) are not included in the survey.

Non-medical public health trainees complete a separate survey.

Trainer All named postgraduate clinical and educational supervisors in the UK are eligible to take the trainer survey.

• Clinical supervisors oversee a trainee's clinical work throughout a placement. They give feedback during the placement and provide a review of the trainee's performance to help educational supervisors determine if the trainee progresses to the next stage of their training.

• Educational supervisors are responsible for the overall supervision and management of a trainee's development during a placement or series of placements. They help the trainee plan their training, achieve agreed learning outcomes and bring together all relevant evidence to determine if the trainee progresses to the next stage of their training.

Although these two roles are distinct, trainers may have responsibility for both.

reporting tool user guide

Documents