1

Appendix 4: Technical audit

Item Structure Practices Performance Accessibility standards

Use structured semantic markup

Images contain descriptive alt tags

Complex images include longdesc tags

Site uses cascading images

Site uses relative font sizes

Browsers without stylesheet capacity are supported

Accessibility is prioritized

Bobby AAA approved

Validate XHTML 1.0 Strict

W3C Web accessibility initiative’s Preliminary Review of Web Sites for Accessibility used to assess performance

Not a full conformance evaluation Website has few functional issues, most

were about conforming to standards Manual review showed few issues No titles visible when images blocked Larger text does not impact website design Low quality monitors have no impact on

website design Minor issues with mouseless browsing Blind users can use IBM Home Page Reader

but not BrowseAloud issues Automated review showed numerous Functional Accessibility Evaluator showed

issues with navigation and orientation HTML Validator showed Patient Opinion’s

site does not conform to XHTML standards SortSite showed the website quality was

below average in 5 of 7 categories

Content Database

Back up to remote servers

Database of 500mb

Use an in house, freetext search

The server is backed up every 6 hours

Use a service-onomy tagging system

Storage isn’t an issue Want to improve searches for the public More features and operations being

developed for subscribers

Content management system

Self-developed content management system

Staff manage some online content functions

Technician can manage all functions

Three faces of website

Admin edit post first if necessary

Staff make minimal changes

Patient Opinion informs patients of any editing

95% posts are published Site users and limitations Public difficulty to search for services and

types of postings Subscribers search postings for trusts

services Some subscribers want quantitative

summaries Patient Opinion staff face needs more work

on its accessibility for patients with learning difficulties and non English speakers

2

Item Structure Practices Performance

Public Subscriber (by

log in): Admin (by log

in): Use a hired

server

CRM system Development list created from requests

Fairly ad hoc timescales

70-80% of posts are published within 48 hours

Requests often go into a development stream

Procedures exist for learning from feedback

Backlogs do occur Software developer has left

Security measures

Have standard windows security in place

Patient Opinion don’t encrypt the database for anonymity

Strenuous updating of security and software

100s of hacking attempts per day Never had any security breaches

Profiling of users

Google analytics shows many details about users

Ability to profile by age, % of feedback which is published and depravation index

Hansard Society Survey

After the home page, the Opinions page is visited most often:

Internet Explorer browser used by nearly 70% of users:

Average time on Site is 00:02:59 5.5 average page views per user 95% of Hansard survey respondents were

frequent users of the internet? 63.5% of Hansard survey respondents

were female 89.6% of Hansard survey respondents live

in England (Hansard) Gender (where known): 30.5% male 69.5%

female (from post details on website) Majority of website users are over 60:

(from post details on website) 60+ females – 49.9%

3

Item Structure Practices Performance

60+ males – 64.3% 60+ overall – 56% Hansard had different age groups, only

47.2% over 55, 16.4% over 65

Language capacities

Enabling Google translate

Translating the site into Welsh, Italian Spanish and Catalan

A local community group help translate posts where necessary

Patient Opinion can’t handle postings in languages other than English

Only one example of groups translating a post

Reliability of the system

Adapting the system can produce errors for admin side

Technician handles any errors

Possible to bring in someone for a serious crash

Whole site has never crashed for two years Lose connectivity twice a year for short

periods

Speed of network/ internet

100mb/s connection

Server is on the internet

Patient Opinion claim to have the capacity to deal with huge surges in demand

Mystery shopping showed that during a Radio 4 appearance the website was inaccessible

Traffic Patient Opinion do regularly measure traffic

Don’t utilise Google Adverts

evidence of Number of hits Hit rates Traffic to the site They can turn ad

hoc queries to reports

The traffic is not rising very much Patient Opinion isn’t able to give their

product away for free Currently 60%-70% of the website hits are

from Google searches 19,974 page titles were viewed a total of

146,804 times in February Average 5243 page views a day. 21,704 people visited this site in February 5.52 Average Page Views per user 8.03 Posts per day in February 2010. 106.83 visits per post in February 2010. Patient Opinion work on the assumption

that there’s 1 posting per 1000 page views as a rough estimate.

4

Item Structure Practices Performance

Inter-operability

The application will work on other Windows servers

Data interoperability RSS and Atom as well as XLN API

Developed using packages (Adobe and MS) not open source

Not reliant on the initial website developers

Support for software/ hardware

Operating System by Asp.net

Hardware quad processor

Technician provides all support since software developer left

Basic office IT is outsourced

Often use freeware which is easy to learn:

Database management uses Visual Studio 2008 and SQL Compare

PO technical person now quite overloaded The team thinks that internal technician is

“very good” Currently risk over-reliance on him due to

lack of adequate replacement within the team

Performance indicators

No internal Service Level Agreements

They produce company-wide standards

Production of documentation

Dedicated blog page on website

“Patient Opinion in the Press” page documents national press coverage

Use software packages for leaflet production

Blogs are produced with reduced frequency

Press articles are linked on the website routinely after publication

Patient Opinion has produced 85 blog entries since May 2008.

33% of website visitors have visited the Patient Opinion blog

There are also 14 articles on the Patient Opinion in the Press page.

21% of respondents had found out about Patient Opinion through the media coverage

5

1. Accessibility 1.1 Structures

Use structured semantic markup: All pages on this site use structured semantic markup.

h1 tags are used for main titles, h2 tags for subtitles (and so on). For example, on this page, JAWS users can skip to the next section within the accessibility statement by pressing ALT+INSERT+3.

Images contain descriptive alt tags: All content images used in this site should

include descriptive alt tags. Purely decorative graphics include null alt tags.

Complex images include longdesc tags: Complex images include longdesc tags or inline descriptions to explain the significance of each image to non-visual readers.

Site uses cascading images: This site uses primarily uses cascading style sheets for

visual layout. The use of tables for layout has been kept to a bare minimum.

Site uses relative font sizes: This site uses only relative font sizes, compatible with the user-specified "text size" option in visual browsers.

Browsers without stylesheet capacity are supported: If your browser or browsing

device does not support stylesheets at all, the content of each page should still be readable.

1.2 Practices

Accessibility is prioritized: For Patient Opinion accessibility has increased importance as the majority of users will have health issues which can range from blindness to mental health issues and may be socially disadvantaged leading to poor quality computer hardware and limited skill in using software. In addition, many of Patient Opinion’s users are elderly who have said throughout the evaluation that they have limited IT skills. Therefore the accessibility of the website, its clarity and easy of use is well attended to and described at length on the website1.

Bobby AAA approved: All pages on this site should be Bobby AAA approved,

complying with all priority 1, 2 and 3 guidelines of the W3C Web Content Accessibility Guidelines as well as US Government Section 508 Guidelines.

Validate XHTML 1.0 Strict: All pages on this site should validate as XHTML 1.0 Strict.

1 http://www.patientopinion.org.uk/info.aspx?pageID=accessibility

6

1.3 Performance

W3C Web accessibility initiative’s Preliminary Review of Web Sites for Accessibility used to assess performance: Accessibility is an important issue for Patient Opinion due to the differing users each website has, with a range of browsers, computers, monitors and disabilities. We have followed the W3C Web accessibility initiative’s Preliminary Review of Web Sites for Accessibility2 which although comprehensive is not sufficient to determine if a Web site conforms to Web accessibility guidelines. However it is sufficient to highlight the kinds of issues that Patient Opinion suffers with regards to accessibility.

Manual review showed few issues: On the whole, the manual review of the website

showed very few issues that users may have had with accessibility. There was no audio content so transcriptions were not an issue.

No titles visible when images blocked: Users without image processing were

mostly without the title of the webpage but apart from this the site was still fully functional as the missing links were accessible elsewhere on the page.

Larger text does not impact website design: Users with poor eyesight would see

that the integrity of the website design is unimpaired by having larger text. Low quality monitors have no impact on website design: Users with low quality

monitors would encounter no functionality or design issues when the site had lower resolution.

Minor issues with mouseless browsing: For users who can’t use a mouse there was

a major problem halfway down the blog page but all other pages had no issues. Blind users can use IBM Home Page Reader but not BrowseAloud: The website

was also usable for blind users who used IBM’s Home Page Reader on all pages tested. However if blind users used other software packages such as BrowseAloud they would be unable to use the site.

Automated review showed numerous issues: using the automated evaluations of

web accessibility, the website did not fare as well. Functional Accessibility Evaluator showed issues with navigation and

orientation: Using the Functional Accessibility Evaluator, the website had only partial implementation on navigation and orientation whilst HTML standards and

2 A preliminary review combines some manual checking of representative pages on a Web site, along with the use of several semi-automatic accessibility evaluation tools.

7

Scripting were almost complete. The areas of navigation and orientation which the website failed were default language where the lang attribute was not specified, and the form control labels which did not meet the criteria. There were also warnings for the accessibility on onclick scripting, the W3C Specifications and the Titles (as in the first part of the accessibility evaluation).

HTML Validator showed Patient Opinion’s site does not conform to XHTML

standards: HTML Validator showed a number of issues and 1 failure on each webpage evaluated indicating that the website does not fully conform to XHTML website standards.

SortSite showed the website quality was below average in 5 of 7 categories:

According to SortSite, the website had quality issues on 8 of the 11 pages tested and the quality was overall below average being below average on 5 of the 7 quality categories. On usability, the website did not breach W3C Best Practices or Readability but violated usability.gov Guidelines. On accessibility, the automated analysis suggests that disabled users will find it impossible to use some pages and difficult to use some others. On compliance with standards, Patient Opinion was validated on W3C CSS but some pages failed validation on the other three measures: W3C HTML/XHTML Validation, W3C Style Guide and W3C Deprecated Features. On the issues of compliance with EU law and compatibility with other web browsers the Patient Opinion site performed well and was above average.

Not a full conformance evaluation: This accessibility evaluation was not a full

‘conformance’ evaluation but highlighted that there may be some issues with accessibility for disabled users in particular.

Website has few functional issues, most were about conforming to standards: It

was also notable that the manual testing showed that the site was almost fully usable on the selected criteria so that the majority of the automated test results pointed to issues which may not have extremely significant implications for users.

2. Content Database

2.1 Structure

Back up to remote servers: Patient Opinion back up to their server is from amazon webservices which are effectively infinite in their amount of storage

Database of 500mb: The Patient Opinion Database is about ½ gig, with around 150

gig available (not near full capacity)

8

Use an in house, freetext search: It’s an in house search, freetext search, which is a part of secret server 2008 which is what PO are backing up to.

2.2 Practices The server is backed up every 6 hours Use a service-onomy tagging system: Tagged by users when they go on the site so

it’s a serviceomy not a taxonomy. PO moderate all tags to ensure that tags meet legal and ethical requirements. Every now and then PO do some weeding of tags to make sure they’re still relevant and useful.

2.3 Performance

Storage isn’t an issue: “Storage isn’t an issue these days.” Want to improve searches for the public: For the public user – PO want to make

searches easy, more straightforward and simple. So they are looking at various search options at the moment to make it easier to search for the precise set of posting that a person is looking for.

More features and operations being developed for subscribers: Subscribers -

easier to derive sets of posting in the search. PO are going to focus on how to once people have a set of postings, defining various sets of operations they want to do such as report creation and alerts.

3. Content management system

3.1 Structure

Self-developed content management system: Headshift (London based) who specialise in working with social organisations, developed the Patient Opinion website. The website has since been redesigned by an internal ‘technician’ a few years ago. This enabled Patient Opinion to have full control of the website, which was seen as important to Patient Opinion.

Staff manage some online content functions: Staff can manage some aspects of the

site; build subscriptions, enter details and postings. Technician can manage all functions: The internal member of staff (technician) is

the key web person however a new programmer will be starting in Oct 20093.

3 Left after a few months

9

Three faces of website: The website has three faces: Public – www.patientopinion.org.uk Subscriber (by log in): They can search posts, set up email system, add members

and generate reports. Admin (by log in): Basic look to it. Not seen by public and subscribers. Admin link

posts to hospitals, trusts and key words. Keywords and patients post codes help link them to services.

Use a hired server: PO hired a dedicated server from a webposting company who set

it up and leave PO to then install secret server 2005, and reporting services. PO log into it every couple of days and upload new versions of the application and “fiddle around … it just runs like that.”

3.2 Practices

Admin edit post first if necessary: When a post is made the admin team look at it first (not public). They may need to edit the post and then they tag it for monitoring purposes and for the subscribers.

Staff make minimal changes: Names of staff are taken out, discriminatory terms are

taken out and spelling and grammar changes are made if it needs to become readable, capital words are taken out and names of conditions are corrected if need be. However, Patient Opinion like to make minimal changes so that the postings remain the voices of the individual. The team report that it has been rare to receive any racist comments.

Patient Opinion informs patients of any editing: Patients are made aware by email

that their posts may be moderated or not published. If they are not published than the patient is made aware of the reasons.

3.3 Performance

95% posts are published. Patient Opinion’s goal is to publish all posts as PO is about patient experience and hearing patient’s voices.

10

4. Site users and limitations Users (in priority) Limitations of website for user Public Limitations for the public include a difficulty to

search for services and types of postings. There is a need to emphases subscriber’s responses and changes. Some public users want to be able to compare hospitals and services but this is not what Patient Opinion is trying to do. They don’t want it to be statistical but to be about the stories behind the figures.

Subscribers It can be difficult to search postings for trusts services. The search function is too complicated and is not ideal. There are plans to develop it. Some subscribers want quantitative summaries but these are not currently available. They offer reports, which say when the post was made, the PCT or Local authority in which the post was made and key words analysis.

Patient Opinion Staff

There is a margin for error with the thematic analysis (language processing). Generally the face for the Patient Opinion staff is fine and the team are happy. They know that it needs more work on its accessibility for patients with learning difficulties and non English speakers. There needs to be a balance between tokenism and effectiveness. It could be hard to deal with the stories if they need to be translated. It needs a lot of thought and a pilot. “Patient Opinion does not represent all as it is not a survey company”. But they do want to appeal to a broader range of people.

11

5. CRM system

5.1 Structure

Development list created from requests: There’s a long list of things that Patient Opinion have planned from requests. They tackle these week by week.

5. 2 Practices

Fairly ad hoc timescales. Patient Opinion don’t have any formal timescales, it’s essentially case by case as they feel this is more useful. There are different views on whether they should have more formality but have tended to think that they are still in learning mode; they will take it case-by-case until they know what’s required.

70-80% of posts are published within 48 hours: PO publish about 40% within 4

hour and a further 30-40% within 48 hours. Some of them therefore take longer as PO may need to clarify something or notify the subscriber.

Requests often go into a development stream: for fulfilling the request, depending

on what the request is it could be something PO could do immediately or if it’s a feature request, it goes into the development stream and done when that particular bit of development is happening. This stream of developments is “as long as your arm”

Procedures exist for learning from feedback: Patient opinion do have a formal

process for learning from feedback, although they actively work with all feedback. All emails suggesting features come to the internal technician, and he looks at them as well as errors or bugs too. The technician will usually reply to them and put any bug fix into the list and features into the feature list.

5.3 Performance

Backlogs do occur: Sometimes PO get a backlog, and can’t guarantee a timescale. Generally written and verbal feedback goes on site within 3-4 working days.

Software developer has left: software developer did help in doing the working on the development stream but he’s not with them now. He stayed for 6 months and has moved onto new work. As such it is expected that the rate that the site is improved will slow down.

12

6. Security measures

6.1 Structure

Have standard windows security in place: PO have got all the standard windows security in place. Passwords are encrypted, they use a salted hash, so they can’t read other passwords and a person needs a new password PO need to send them a new one as the old one is not in the PO database.

6.2 Practices

Patient Opinion don’t encrypt the database for anonymity. Strenuous updating of security and software: PO lock down all our software to the

maximum degree, update software, don’t expose services that aren’t required, switch off things that don’t need to be running. PO maintain the server for security. They have just moved to Window 2008 ; moving forwards constantly on security.

6.3 Performance

100s of hacking attempts per day: According to the technician “If you look at the server logs show hacking attempts, over a 24 hour period you’ll see 100s of hacking attempts.”

Never had any security breaches: Over the four years they’ve been up Patient

Opinion have had no security issues.

7. Profiling of users

7.1 Structure

Google analytics shows many details about users: Using Google analytics, Patient Opinion can see:

- the pages they visit most often

- web browser used

- connection speed

- time spent on the website

- when they visit

- average page views per user

Ability to profile by age, % of feedback which is published and deprivation index: Patient Opinion are able to profile their users by age group and gender using

13

the data left by posters. They can also see the postings (which are easier to analyse) which include reports, percentage of feedback which is posted on the website, and a deprivation index of the posters.

Hansard Society Survey: Hansard survey able to answer questions such as: Are you

a frequent user of the internet? What is your gender? How old are you? Where do you live?

7.2 Performance

After the home page, the Opinions page is visited most often: Page Title Page Views Avg. Time on Page Patient Opinion (home page) 10,646 00:00:56 Opinions | Patient Opinion 6,427 00:00:24 Search for a Hospital or Other 3,418 00:00:23 Service | Patient Opinion Patient Opinion | Review opinion 2,919 00:01:18 Patient Opinion | Pending postings 2,190 00:01:20 Share your story (step 1 of 4) | 1,669 00:02:16 Patient Opinion

Internet Explorer browser used by nearly 70% of users: Browser % visits Internet Explorer 69.85% Firefox 17.24% Others 11%

Average time on Site is 00:02:59 5.5 average page views per user 95% of Hansard survey respondents were frequent users of the internet?

Yes 95.0% 191 No 4.0% 8

14

63.5% of Hansard survey respondents were female

Female 63.5% 125 Male 35.5% 70

89.6% of Hansard survey respondents live in England (Hansard)

England 89.6% 180 Northern Ireland 0.0% 0 Scotland 5.0% 10 Wales 4.0% 8 Other (please specify)

1.5% 3

Gender (where known): 30.5% male 69.5% female (from post details on website) Majority of website users are over 60: (from post details on website) Age Group Female Male

Total

0-10 5(0.2%) 3(0.2%) 8(0.2%) 10-20 30(1.4%) 14(0.9%) 44(1.2%) 20-30 108(4.9%) 60(3.7%) 168(4.4%) 30-40 252(11.5%) 103(6.4%) 355(9.3%) 40-50 307(14%) 168(10.4%) 475(12.4%) 50-60 397(18.1%) 232(14.3) 629(16.5%) 60-70 411(18.7) 410(25.3%) 821(21.5%) 70-80 401(18.2%) 395(24.4%) 796(20.8%) 80-90 245(11.1%) 214(13.2%) 459(12%) 90-100 37(1.7%) 21(1.3%) 58(1.5%) 100- 6(0.2%) 2(0.1%) 8(0.2%) Total 2199(57.6%) 1622(42.4%) 3821(100%)

60+ females – 49.9% 60+ males – 64.3% 60+ overall – 56% Hansard had different age groups, only 47.2% over 55, 16.4% over 65

Under 18 0.5% 1 18-24 1.5% 3 25-34 10.0% 20 35-44 20.9% 42 45-54 19.9% 40 55-64 30.8% 62 Over 65 16.4% 33

15

8. Language capacities

8.1 Structure

Enabling Google translate: PO are currently enabling the option for Google translate at the moment so that if someone with a foreign language browser uses the site they’re given the option to see the posts in their language from Google translate.

8.2 Practices

Translating the site into Welsh, Italian Spanish and Catalan: Put a lot of work into translating site into different language – Welsh, Italian Spanish or Catalan. Also making other translators able to help PO translate the site. Site moving in that direction of being accessible in a whole range of languages.

A local community group helps translate posts where necessary: For users in the

UK PO will try to find a local community group to help translate

8.3 Performance

Patient Opinion can only handle postings in English: Postings come in English and PO make it clear that we can’t handle postings not in English.

Only one example of groups translating a post: PO have one example of a Chinese

woman who used a community group to translate her post and post it on the website

9. Reliability of the system

9.1 Practices

Adapting the system can produce errors for admin side: Produce some errors when PO do something more complex than they thought; that just affects the admin side, not the public facing website.

Technician handles any errors: He developed the site so handles the errors and

knows how it works. Possible to bring in someone for a serious crash: Could do external if something

serious happens.

16

9.2 Performance

Whole site has never crashed for 2 years. It used to crash under the old system but after PO re-wrote the site and re-launched it, it has not crashed.

Lose connectivity twice a year for short periods: Occasionally there’s some outage

in that part of the internet and PO will lose connectivity for around 20 minutes or so – this happens around 2 times a year.

10. Speed of network/internet

10.1 Structure

100mb/s connection Server is on the internet, so it’s the same as anybody else’s.

10.2 Performance

Patient Opinion claim to have the capacity to deal with huge surges in demand: When asked whether they have the capacity to deal with sudden high demand, they replied “Yes, occasionally we have had big surges. For example when we were on a TV programme last year we had a big surge. But we can easily handle a huge upsurge in traffic – can have 100 fold increase in traffic, probably closer to a 1000.”

Mystery shopping showed that during a Radio 4 appearance the website was

inaccessible: When a researcher tried to go on the website after an appearance on Radio 4 they couldn’t log onto the website. PO said they did not know why that was as they were able to use the site on that day.

11. Traffic

11.1 Practices

Patient Opinion do regularly measure traffic. They don’t utilise Google Adverts: PO have data from their google ads; Google Ads

works by organisations bidding for keywords (such as ‘patient’ or ‘opinion’) but the system is very complex. They currently aren’t fully utilising their Google ads as they don’t have time to tweak system constantly.

17

On the server logs, they have evidence of:

- Number of hits

- Hit rates

- Traffic to the site They can turn ad hoc queries to reports which are kept and can have access to.

11.2 Performance

The traffic is not rising very much: “At the moment we don’t think the traffic is rising very much. We’re looking at all kinds of ways to address this. We wouldn’t need a large increase to get a big rising”

Patient Opinion isn’t able to give their product away for free: At this point Patient

Opinion isn’t able to give their product away totally as, although traffic is increasing it isn’t yet high enough to give it away for free though they would like to. They want to have an impact on over 1 million people using – this is only possible by using the internet.

Currently 60%-70% of the website hits are from Google searches Number of hits: 19,974 page titles were viewed a total of 146,804 times in

February Hit rates: Average 5243 page views a day. Traffic to the site:

- 21,704 people visited this site in February

- 5.52 Average Page Views per user 8.03 Posts per day in February 2010. 106.83 visits per post in February 2010. Patient Opinion work on the assumption that there’s 1 posting per 1000 page

views as a rough estimate.

18

12. Interoperability

12.1 Structure

The application will work on other Windows servers. Data interoperability RSS and Atom as well as XLN API: Data interoperability is

more interesting which is the degree to which we share data outwards on the web – open data agenda. PO provide RSS and Atom feeds industry standards of interoperability as well as XLN API so that people can pull out any info they need. Patient Opinion are more interested in data interoperability.

Developed using packages (Adobe and MS) not open source: The site was

developed first in Cold Fusion, owned by Adobe and then developed in asp.net which again isn’t open source it’s MS technology. Could be redeveloped in open source technology if needed but the technician would need to learn more languages to do that and there’s no particular requirement for that.

12.2 Practices

Not reliant on the initial website developers: Patient Opinion were only reliant for the first two years, and haven’t been back to them since; they’re self sufficient.

13. Support for software/hardware

13.1 Structure Operating System by Asp.net: web architecture on MS Windows server and stored

in a SQL (Structured Query Language) server. Hardware quad processor: with the server room in London. They manage but don’t

own the server. Technician provides all support since software developer left: They don’t have

continuing support and development contract with an external web development organisation as the internal technician fulfils this function himself; he also does all content updates.

Basic office IT is outsourced: The office IT (such as printer problems) is done by an

external IT company who are paid £1000 a year. All other IT is done in house by themselves, whether it be pushing forward a certain idea, producing a web platform, or talking to the NHS.

19

13.2 Practices

Often use freeware which is easy to learn: Technician chooses the software himself,

often he needs something fast and so it needs to be easy to use initially. Most software will be free to use initially and later PO will subscribe. Some languages are more elegant than others and he tends to use those.

Database management uses Visual Studio 2008 and SQL Compare: Patient

Opinion use Visual Studio 2008 and SQL Compare for database management. They use GIT assembla for management which allows code depository – GIT keeps record of all changes that have ever been made. SQL Compare is only used by Technician; usually he will spend one day each week on software development.

13.3 Performance Technician now quite overloaded: It is a lot of work for the internal technician as

IT and service needs are constantly changing and developing, which can be frustrating.

The team thinks that the internal technician is “very good”: Changes are made if

the change will benefit more than one subscriber. Currently risk over-reliance on the internal technician due to lack of adequate

replacement within the team

14. Performance indicators

14.1 Structure

No internal Service Level Agreements: PO don’t have timescales for response times or other issues.

They produce company-wide standards: PO have got performance indicators for

the site, company-wide, with some aims over the next two years on traffic and response levels. Aims for traffic increase, postings and increases. However these aren’t available for the evaluation.

20

15. Production of documentation

15.1 Structure

Dedicated blog page on website: There is a dedicated blog page on the main website, with a search function, an archive organised by month, a list of tags with the font size related to the commonness of use in the blog, categories of blog, and RSS feeds for other related blog sites such as MySociety. Blogs are produced on their main site rather than a blog site such as wordpress.com.

“Patient Opinion in the Press” page documents national press coverage: Also

within the website, though less visible, is a page on “Patient Opinion in the Press” within the “About Us” page.

Use software packages for leaflet production

15.2 Practices

Blogs are produced with reduced frequency: Produce blogs, increasingly less. In the first three months (May 2008-July 2008) there were 26 entries whilst in the last three months (Dec 2009-Feb 2010) there were 3 entries. Still have at least one entry per month.

Press articles are linked on the website routinely after publication.

15.3 Performance

Patient Opinion has produced 85 blog entries since May 2008. 33% of website visitors have visited the Patient Opinion blog: A significant

number of website visitors (33%) have visited the Patient Opinion blog. Partly due to the blogs, 42% of website users report that they have learnt new things through the website.

There are also 14 articles on the Patient Opinion in the Press page. 21% of respondents had found out about Patient Opinion through the media

coverage: These articles have great importance to Patient Opinion as 21% of respondents to the Hansard Survey of Patient Opinion reported that they had found out about Patient Opinion through the media coverage.