L’VIV SUMMARY

II. EXECUTIVE SUMMARY

This report is the outcome of an external quality assurance assessment of Faculty of Law at Lviv

National University conducted by a group of international and Ukrainian experts. The project

was undertaken by the FAIR Project under funding from the United States Agency for

International Development (USAID). The purpose of this project was twofold: first, to develop

and implement a methodology for external assessment of Ukrainian law faculties and second, to

conduct such an assessment to develop concrete recommendations for the improvement of legal

education at the Faculty of Law at Lviv National University specifically and other Ukrainian law

faculties in general.

The assessment team identified five key stakeholder groups that had to be involved in the

assessment: administrators and faculty leadership; teachers; students, graduates and legal

employers. The team employed a variety of assessment methods during the assessment process

including computer based surveys; individual interviews; focus group discussions; site and

classroom visits; and the review of relevant documents, examinations, student papers and

textbooks.

The criteria for quality assessment used as a part of this process were adapted from the European

Higher Education Quality Standards. They focused on seven key elements: Policy and

Procedures for Internal Quality Assurance; Approval, monitoring and periodic review of

programs and awards; Admission and Assessment of students; Quality assurance of Teaching

Staff; Curriculum, teaching methodologies, learning resources and student Support;

Administration and Information Systems; and Public information.

Under each of these seven elements certain desired optimal outcomes are listed that the law

faculty is compared to. Under each of these optimal outcomes are listed the key findings

regarding that area and the specific recommendations that are the result of these key findings.

This report has divided the recommendations into two groups, those that can be implemented by

the law faculty or University on their own and those that will require a change in law or action

by the Ministry of Education or other higher authorities.

The assessment team found many positive attributes to the law faculty and were impressed with

the commitment to quality. However, some areas were identified as needing further attention as

the faculty develops.

The detailed findings and recommendations should be consulted to get a full picture of the

report, but some areas raised by the report can be summarized here.

The law faculty should work to develop its own quality assurance framework on the

basis of quality assurance system which would be created by the Ministry of Education

and Science of Ukraine following the new Law on Higher Education.

The law faculty should implement a more effective evaluation system that involves

students.

The law faculty should engage in a strategic planning process that includes a review of

current programs.

The law faculty should better promote the strengths of the law faculty to prospective

students in order to attract the best students.

The method of student assessments should be reviewed and changed to ensure a more

relevant mix of testing of skills and knowledge in a fair manner.

The law faculty should engage legal practitioners more in the teaching and mentoring of

students at the University.

The law faculty curriculum should be revised to allow for the introduction of elective

courses to students.

Basic legal skills courses should be a mandatory part of the bachelor’s and master’s

degree studies. Legal skills development should be an expected and accepted part of the

faculty’s legal education.

The law faculty should develop the capacity of the law faculty to teach using modern

interactive teaching methods including those appropriate to legal education.

Legal textbooks and materials should be revised to serve as an effective resource for

faculty and students in teaching materials in an interactive method.

Students should be involved to a greater extent in the operations of the law faculty,

including student evaluations and in faculty governance. A less formal environment

should be created inside and outside the classroom between students and faculty.

The law faculty should take full advantage of modern communication methods such as

social media, the internet and web sites to increase communications internally and

externally.

The law faculty should create an external relations department to assist it in promoting

the law faculty and its students outside the University.

The assessment team would like to further acknowledge that the assessment has occurred against

the backdrop of significant political and legal changes in Ukraine that present both a challenge

and an opportunity for the law faculty in the near future.

CHERNIVTSI SUMMARY

I. EXECUTIVE SUMMARY

This report is the outcome of an external quality assurance assessment of Yuriy Fedkovych

University Faculty of Law conducted by a group of International and Ukrainian experts. The

project was undertaken by the FAIR Project under funding from the United States Agency for

International Development (USAID). The purpose of this project was twofold: first, to further

pilot and develop the USAID FAIR Justice Project methodology for external assessment of

Ukrainian law faculties, and secondly, to conduct an assessment and to develop concrete

recommendations for the improvement of legal education at Yuriy Fedkovych and other

Ukrainian law faculties.

The assessment used the Methodology for External On-Site Legal Education Quality

Assessment developed by FAIR Justice consultants in 2014. This methodology identifies five

key stakeholder groups that should be involved in the assessment: administrators and faculty

leadership; teachers; students, graduates and legal employers. In line with the 2014

methodology the team employed a variety of research methods during the assessment process,

including online surveys; individual interviews; focus group discussions; classroom

observations; and the review of relevant documents, examinations, student papers and

textbooks.

The fourteen criteria for quality assessment used as a part of this process are adapted from the

European Higher Education Quality Standards. They are grouped under seven key elements

that provide the framework for both key findings under that element, and specific

recommendations that emerge as a consequence of these findings, both for the faculty

specifically, and for the Ministry of Education (which administers the system of higher

education in Ukraine). The seven elements are: 1) Policy and Procedures for Internal Quality

Assurance; 2) Approval, monitoring and periodic review of programs and awards; 3)

Admission and Assessment of students; 4) Quality assurance of Teaching Staff; 5) Curriculum,

teaching methodologies, learning resources and student Support; 6) Administration and

Information Systems; and 7) Public information.

The assessment team found many positive attributes of the law faculty at Yuri Fedkovych in

Chernivtsi. They were impressed with the commitment to students and to improving the

quality of legal education offered to them that the administration and teachers demonstrated

throughout. However, some areas were identified as needing further attention as the faculty

develops. The detailed findings and recommendations are contained in the full report proper,

but key recommendations under each element are summarized here:

Element 1: Policy and Procedures for Internal Quality Assurance

The school should move to consider “Quality” an on-going process ensuring the

delivery of agreed high quality education standards, and “quality assurance” as the

means through which the school guarantees that the quality of the education it

provides is being maintained and improved. It should seek to develop a “quality

culture”; that is, the creation and continuous utilisation of meaningful internal

institutional quality assessment mechanisms.

In pursuit of this, the law school’s existing Concept should be a starting point for a

faculty-wide consultative and participatory process of self-assessment and strategic

planning, where the faculty’s overall mission, and its goals and objectives in pursuit of

that mission, can be developed further, formulated in concrete terms, with timeframes

for achievement. Institutional self-assessment can then evaluate the success of the

school in achieving its goals.

Devising an improved quality assurance process should be a central focus of the

participatory strategic planning process recommended. This should include clear

policies and procedures that ensure the continuous assessment of the programs and

awards on offer, the faculty’s teaching, as well as its examinations regime. It assess

effectiveness, track results over time, and use the results of self-assessments and

external assessments (such as this one) to revise and improve structures and processes,

curricula and teaching.

Element 2: Approval, Monitoring and Periodic Review of Programs and Awards

The Faculty should work out a clear, efficient and objective process for monitoring

and revising courses. Those instructions should come in line with the provisions of the

Law of Ukraine "On Higher Education" adopted on July 1, 2014 concerning the

autonomy of universities that includes the right of the university to set up its own

standards and forms of teaching. There should be positive and aspirational guidelines

for course excellence and an expectation that courses will be reviewed annually for

improvement and revisions with necessary collegial support.

Element 3: Admission and Assessment of Students

The faculty should modify its approach to examination, decreasing the reliance on oral

exams. Exams should fit the nature and objectives of the course. Anonymous written

exams work well in some courses, research and written papers in others, and oral

exams in still others. Exams should be designed to test a wide range of skills, in

particular knowledge, critical thinking and relevant legal procedure, not only to

demonstrate memorization skills. A greater role should be found for written exams

with a variety of problem and scenarios based exam questions, and extended essays

requiring independent research.

Element 4: Quality Assurance of Teaching Staff

Peer review and evaluation of teaching can be very helpful, but the current system

when only teachers assess each other’s work, and where this process seems in some

way ad hoc, is insufficient to ensure high quality teaching and objective feedback.

Instead, a bifurcated process may work well in this setting. For new teachers there

would be assigned a teaching mentor whose role is to observe preparation and

teaching on occasion, be available to answer any questions or assist with

improvement, and may critique as appropriate. However, this person is excluded

from the reporting and evaluation process. The evaluator(s) independently do the

teaching evaluation and recommendations.

All students should have the ability to anonymously use a standardized teacher/course

evaluation form at the end of every semester but before grades are issued. The forms

should be deposited with a neutral or independent holder until after grades are issued

and then provided to the faculty member for self-assessment and improvement. This

data can also be used for Chairs to evaluate teachers, as part of ongoing internal self-

assessment, and for administrative purposes.

Element 5: Curriculum, Teaching Methodologies, Learning Resources and Student

Support

Teaching and examination should emphasize more the understanding and application

of knowledge. For this to happen, not only must teaching and assessment

methodologies be updated, but the content and focus of courses should be re-

appraised. Teachers should be encouraged and supported to develop distinctive

content, methodologies and testing.

The law faculty should consider changing the class format that seems to be

predominant in current classroom teaching. The current format is very formal, with

limited potential for effective interaction or full use of critical thinking. Teaching

methods used should seek to be interactive, using case studies, role plays (including

moot cases), presentations and the socratic dialogue (among others), to develop key

transferable skills such as problem solving and analytical thinking among the students.

Emphasis should be on the potential of students to apply knowledge obtained in and

out of classes. Students should have ample opportunity to pursue independent research

outside of prescribed texts

To improve the development of practical skills and to help students gain experience

allowing them to make informed choices about their career path, the internship

program should be improved. The law school should consider including a wider range

of receiving institutions, including local government, courts, prosecutors, law firms,

and civil society groups. These institutions should be engaged in the design of the

internship program, which should allow for the receipt of credit. Internship or practical

credits should be a required aspect of the law school curriculum. Students should be

distributed for internship (practice) in smaller numbers spread throughout the

academic year.

The law school should seek to modify a schedule of classes which currently seems to

overload both students and teachers. Modern learning methodologies emphasize the

importance of independent study in developing student skills - too many classes leave

in insufficient time for independent study and reading. Teachers complained of too

many teaching hours in general, while there is sometimes a third “shift” – when the

classes end at 20.30 PM.

Element 6: Administration and Information Systems

The faculty, in particular the administration, should seek to cultivate a more

open and participatory environment, and ensure decision-making is transparent and the

logic and basis of decisions are understood. The more teachers and students can play a

role in school planning and decision making, the more they will respect and implement

the positive changes the faculty is looking to implement.

The Faculty should seek to develop its E-University on-line system further. This

system should include profiles for all students including details, grades, schedules, and

any other relevant information. It should be systematically updated. The responsibility

of updating the e-courses could be given to the chairs of the Faculty. This would bring

the law school in line with international best practices for information management.

Element 7: Public Information

The content of the website should be enriched to include all data on applications,

courses, students, as well as important information on employability statistics of law

graduates. All advertisements about upcoming events and extra curricular activities

should also be published to demonstrate a varied academic life within the school. In the

ideal scenario, the website would also be available in English to allow those from other

countries (and potential foreign students) learn about the school, and should be further

developed to become a one stop portal for all law school information, for current and

prospective students, teachers, and administrators.

The Faculty should consider setting up an official organization of its alumni. The

alumni could contribute to the life of the school through official fund raising for

projects, administrated by the Faculty. Such Projects could involve students and

teachers in legal practice, financing publications and projects of the teachers, and

assisting students in finding the jobs upon graduation.

ODESA SUMMARY

I. EXECUTIVE SUMMARY

This report is the outcome of an external quality assurance assessment of National University

Odesa Law Academy School of Advocacy conducted by a group of International and Ukrainian

experts. The project was undertaken by the New Justice Project under funding from the United

States Agency for International Development (USAID). The purpose of this project was twofold:

first, to further pilot and develop the USAID FAIR and New Justice Projects methodology for

external assessment of Ukrainian law schools, and secondly, to conduct an assessment and to

develop concrete recommendations for the improvement of legal education at National

University Odesa Law Academy and its School of Advocacy as well as other Ukrainian law

faculties.

The assessment used the Methodology for External On-Site Legal Education Quality Assessment

developed by FAIR Justice consultants in 2014, and updated in 2017 by New Justice consultants

in accordance with European Standards and Guidelines for Quality Assurance in the European

Higher Education Area (ESG) of 2015. This methodology identifies five key stakeholder groups

that should be involved in the assessment: administrators and faculty leadership; teachers;

students, graduates and legal employers. In line with the updated methodology the team

employed a variety of research methods during the assessment process, including online surveys;

individual interviews; focus group discussions; classroom observations; and the review of

relevant documents, examinations, student papers and textbooks.

The associated criteria for quality assessment used as a part of this process are adapted from the

revised European Higher Education Quality Standards. They are grouped under ten key elements

that provide the framework for both key findings under that element, and specific

recommendations that emerge as a consequence of these findings, both for the university and a

School of Advocacy specifically, and for the Ministry of Education (which administers the

system of higher education in Ukraine). The ten elements are: 1) Policy and Procedures for

Internal Quality Assurance; 2) Design and Approval of programs; 3) Student-centered Learning,

Teaching and Assessment; 4) Student Admission, Progression, Recognition and Certification; 5)

Teaching Staff; 6) Curriculum, Learning Resources and Student Support; 7) Information

Management and Administration; 8) Public Information; 9) On-going Monitoring and Periodic

Review of Programs; 10) Cyclical External Quality Assurance.

The assessment team was fascinated with the openness and detailed preparation of the

Administration, Faculty, Staff and Students of the NUOLA School of Advocacy as well as of the

University itself. The positive environment allowed the assessment team to see many

commendable attributes at NUOLA and to fully explore all subject areas of the assessment in

order to prepare a comprehensive report. They were especially impressed with the readiness of

faculty and University administration to improve the quality of legal education they provide.

This was evident from the fact, for example, that the School of Advocacy conducted its own

internal quality assurance assessment in November 2016, based on the Methodology of 2014,

and provided the team with report on the results of assessment. Data of such assessment were

also used to finalize this Report.

The detailed findings and recommendations are contained in the full report proper, but key

recommendations under each element are summarized here:

Element 1: Policy for Internal Quality Assurance

University structure should be constructed in the way to effectively exercise all the needed

functions to provide quality higher education and to avoid any duplication of various structural

elements.

Faculties and institutes should be given more powers to deal with their internal affairs by

amending properly the University Charter and establishing faculty councils in particular. Faculty

councils may be empowered with the rights to approve educational programmes, topics of

research, decide issues regarding methodology of teaching, organisational and tutoring issues.

To ensure that Division on Quality Legal Testing and Centre for Higher Education Quality

operate effectively and may put into effect tasks set for them – it is of crucial importance to

appoint independent and full-time staff for these divisions.

Legal clinic course should be introduce and studetns, who work at legal clinic as cnsultants have

to gain credits for this. Street Law component should also be developed and implemented by

legal clinic.

Professional ethics and human rights issues should be considered by the University

administration throughout its all policies and strategy – both, relating to administrating its staff

and institution as whole and as an orienting element of the curricula and educational syllabi non-

depending on the particular subject of the course or program.

The University shall encourage students to take a more active part in students’ self-government

and all processes of university activity, such as: quality of teaching and assessment methods,

curriculum and schedules, facilities and support etc. University and the faculty should ensure that

the students must be overwhelmingly aware of their rights and opportunities to take part in these

processes.

Under the Strategic Plan there should be a practical and specific operative plan with consequent

steps, deadlines, responsible persons and controls to implement the strategy.

University and a faculty should seek to and take consequent steps to establish solid and dynamic

alumni community. Alumni might also be involved into various processes of the institution

activities, such as: membership for special students scholarships and awards; practice and

internship bases etc.

Element 2: Design and Approval of Programs

Students should be more involved in the process of amending the curriculum and providing more

feedback on class development. Flexibility should be inherent in the process of curriculum

development especially in terms of developing the knowledge required for each subject area in

light of frequent changes in the law.

As a key part of the requirements of the program in the Faculty of Advocacy legal skills should

be a mandatory component. Legal skills including oral and written advocacy skills, legal

reasoning and analysis, negotiation, client interviewing and counselling skills and a

understanding of court process. This requirement is particularly important to the mission of the

department.

Legal skills should be incorporated into classes and identified as a key aspect of the curriculum.

As a key aspect of this inclusion students should be graded and receive credit on activities

involving development of legal skills. This includes appropriate credit for work in the legal clinic

and other experiential learning.

The curriculum and individual class syllabi should be reviewed to see where elements of legal

ethics and human rights can be included. This should be done in a comprehensive way with

recognition of what elements of ethics and human rights knowledge students should have on

graduation.

The system of selection of ‘specializations’ in the faculty should be changed to allow students a

substantial opportunity to choose individual classes as electives. The University should look into

how to make it easier for students to choose elective courses from other departments and receive

full credit within their only faculty.

Element 3: Student-centered Learning, Teaching and Assessment

Creating of an internal supportive program for encouraging active student participation in the

learning process needs the benefit of new ideas and concepts that flow from domestic and

international exchange opportunities. The demonstrated openness and creativeness of the

NUOLA leadership supports the possibility for creation of an Institute for Creative Legal

Education, focused on developing innovative teaching and active student participation in the

learning process. An Institute for Creative Learning should start for the first year or two as a

faculty division. NUOLA faculty who wish to participate would meet regularly to discuss, and

think-through and agree on creative teaching methodologies and courses, which would then be

demonstrated to the full faculty. It would explore what is being done at law schools around the

world and how that would interface with Ukrainian legal education. Once it was comfortable

with the process and benefits it could become an Institute which adds workshops for other law

faculties from around Ukraine and promotes publication of its work. An Institute like this would

put teaching at NUOLA on the cutting edge of the evolution in teaching that technology and the

internet’s access to ideas are creating and position NUOLA as a leader in Creative / Innovative

teaching for Ukraine. NUOLA could expand its legal clinical offerings or develop more student

assigned work through the practicum model.

Using concepts of student-centered learning, students could be given more independent study

opportunities that promote their critical thinking and support innovative course development by

allowing students to propose research topics for approval.

Practical ethics education requires students’ introduction to real-life and on-going attorney and

client ethical dilemmas by integration of the common and relevant ethical issues as they apply

within each subject matter. An important development for the school and its regional goals

would be focusing more learning on critical thinking, with refinement of practical and analytical

legal logic abilities.

Anonymous examination should be incorporate into student assessments. This will address a

fundamental flaw inherent in oral exams that knowledge of the student’s identity and external

issues or opinions can affect the oral assessment process and grade. Likewise, abolishing the

student monitor system and using common text messaging systems would promote a greater

sense of equal treatment based on course academic performance.

There is no interference with the issue of feedback between students and teachers when the

assessment is anonymous. Firstly, all the assessment criteria should be well explained

beforehand as described in issue 4.1. Secondly, anonymous assessment should be built in a way,

that each student has his or her coded indicator (letter, number, etc.). So, later there won’t be a

problem to define the student in order to provide feedback.

Practical oral communication skills are one important skill for the job of an attorney, however

there are other important skills such as critical thinking, application of facts to the law, and

written communication that should be tested as part of the educational process. Restricting most

exams to one method precludes an in-depth examination of these important skills.

Students should have the possibility to anonymously evaluate the courses and teachers, on a

regular and systematic basis, as it is extremely helpful to teachers continually seeking teaching

improvement, provides a regular and standardized method for students to participate in course

development and teacher improvement..

Appeals on improper student, staff or faculty conduct should center on the respect for privacy

while at the same time demonstrating transparency to promote trust in the process. Appeals on a

grade are much more difficult. NUOLA will benefit from establishing an inclusive committee

(faculty, staff, administration, students and alumni) to discuss these difficult matters and

determine what process and standards are best for the entire school community.

NUOLA should capitalize on the willingness among faculty, administration and students to

discuss, draft, adopt and implement a community comprehensive code of conduct or honor

system. A comprehensive code, recognizing that integrity and ethical conduct are universal in the

law school community with each group having an active role in enforcing and honoring the code

is the most successful and sustainable method.

Element 4: Student Admission, Progression, Recognition and Certification

The University should take care of providing transparent and meritocratic procedures of students

admission and progression at various levels of studying. Each program or course needs to have

clear and well-explained criteria of admission, if relevant. But what is even more important

- these should include a description of the students progress in this particular program or course

- i.e. pre-defined and reasonable grading system, assignments types illustrations etc.These

procedures must be communicated directly to students and fairly beforehand, so that a student

knows exactly what he or she is expected to do, to know and to demonstrate when being assessed

in order to enter and complete particular course or program. Such requirements and criteria

should be developed based on the purpose and goals of the discipline and strongly correspond to

them.

Where possible, the University should define and consolidate reasonable criteria for students

recognition, considering the provisions of the Lisbon Recognition Convention and explanatory

documents related to it.

When designing the assessment (evaluation) form – either for particular course, or any other

assessment or monitoring – one should consider and identify the purpose of such testing first and

then choose the method which best fits to achieve it.

The University and a faculty should encourage participation of students in various law students

competitions by supporting them academically, financially where possible. Also the University

should consider the possibility of giving credits for some participants for brilliant performances.

Element 5: Teaching Staff

The University should develop a program for all faculties on modern interactive teaching

methods. This program should include: training in modern interactive teaching methods for all

faculty; expectation of the University and the Department that these methods should be used in

all classes; support for faculty in developing these skills; requirement that new faculty undergo

appropriate training when hired; and inclusion of teaching method use in faculty evaluation.

As resources become available projectors and computer equipment should be made available in

all lecture rooms in the department.

Additional opportunities should be made available for International Teaching and Research

opportunities for faculty, more language training made available, and faculty should be

encouraged to participate in these programs. Where possible, additional University resources

should be made available to facilitate International experiences for faculty.

The University and the Faculty should discuss the creation of additional opportunities for faculty

to be involved in chair, department and University governance.

Element 6: Curriculum, Learning Resources and Student Support

NUOLA needs to encourage and promote diverse teaching methodologies in teaching practices

(not necessarily technology) that respect diverse talents and ways of learning. NUOLA has a

strong library and library staff, a talented IT department, strong faculty and very bright students.

The pieces are all there for the creation of very innovative, effective learning in each classroom.

Understanding legal theory is essential but equally essential is how that theory interfaces and

instructs the application of laws to society and the effective role of lawyers. Hard skills such as

learning laws, research, oral advocacy, writing advocacy, including proper use of forms are

among the hard skills that NUOLA can enhance or incorporate into its existing curriculum and

course objectives, at the same time soft skills are growing in importance.

The job fairs should be expanded, as there was meaningful enthusiasm among the students,

alumni and Council of Advocates. NOULA should consider programs of regular on-campus

interviews and additional job fairs perhaps focused on special practice areas if the demand is

determined.

NUOLA needs to develop better resources to support faculty/student development, which in turn

promotes teaching development and better student engagement, which in turn will attract better

students and improve faculty satisfaction.

Element 7: Information Management and Administration

Courses taught in foreign languages should be included into regular curriculum and students

shall receive credits for completion of such courses along with those which are taught in

Ukrainian or Russian

University and the Faculty should involve and encourage all stakeholders of the educational

process to actively use LMSs (such as Moodle, EduBrite etc.) for various processes of study

process, including but not limited to communication and contacting between students and

teachers staff, publishing syllabi and submitting written tasks etc.

Programs (syllabi) should be comprehensive, well-structured and correspond to actual teaching

process. Volume of supporting materials needed for particular class must be realistic for a

student to deal with.

Element 8: Public Information

University web-site should be organized in the way that all important documents were well

grouped and easily accessible.

Of vital importance is to develop, publish and duly update clear and comprehensible description

of the university programs and courses, what will include at least the following elements:

curricula and syllabi detailed and specific information about educational content and plan of

studying, methods of teaching, admission and assessment criteria, credit amounts, expected

outcomes.

Element 9: On-going Monitoring and Periodic Review of Programs

The University should explore additional mechanisms that would allow peer review and program

development on the Faculty level in a way that allows all the chairs of the faculty of Advocacy to

act together in programming input and design.

Methods to increase student involvement in program review and development should be

implemented by the University, especially on the Chair and Faculty level consistent with and in

partnership with increased peer and faculty member involvement in the process.

Element 10: Cyclical External Quality Assurance

The University should continue to engage in external assessments as a tool to enhance University

advancement. In addition, the University should develop mechanisms to implement and evaluate

any recommendations as a result of this or other external assessments.

KHARKIV SUMMARY

II. EXECUTIVE SUMMARY OF THE ASSESSMENT RESULTS

This report is the outcome of an external quality assurance assessment of Yaroslav Mudryi

National Law University Personnel Training Institute for the Bodies of Justice of Ukraine

(hereinafter – Institute of Justice) conducted by a group of International and Ukrainian experts.

The project was undertaken by the New Justice Program under funding from the United States

Agency for International Development (USAID) and its purpose was to conduct an assessment

and to develop concrete recommendations for, if appropriate, the improvement of legal education

at Yaroslav Mudryi National Law University Institute of Justice as well to spread and consider

them in the national legal education system at whole.

The assessment was conducted according to the Methodology for External On-Site Legal

Education Quality Assessment developed by FAIR Justice consultants in 2014, and updated in

2017 in accordance with European Standards and Guidelines for Quality Assurance in the

European Higher Education Area (ESG) of 2015.

This methodology identifies five key stakeholder groups that should be involved in the

assessment: administrators and faculty leadership; teachers; students, graduates and legal

employers. In line with the updated methodology a variety of research methods was employed

during the assessment process, including online surveys; individual interviews; focus group

discussions; classroom observations; and the review of relevant documents, examinations,

student papers and textbooks.

The associated criteria for quality assessment are grouped under ten key elements that provide

the framework for both key findings under that element, and specific and applicable

recommendations that emerge as a consequence of these findings, both for the university and the

Institute of Justice specifically, and for the Ministry of Education, which administers the system

of higher education in Ukraine. The ten elements are: 1) Policy and Procedures for Internal

Quality Assurance; 2) Design and Approval of Programs; 3) Student-centered Learning,

Teaching and Assessment; 4) Student Admission, Progression, Recognition and Certification; 5)

Teaching Staff; 6) Curriculum, Learning Resources and Student Support; 7) Information

Management and Administration; 8) Public Information; 9) On-going Monitoring and Periodic

Review of Programs; 10) Cyclical External Quality Assurance.

The assessment team is pleased to admit the openness and thorough preparation of the Yaroslav

Mudryi National Law University and Institute of Justice Administration, Faculty, Staff and

Students. This was evident from the fact, for example, that the Institute of Justice conducted its

own internal quality assurance assessment in September 2017 and provided the team with report

on the results of assessment. Data from that self-assessment were also used to finalize this

Report.

The positive environment allowed to see many commendable attributes at Yaroslav Mudryi Law

School and to fully explore all subject areas of the assessment in order to prepare a

comprehensive report. Experts were especially impressed with the readiness of faculty and

University administration to improve the quality of legal education they provide, their interest in

innovative teaching and learning methods and desire to implement such methods. Experts have

witnessed the continuing efforts of the faculty and administration to work in many areas the

report mentions. Some recommendations from this report have already been set by the University

as goals for the next several years.

The detailed findings and recommendations are contained in the full report proper. Key

recommendations under each element are summarized below:

Element 1: Policy for Internal Quality Assurance

The University has to develop its own sufficiently detailed quality assurance framework ensuring

continuous assessment and improvement of the faculty’s teaching, examinations and programs

updating. The University should establish a special structural division, which would deal with

policies and procedures for the assurance of quality in legal education.

Although a training-methodological department temporarily manages the quality of education,

this new structural and University staff division should have independent status with real powers,

a separate location of suitable size to exercise its functions and trainings, and salaried positions

for the staff. Its staff should not be subordinated to any chair or institute (faculty) but promote

teaching excellence University wide and later Ukrainian wide. A special instrument to regulate

the functioning of the body has to be developed in a horizontal way by involving the widest

range of stakeholders. It is especially important that the staff of this newly established body

participate in drafting the instrument governing its functioning with input from Administration,

Faculty and Students. The ‘good start’ of the functioning of this body is important for its future

autonomy and growth.

Internal procedures for the assurance of education quality as a starting point should use methods

and techniques employed by experts in the current assessment. It is crucial to have the

standardized, anonymous and constituently given evaluation process. The methods used should

be both quantitative and qualitative in nature. Quantitative data have to include information

about grades, number of students willing to live in dormitories and actually living there, numbers

and employment data of graduates etc. Qualitative data have to be based on written feedback and

interviews with students, graduates, teachers and administration. The newly established division

should regularly monitor whether a number/percentage of elective courses actually taught meets

the legislative requirements and students preferences.

Students should be provided with easily enforceable opportunities to change their career choice

in their second or third year of bachelor studies. Every five years the University should review its

structure to make it transparent, coherent and fair. The University should consider creating a new

chair of EU law and one for Professional Responsibility/Anti-Corruption.

Course evaluations need to be carried out regularly in each course each semester and internal

surveys on a systematic basis, based on official regulations and developed in a way to consider

best practices and stakeholders’ opinions. Anonymity must be assured for those answering

surveys and filling out forms. Electronic means are preferred in gathering information from

surveys and course evaluations. The evaluation forms need to be deposited with a neutral,

secured folder until the grades are issued and afterwards provided to the respective staff member

for evaluation and self-improvement.

The University Academic Council should review its powers as provided by the Law of Ukraine

‘On Higher Education’, the University Charter and its Regulation, and delegate some of them,

where appropriate, to the Institute’s Academic Council. Delegated powers should be sufficiently

specific, clear and well communicated to the Institute’s management, chairs and students. A

review University Academic Council powers should be carried out every five years. The changes

should actually enhance use of institute authority, not just a structural change to meet a

requirement.

The University should consider creating an Honorary or Distinguished Advisory Board in the

University consisting of respected alumni and practitioners. It should be flexible in operation and

having no controlling powers over the University. Work in the Board should be volunteer,

treated in an honorable manner and recommendations given due consideration. The Advisory

Board may suggest bringing important improvements in curriculum or in the educational process

in general from the practical point of view. The selection of Advisory Board members should be,

first of all, based on merit, transparent and fair, and also take into account international best

practices in this sphere1. The University should establish what the objective or the purpose of the

Board is. It is not sufficient that the functions of the Board are performed by the University

Alumni Association or educational services customers.

The University has already developed its own Code of Academic Ethics, still to be adopted. The

efforts of the University community should be commended for this achievement. The draft Code

1 To find out more about Advisory Board operation and tasks, please see: New Justice Report: Law School External Advisory Boards. Importance, Implementation and Best Practices, 2017.

of Academic Ethics should provide for more detailed procedural rules on violations of academic

ethics, namely it should describe how the Executive Committee is formed and how it deals with

ethical violations. To ensure this system is respected and enforced properly in the University,

students must be involved in the drafting process as much as possible and in the subsequent

enforcement and maintenance. PhD students should be asked for the opinion about how the

system of honor/academic integrity should look like on an equal footing with bachelor and

master students. While the draft Code of Academic Ethics is focused on ethical rules and

enforcement, the University should consider supplementing the draft Code with a document of

principles and values which focus on the ideals and aspiration of an ethical university

community. Although the University staff has voted to develop and implement the Code of

Academic Ethics by 2019, meaningful student involvement is important for the process.

The goals in the University Strategic Plan 2016-2020 as described should be implemented as

soon as possible. On-going changes to the Strategic plan should be easily adoptable so that this

instrument functions as a living, changeable document rather than a ridged program, which is not

subject to any modification. When modifying the existing plan, or adopting the new one, the

University should listen to all the stakeholders and build commitments to ensure the Strategic

plan gets successfully implemented.

Both the 5-year Strategic plan and the implementation Plan for 2017 should become more

specific in their goals and schedule of implementation. The annual implementation plans should

indicate different representatives will come from administration, chairs, teachers and students, as

persons responsible for the implementation, executives or stakeholders whose opinion should be

considered.

The University and Institute should foster cooperation with graduates through the alumni

association by inviting graduates to participate in the appointment of special scholarships and

awards, give lectures and conduct practical training, provide legal expertise and their suggestions

on the improvement of educational process. The alumni association should consider creating

specialized sub-groups for alumni of certain legal professions (i.e. Prosecutors division, Lawyers

division, Judges division) in order to develop alumni programming in specific areas.

Distinguished alumni should be considered for invitation to the proposed Board of Advisors. The

practice of students’ internships at the graduates’ law offices should be restored and enhanced

with collaboration with the alumni association.

Element 2: Design and Approval of Programs

Students should be more involved in the process of recommending amendments/supplements to

the curriculum and providing more feedback on class development. This should happen at the

Chair and Faculty/Institute level and should be consistent with greater involvement of faculty

members on these levels as indicated in Element 5.

Flexibility should be inherent in the process of curriculum development especially in terms of

developing the knowledge required for each subject area in light of frequent changes in the law.

As a key part of the requirements of the program in the Institute of Justice legal skills should be a

mandatory component of all courses which should be incorporated into the curriculum and into

each individual course as appropriate. Legal skills including oral and written advocacy skills,

legal reasoning and analysis, negotiation, critical thinking and a practical understanding of court

process, etc.

Legal skills training should be incorporated into classes and identified as a key aspect of the

curriculum. As a key aspect of this inclusion, students should be graded and receive credit on

activities involving development of legal skills. This includes appropriate credit for work in the

legal clinic and other experiential learning. Critical thinking is an essential component of all

legal skills classes.

The University should consider allowing students to choose an institute or faculty and therefore a

specialization toward the end of their second year of bachelor studies. The first two years could

be organized as a common set of core courses essential for all first and second year bachelor

students, organized by faculty designed for these two initial years, and then allow students to

choose the institute/faculty they would like to continue in after core learning. This would allow

students to make a more informed choice of career paths.

The institutes/faculties would also be able to create a more focused program if they are dealing

with upper year students who are interested in a particular specialization instead of focusing on

general education in the first and second years.

The curriculum and individual class syllabi should be reviewed to see where elements of ethics,

legal ethics and human rights philosophy should be included. This should be done in a

comprehensive way with recognition of what elements of ethics and human rights knowledge

students should have in the University community and after graduation in the legal profession

and society at large.

The curriculum should include a mandatory legal professional ethics course in all

institutes/faculties besides deontological training. In upper classes and specializations courses

this can be added to address in more detail the professional ethical legal obligations of particular

legal professions depending on the institute/faculty specialization.

Case law of the European Court of Human Rights should be incorporated in courses taught to

both bachelor and master students. It is not only courses on human rights that have to include

ECHR case law, but also courses on civil law, civil procedure, criminal law, criminal procedure,

administrative law and administrative procedure, etc. Lecturers should consider incorporating a

human rights component in all courses.

The University should reach the requirement of 25% elective courses as soon as possible.

Students should be truly allowed to select individual elective courses by themselves, especially

in the third and fourth years of study. The institutes/departments and class monitors should not

step into this process or make this decision for the students or incline students to take certain

course.

The University should consider a University wide elective system by designating certain class

times during the week when only electives are offered by each chair and allowing the students in

each institute/faculty to freely choose those electives that they are interested in. No

institute/faculty could schedule mandatory courses during these elective periods so that students

would not have a conflict between mandatory and elective courses. In addition, students of the

3rd and 4th years of bachelor students and master students could all take these courses as electives

from any institute/faculty leading to greater educational diversity for the students and faculty.

Element 3: Student-centered Learning, Teaching and Assessment

The University should take care to encourage students at active participation in the learning

process – both, while drafting, designing and approving the programs, as well as during the

educational courses. To do so, the University might use anonymous surveys among students,

teachers, alumni and legal practitioners, public discussion of the draft programs, joint meetings

with the students and their self-governmental bodies to collect their opinions about the current

programs and the preferable changes, and additionally – the evaluation of all the courses and

faculty members, provided by the students.

The University might also consider establishing an Institute for Creative Legal Education,

focused on developing innovative teaching and active student participation in the learning

process.

The University should not underestimate the importance of the legal ethical issue when

designing the whole educational process.

To make students more familiar with the legal professional realities, the University should invite

on a systematic basis practicing lawyers from different law fields, including alumni, to

demonstrate various ethical dilemmas that arise in legal practical environment and discuss

possible solutions to these dilemmas with the students.

The University has to ensure that its internal policies are developed in a way considering the

diverse students needs and expectations, including the flexibility of the schedule, effective

communication between a teacher and a student, considering the students individualities and

personal abilities.

Methods used for teaching should also be diverse, multifunctional and aimed at training different

skills, depending on the educational goals and legal market demands.

The exams should not be regarded as a means of teaching, but as a method of the knowledge

evaluation. Thus the University has to incorporate the exam and testing system that would meet

the requirements of objective and independent evaluation.

Electronic learning management system (ELMS) must operate properly and be a platform for the

comprehensive information exchange during the learning process between university

administration, teachers and students. The University should ensure that the staff and students

are well aware of its options and operation.

The University should have policies that exclude the possibility of prioritizing particular students

over others. One of the possible steps in this direction is to quit the ‘students monitor’ system.

The University should ensure that not only the assessment criteria, regulations and procedures

are established, properly published and applied fairly and consistently.

Anonymous testing promotes for more objective and independent assessment. Automatic testing

may also save lots of the teachers’ workload and the teachers’ efforts may be used for more

productive types of teaching activity – such as, arranging and completing a group project with

the students, organizing debate or any other.

The University and Institute should implement mechanisms that would stimulate active

involvement of students with different learning process into educational activities.

The University should implement as quickly as feasible a written evaluation for each course in

each semester. These evaluations be structured in such a way as to allow full, anonymous student

evaluation of each course and faculty member on a variety of issues involved course content,

teaching methods and evaluation. This system should be established following best practices and

with feedback from faculty and students. Establishment of such a system should also involve

details on who the evaluations will be used in the review of curriculum and faculty work.

The University should ensure effective and transparent procedures for the complaints that relate

to person’s behavior and actions. Such procedures may base on the University Code of Honor

and involve people from different stakeholders groups.

The procedures for grade appeals (particularly, outlined in the paragraph 4.12 of the University

Regulation on the Organization of the Educational Process) should be trustworthy,

understandable and effective and provide for more guarantees for students and procedural

safeguards.

Code of Honor adoption must be transparent at all stages and open to all the interested parties, as

the Code of Honor relates to absolutely every person who takes part in the educational process.

There should be technical facility and sufficient time for everyone desiring submit to his or her

own suggestions – both, at the drafting and finalization of the document.

As to the Code of Honor content – it must, first of all be very well balanced and non-

discriminatory. University has to be very clear as to what provisions of this future document are

enforceable as such and – thus, may be set down as mandatory rules of behavior, and what

should be declared as desired norms of conduct.

Element 4: Student Admission, Progression, Recognition and Certification

The University should take care of providing transparent and meritocratic procedures of student

admission and progression at various levels of studying. These procedures must be

communicated directly and fairly to students beforehand, so that a student knows exactly what he

or she is expected to do, to know and to demonstrate when being assessed in order to enter and

complete a particular course or program. Such requirements and criteria should be developed

based on the purpose and goals of the discipline and strongly correspond to them.

Where possible, the University should define and consolidate reasonable criteria for student’s

progression, recognition and certification, considering the provisions of the Lisbon Recognition

Convention2 and explanatory documents related to it.

Consider, if possible, the procedure and format of re-taking exams, or just as a part of ordinary

procedure to have few trials to take an exam, in order for a student who wants to get a better

grade, not just ‘fail’ or ‘pass’. This might be pretty easy if the testing is digitized (e.x., through

Moodle or EduBrite) and checked automatically.

2 The Lisbon Recognition Convention, 1997. - [E-resource]. – Regime of access:

https://www.coe.int/t/dg4/highereducation/recognition/lrc_EN.asp

When designing the assessment (evaluation) form – either for particular course, or any other

assessment or monitoring – one should consider and identify the purpose of such testing first and

then choose the method which best fits to achieve it. A combination of various testing methods

shall be introduced and used as appropriate.

Participation in experiential learning is an important method of legal education and a way for

effective student education. This should be encouraged by the university and awarded with

academic credits, for example – as an alternative to writing course academic papers.

Students should be given a choice – either to conduct scientific research and defend a paper, or –

for example – to take part in law students competition and, in a case of good performance – be

awarded within the same grade rate as for an academic paper. For instance: writing a memorial

for the Jessup Competition, is often much more complicated and knowledge-worthy than writing

an ordinary paper. Moreover, as the whole Competition is in English, such participation brings

added value on top of developing knowledge in international law itself, as well as other language

professional skills.

Awarding credits for experiential education also reinforces the importance of this form of

education and recognizes that it is an important part of the curriculum of the University and the

Institute.

Element 5: Teaching Staff

The University should continue promoting academic initiatives from the staff. For instance, the

University may allocate funds to create a special award for the influential academic research, based

on citations or other factors, or for a member of staff who has taken the lead on moot court or other

experiential education. These awards may be given to a certain academician based on voting

through secret ballot among the University staff or by the leadership of the University.

The University should recognize that faculty development should occur not only in the area of

their personal academic achievements but also in the faculty members’ ability to teach, mentor and

develop students. Professional development should include pedagogical skills as well as

substantive areas of legal knowledge.

The University should consider recruiting lecturing staff without PhD degrees, but with

sufficient practical experience and skills and creating a “Professor of Practice” system in the

University.

The faculty must admit that such teaching methods, as moot courts or legal clinics are just as valid

as scientific method of legal education, traditional research and writing and should ensure that

faculty members who work in these so called “practice” methods are recognized for their work

and that such work is encouraged as a legitimate form of legal education at the university.

The University should develop a program for all faculty on modern interactive teaching methods.

This program should include: training in modern interactive teaching methods for all faculty;

expectation of the University and the Institute that these methods should be used in all classes;

support for faculty in developing these skills; requirement that new faculty undergo appropriate

training when hired; and inclusion of teaching method use in faculty evaluation.

It is crucially important that - as resources become available - projectors and computer equipment

should be made available in all class rooms in the University.

Additional opportunities should be made available for International Teaching and Research

opportunities for faculty, more language training made available, and faculty should be encouraged

to participate in these programs. Where possible additional University resources should be

accumulated and support the facilitation International experiences for faculty.

The University should ensure additional opportunities for faculty to be involved in chair,

institute/faculty and University governance structures.

Element 6: Curriculum, Learning Resources and Student Support

Taking into account the specificity of the students audience, teaching methodologies should

combine both: andragogical and pedagogical approaches, techniques and principles, i.e.

listening, studying and doing.

The University should accumulate the knowledge of the University students and teachers, who

have already gained some external experience of learning and teaching, properly adopt and

incorporate new or alternative approaches, based on its internal potential.

The University should increase the number of staff members working at the legal clinic. The

University should also allocate additional premises for the operation of the legal clinic closer to

the central building of the University. More attention in the legal clinic should be paid to legal

training of students and stronger supervision policy. A special course on legal clinic, negotiation

and other practical legal skills should be introduced in the curriculum and students should

receive credits for their work in the clinic. The University should also consider creating several

legal clinics or several departments within the existing clinic possibly attributed to different

institutes (faculties) and specializing on different aspects of law, for example, family law clinic,

tax law clinic, labor law clinic, etc.

More (or at least some) attention should be given to soft skills training, including critical

thinking, listening and legal writing, legal reasoning, document drafting, self-presentation, time

management, general and business administration and project management, emotional

intelligence, team work, correspondence and others.

Students should have sufficient and convenient access to library resources and thus the library

should be open at least till 8 pm Monday to Saturday. To implement this – a two-shift work may

introduces, for instance, from 8-30 am till 2-30 pm and from 2-30 pm till 8-30 pm.

The institution should consider establishment of a special division that would be responsible only

for the organization and mentorship of students practice and invent other effective mechanisms

and procedures which would secure that students gain the productive practice. The University

should keep a contact with the entity where the practice is provided and monitor the quality of

such practice.

It is important that the University considered the adjustment of the curricula in a way, that

students could effectively learn the corresponding foreign languages. It is essential, that foreign

language study promotes learning of legal terminology and vocabulary, like Legal English

(English for Lawyers), Legales Deutsch, Français Juridique, etc. English language courses

should be expanded from one to a minimum of two years of study in the bachelor program. If

feasible advanced Professional theme English language courses should be available to students

for a third year and should be available to Master’s students.

Include into the curriculum substantive courses that are taught in a foreign language. These

courses could be taught to upper division bachelor and master’s students. Foreign language

course should be fully credited as a part of the curriculum of studies.

Element 7: Information Management and Administration

The University educational system ELMS (http://acs.nlu.edu.ua/) should function properly in

order to have the necessary information about study programs easily collected, analyzed and

used.

The University should review how it collects and uses information in a systematic way that

would be helpful for evaluation of programs and development of innovative approaches.

The University and the Faculty should take steps in order to strengthen and promote the students

councils; competences and procedures of student self-government bodies should be properly

communicated to students on a regular basis. Steps should be taken to enhance the involvement

and role of student councils at the institute/faculty level where feasible.

Student council(s) meetings should be well publicized in advance and their functions should be

available to all students with open and transparent meetings.

Direct communication via electronic (computer) means makes the process of communication

between the stakeholders of the educational process much more effective and faster. It saves

time, reduces the possibility of miscommunication through “other’s mouths”, and eliminates

possible bias or favoritism in the use of the monitor system.

The University and the institute should first ensure the technical element of any of the electronic

communication functioning. Afterwards – administration and teaching staff, as well as students

should be instructed on how and to what extent, for what purposes they may use it. Actually,

electronic communication allows a plethora of constructive functions, including, but not limited

to setting the syllabi, tasks for students; arranging them into different groups for various projects

and activities within study programs; conduct of assessments and evaluations, surveys and much

more.

In light of our recommendations involving a more truly elective class system and

recommendations on foreign language classes the University should reconsider its system for

timetabling classes to accommodate reasonable changes so as to create opportunities for students

to study outside their institutes/faculties and beyond their particular group of students. These

timetabling elements should be done in a way that allows students to choose these alternatives at

times when mandatory classes are not being held so as to avoid a direct conflict.

Element 8: Public Information

The University should put more public focus on regular bachelor and master programs, namely

indicate mandatory courses, courses within specializations and electives. This would enhance the

current showing of content of joint degree master programs and certificate courses.

The University should ensure that the ELMS is used to the maximum of its capabilities and

functions properly, as clarified in the previous Element 7.

All the materials of the alumni association should be digitalized and put on-line.

The University web-site rather than the web-sites about entrance campaign, repository and

ELMS needs to show the list of courses taught in bachelor and master programs in a transparent

manner. It should be clear which department (chair) gives which course, at which institute

(faculty) and year, the audience of the course and the learning outcomes. This should be done in

a way to ensure that prospective students are well informed when making their conscious choice

on their future education, profession and specialization. Information about programs and courses

should be sufficiently detailed so as to include information about the content of the course, skills

to be acquired, the number of lectures (seminars) and the form of assessment (exam or else).

Now one may find only some partial information about bachelor and master programs on the

University web-site, and receive general understanding of the education without the specifics.

Although internal regulations, namely the regulation on the Organization of Educational Process

and the regulation on the Monitoring of Students’ Knowledge, are available on the University

web-site, it would be more user-friendly to additionally put simplified information about the

assessment criteria and procedures in a separate section of the University web-site.

The web-site should contain information on the employment of graduates and employment

possibilities. More focus on the University web-site has to be put on the quantitative data,

including statistical information about students’ achievements, pass rates and number of

expulsed students.

The University should continue to make all the papers and dissertations readily available and

ensure that they are not removed from the web-site, even three months after the public defense.

Element 9: On-going Monitoring and Periodic Review of Programs

Procedures for the programs monitoring should be flexible enough and responsive to the

contemporary changes and demands of the legal market and global higher education tendencies,

including, but not limited to, students academic mobility.

The process of the Programs designing and amending should involve all the interested parties,

i.e. – the University, chairs and departments administrators, faculties and students and also the

legal practitioners.

The University should consider procedures that would allow peer review and programs

developing with all the stakeholders, including students, on a partnering basis.

Element 10: Cyclical External Quality Assurance

The University should continue to engage in external assessments and maintain mechanisms and

procedures to effectively evaluate and implement the recommendations coming from such

assessments.

The University might apply the mechanisms for internal assessment at different levels – a chair,

department and university as a whole and for different purposes, like investigating a particular

topic (e.x., academic freedom, evaluation system, curriculum development, etc.).

Establishing advisory boards, as discussed in the subsection 1.2 will also greatly contribute for

the external and internal quality assurance efficiency.