Journal of FamiCy Therapy (1989) 11: 297-306

Short Paper Section

The reliability of video viewing of family therapy interviews*

Eddy Street? and Hugh Foot:

The use of video-taped material is common in family therapy practice and training. It is assumed that the video material conveys the same impression as when the interaction is viewed ‘live’. In this study the same eight interviews were viewed by two groups ofexperienced therapists, one group ‘in vivo’ and the other ‘via video’. The groups were required to rank concepts derived from Structural Family Therapy as relevant to each of the interviews. O n six ofthe eight interviews there was substantial agreement in the ranking of concepts by the two groups lending general support to the assumption that judgements offamily interactions are not influenced by the medium through which they are observed.

Introduction

Family therapists make frequent use of video, and video material is widely employed for both training and demonstration purposes. In such circumstances it is invariably assumed that the video viewer observes the same events that confront the therapist dealing face-to-face with the family. The typical use of‘ video material is illustrated by Tomm and Leahy’s (1980) study of different types of family therapy training for medical students; in one condition students received the traditional lecture approach which made use of video examples for demonstration purposes. The second condition had groups of students interviewing

* Accepted vcrsion received May 1988. t Clinical Psychologist, Preswylfa Child and Family Centre, Clive Road, Cardiff

Reader in Psychology, School of Psychology, P.O. Box 901, University of Wales CF5 IGN.

College of Cardiff, Cardiff CFI 3YG. 297

0163-4445/89/030297 + 10 $03.00 0 1989TheAssociationforFamilyTherapy

298 E. Street and H. Foot

families and using videos of the interviews for supervision and discussion. In the comparisons made between these conditions it was clearly assumed that there is direct translation from the interpretation of the interaction observed ‘in vivo’ to that observed ‘via video’.

A typical feature of family therapy practice involves the use of supervision through a one-way screen (Montalvo, 1973). This type of work rests on the awareness that important elements of atmosphere, ambience and even interaction are often interpreted differentially by the therapists working in the room with a family and by colleagues observing through the screen (Roberts, 1987). No doubt the therapists conducting the interviews are under additional pressure of involvement and self- presentation which do not impinge directly upon their observing colleagues even though they may all be involved in corporate decision- making about the course of therapy (Speed et al., 1982). If such differences are found when therapists observe the same ‘live’ interview at the same time, it is equally plausible that differences will be found between observing an interview ‘live’ and at a later time on video. The observational context is clearly different, especially in terms of its ‘therapeutic urgency’. In live situations where the therapists have to make immediate decisions about the course ofthe interaction or the type ofintervention to initiate, their perception is as much focused upon their own control of the situation and their own performance as upon the nature of the dysfunctional interactions amongst family members. Observation ofa videotape ofthe interview releases the observer from the immediacy of the clinical situation and is likely to focus attention upon the interactionper se and away from what the therapist is or is not doing.

Clearly, the assumption that the therapist and the video-viewer watch the ‘same’ event is one that requires empirical testing. If differences in observers’ perceptions to live and video-recorded interviews are demonstrable, then this may have important implications for the use of video in the training of family therapists. This small-scale study involving families presenting with various dysfunctional problems was designed to compare judgements amongst trained family therapists of live and video-recorded family therapy interviews.

Method

Collection of livejudgements

A Clinical Team working within the structural family therapy model (Minuchin, 1974) was recruited. It was composed of four individuals: two men and two women. Each person had more than five years’

Video viewing reliabiliQ of famil_y theraky interviews 299

experience of family therapy. The practice of the team was to have three members viewing through the one-way screen while one member interviewed the family, and for this therapist to consult with the team some 25-40 minutes into the interview. For the purpose of this study, involving eight initial interviews, the therapist was informed via telephone that 30 minutes of the interview had elapsed; the therapist then left the room as soon as practically possible. Prior to any discussion, members of the team made independent judgements of the family.

These judgements of the family interaction were obtained by ranking seven structural concepts of dysfunction as used by Street and Foot ( 1984) in a study on training observation skills. The seven concepts were: Enmeshment (E); Disengagement ( D ) ; Lack of Maintenance of Generational Boundaries (LGB); Chaotic Parental Discipline (CPD); Non-resolution of Conflict (NRC); Conflict Avoidance (CA); and Rigidity ofApproach (RA) . Members ofthe team were familiar with the concepts, their use and definitions.

To examine the perceived prevalence of each of these forms of dysfunction for each interview, a composite ranking of all the concepts was obtained by an indirect method which involved rank ordering different combinations of concepts using an incomplete latin square as recommended by Durbin ( 195 1) and Shucker ( 1959). For this purpose the concepts were combined in threes in such a way that each concept was paired with each other concept no more than once. The judges then made separate sets of rankings within triads of the concepts in order to obtain an overall ranking of the seven concepts (see Table 1 ) . Once the individual ranks were derived for each interview, the ranks for the team as a whole were calculated. This was achieved for each interview by assigning the numerical value of 1 ,2 ,3 , . . . to the concepts ranked first, second, third, . . . by each individual and summating them.

Collection o f video judgements A therapy team composed of three different individuals working within the same agency and utilizing the same theoretical model was recruited to form the Observing Team. ‘The therapists in this team were all female and all had more than fi\7e years’ experience in family therapy. They were familiar with the seven structural concepts and their definitions. The Observing Team watched the video interviews of the eight families seen by the Clinical Team. At the end of the first half-hour of the interview the members of the Observing Team were required to make their own independent judgements of the family’s interaction in an

300 E. Street and H. Foot

identical fashion to the Clinical Team using the same ranking method. The overall rankings for each interview produced by the Observing Team as a whole were similarly obtained as for the Clinical Team. These rankings are presented in Table 2 along with the Clinical Team rankings.

TABLE 1. Family interaction ranking

In each block ofthree concepts circle the ‘M’ for that concept which you thought most evident from the session and then circle the ‘L’ for that concept you considered least evident.

1. (c) Lack of maintenance of generational boundaries (b) Disengagement (a) Enmeshment

2. (b) Disengagement (d) Chaotic parental discipline (0 Conflict avoidancc

3 . (c) Lack of maintenance of generational boundaries (e) Non-resolution of conflict (0 Conflict avoidance

4. (c) Lack of maintenance of gcnerational boundaries (cl) Chaotic parental discipline (g) Rigidity of approach

5. (a) Enmeshment (e) Non-resolution orconflict (d) Chaotic parental discipline

6. (a) Enmeshment (g) Rigidity of approach (9 Conflict avoidance

7. (b) Disengagement ( e ) Non-resolution ofconflict (g) Rigidity of approach

Scoring: M = 2; unmarked = 1: L = 0

hl L M I. M L

M L ?v1 L M L

M L M L hl L

M L M L M L

M L M L M L

hl L M L M L

The composite ranking is obtained by summing the scores li-om all the triads and then rank-ordering the concepts according to their scores.

Video viewing reliability of farnib therapy interviews 301

TABLE 2. Rank ordering o f concepts clinical ( C ) and observing ( 0 ) teams and the inter- and intra-team correlation coefficients

Correlation Inter- Intra- team team

Session Rank (Spearman’s (Kendall’s Team 1 2 3 4 5 6 7 rho) W)

C D NRC RA LGB CPD CA E 0.40

0 D RA NRC CPD CA LGB E 0.75t

C RA NRC E CA D CPD LGB 0.71 t 0 RA NRC CA E CPD LGB D 0.61*

C NRC E CPD D LGB CA RA 0.43

0 NRC CPD RA D CA LGB E 0.62*

C NRC CA D CPD RA LGB E 0.717

0 D CA NRC CPD E LGB RA 0.927

C E LGB D RA CPD NRC CA

0 E NRC RA CA LGB CPD D 0.64*

C E LGB NRC CPD RA D CA 0.79t

0 E LGB CA NRC CPD RA D 0.68t

C E CPD CA LGB NKC RA D 0.667

0 E CPD NRC LGB RA CA D 0.877

C D NRC CA RA CPD E LGB 0.867 0 NRC D CA CPD LGB RA E 0’79* 0.841

0.81 t

0.80t

0.21

0.7 1 *

0.03 0.35

0.66*

0.75*

E D LBG CPD NRC CA RA

t *

Enrneshrnent. Disengagement. Lack of Maintenance of Generational Boundaries. Chaotic Parental Discipline. Non-resolution of Conflict. Conflict Avoidance. Rigidity of Approach. p < 0.05 level. p<0.01 level.

302 E. Street and H. Foot

Results

Rank correlation coefficients and associated probability levels between rankings of the Clinical and Observing Teams are given in column l of the correlations in‘l‘able 2. These correlations are relatively high and show that the Observing Team’s rankings of the concepts significantly matched those of the Clinical Team for six of the eight sessions. O n two sessions (3 and 5) there are low significant correlations indicating rather divergent rankings by the teams. Closer inspection of Sessions 3 and 5, however, reveals that the low correlations are due substantially to obverse relationships between just two concepts in each case. O n Session 3 , E and RA are obversely related; there is remarkably close agreement on the other concepts. Similarly on Session 5, NRC and D are obversely related and the other concepts, although more discrepant than on Session 3 , are not entirely unrelated (e.g. E was ranked first by both teams).

Of course, high and low inter-team agreements can only be properly evaluated in the light of inter-rater reliability within teams. If members ofthe Clinical and Observing Teams are divergent in their within-team rankings, then little meaning can be attributed to between-team correlations. Column 2 of the correlations in Table 2 indicate the intra- team reliabilities and associated probability levels using Kendall’s Coefficient of Concordance. It is clear that both Sessions 3 and 5 which yielded least inter-team agreement also showed the lowest reliabilities within teams. These results suggest strongly that the patterns of dysfunctional behaviours exhibited by these two families proved to be the most difficult for both teams to discriminate. The other six sessions with higher inter-team agreement showed significantly higher reliabilities with one exception (Clinical Team, Session 1 : Kendall’s W = 0.40).

Whilst families may vary in the discriminability of their patterns of dysfunction, it is also possible that clinical concepts differ in ease of diagnosis. Intra-class correlation coefficients (ICC) via the analysis of variance were calculated for each clinical concept across both Clinical and Observing Teams. This method provides an expression for the variances within and between sets of measures and is an appropriate measure ofreliability when n judges rate k cases on a particular concept or dimension (cf. Bartko and Carpenter, 1976). Although all concepts showed significant amounts of agreement between therapists, there was considerable variation in their reliabilities: ICC = 0.70 for Enmeshment ($<0.01); 0.55 for Disengagement ( p < O . O l ) ; 0.37 for Lack of

Video viewing reliability of farnib therapy interviews 303 Generational Boumdaries @<0.01); 0.32 for Rigidity of Approach @<0.01); 0.22 for Conflict Avoidance @<0.05); 0.16 for Non- resolution of Conflict @< 0.05); and 0.15 for Chaotic Parental Discipline < 0.05). Enmeshment and Disengagement, therefore, showed the highest reliabilities and Kon-resolution of Conflict and Chaotic parental Discipline the lowest. Clearly the concepts did differ in their discriminability.

Discussion

Results point to considerable similarities in the judgements of Clinical and Observing Teams for most of the family interviews. Therefore, we can have some confidence that, for training purposes, family dysfunction can as readily be discriminated by means ofvideo-recorded interviews as by live interviews. I t is particularly remarkable that such relatively high consensus was achieved on first interviews with families when the therapist is only just exploring issues with the families and at the stage of probing for any sign of dysfunction.

Problems in the interpretation of family interactions do occur, however, in certain cases, and the results highlight three factors which need to be taken into account when comparing the judgements of different therapists.

First, some families are inherently more difficult to assess than others on the basis of their patterns of behaviour: signs of dysfunction are less visible, and considerable variations exist in the interpretations of trained therapists, irrespective of whether interviews were live or video- recorded. The low reliabilities ofjudgements within teams in addition to the poor inter-team agreement on sessions 3 and 5 support this view. Of course, it may also be that therapists have to spend a much longer time with some families than with others before the nature of the problem becomes apparent. Since the interviews were first interviews it might take the therapist considerably more than half an hour with some families to come to any tentative decisions about the type ofdysfunction that are presenting.

The second factor is that some clinical concepts are more difficult to identify and discriminate than others, a problem raised by Street and Foot (1984) in relation to the training of family therapy observation skills. It is interesting to note from Table 2 that the most reliable concepts (Enmeshment and Disengagement) are the concepts most likely to be afforded the highest or lowest rankings. O n 10 out of the 16 series of judgements by the two teams either E or D features in first rank order

304 E. Street and H. Foot

position and on 10 they feature in last rank order position. So these concepts tend either to be judged as present (pervasively so) or as virtually absent. Less reliable concepts such as CPD and CA tend to occur predominantly in the middle rank positions or equally across all rankings which suggests that judges have greater difficulty in identifying them.

Related to the question of concept discriminability is the third factor, that concepts cannot be assumed to be independent of each other. Identification of one concept may carry with it implications for other concepts. A good example of this is in relation to the concepts of Enmeshment and Disengagement. Conceptually Enmeshment refers to the intrusion of family members into each others’ lives and their over- involvement in each others’ thoughts and beliefs (Street and Foot, 1984). Disengagement refers to the very opposite phenomenon: the distancing of one family member from the family system and the detachment from emotional ties within the family. It is not surprising, therefore, that when E is ranked high D is ranked low, and vice-versa (Table 2). Sessions 1 and 7 are prime examples of this obverse relationship between E and D. Evidence from a factor analytic study using other video-taped examples of family therapy interviews confirms this impression (Street 1988). In similar vein Chaotic Parental Discipline and Non-resolution of Conflict are positively related to each other; indecision over conflicts and tensions is entirely consistent with lack of discipline occurring within the family.

The implication of these positive and negative relationships between concepts is that, if a particular concept is clearly identified by the therapist, judgements about the presence or absence of other concepts may be made by inference rather than by direct observation. There might also be hierarchical structurings amongst all concepts which hinge upon the initial identification of one or two key concepts.

These points, of course, apply to the judgements of family therapy interviews irrespective ofwhether they are observed live or on video, and they may help explain why there can be such variation in the judgements of the therapists. None of these points, however, addresses the basic issue as to whether the medium ofobservation itselfinfluences judgements. As mentioned earlier, there may be certain factors conveyed in the emotional atmosphere of the moment which are picked up in the live interview situation but are missed in the observation of a video-tape. Alternatively, video-viewing of an interview may result in the observer becoming more aware ofsubtle aspects ofan interaction missed in the live interview. Inevitably it is easier for the therapist to form a more global impression of the behaviours of all family members through video

Video viewing reliabilit3, o f farnib therapy interviews 305

observation than when interacting in uiuo with one member on whom the therapist’s whole attention may be focused. Both perspectives may be equally valid and useful in coming to an interpretation of family functioning and, for training purposes, it may be essential to mix, carefully selected and edited video-tapes clearly depicting key concepts with the ‘rough and tumble’ reality of live family interactions which are likely to illustrate the sheer complexity ofthe phenomena to be observed.

Unfortunately, perhaps, the study does not provide any other clues concerning differences in judgements actually experienced by the therapists operating live or via video. It should be the task of further research to explore therapists’ cognitions resulting from the use to the two media. Given the low reliabilities within and between the Clinical and Observing Teams on sessions 3 and 5 in our study, it is our view that difficulties in assessing those particular families and in identifying the key concepts that characterize their interactions probably account for most of the variability in judgements and that media differences as such contribute relatively little. But this does not mean to say that in other circumstances or with other families media effects would not be more influential. As it is possible for some video material to convey different impressions to different individuals, trainers would be well advised to seek consensual agreement from experienced colleagues that what is actually shown on a training videotape is what is intended to be shown. Failure to engage in this process may result in trainees becoming unintentionally confused and misdirected.

Acknowledgements

The authors wish to acknowledge the assessors’ very helpful comments on this paper; specifically, David Will for the idea of ‘therapeutic urgency’ and Ivan Eisler for his advice on statistical analysis.

References DCRBIK, J. (195 1) Incomplete blocks in ranking experiments. Briti.rh Journal of Psychology

MINUCHIN, S. (1974) Families and Family Theraky. London. Tavistock. MONTALVO, B. (1973) Aspects of live supervision. Family Process, 12: 343-359. ROBEKTS, W . (1987) The strengths and weaknesses in co-therapy. In: A. Bertovim, G.

Gore11 Barnes and A. Cooklin (Eds) Family ‘TherapJ: Complementary Frameworks of Theory and Practice, London. Academic Press.

SHCCKER, R. E. (1959) A note on the use oftriads for paired comparisons. Psychometrika, 20:273-276.

(Statistical Section), IV (Part 11): 85-90.

306 E. Street and H. Foot

SPEED, B., SELIGMAN, P., Km(;sroN, P. and CAIIE, B. (1982) A team approach to

STREET, E. (1 988) The Training oJFamily Therapy Observation Skills. Unpublished doctoral

STREE.1, E. and FOOT, H . (1984) Training family therapists in observational skills.

TOMM, K. and LEAHY, M. ( 1 980) Training in family assessment: a comparison of three

therapy. Journal of Family ‘Therapy, 4: 27 1--284.

thesis, University of \.Vales College of Cardiff, CardiK.

Journal of Family Therapy, 6: 335-345.

teaching methods. Journal oJMarital and Famib Therapy, 6: 453-458.