Annals of Library Science and Oocumentation 1985, 32( 1-2), 1-6

THE UNESCO COM~10NCOMMUNICATION FORMAT

Unesca's Common Communication Format(CCFJ like other exchange formats enables freeexchange of bibliographical records withiniri[ormation community. To make it more effec-ti: ", efforts should b~ for making conversionsin existing formats and their easy availability.Studies the CCF in relation to other formatslike UK MARC, AGRIS, INIS and MEKOF-2.

INTRODUCTION

/ The Unesco Common Communication Format(CCF) [1] is an exchange format intended foruse by agencies Within the information com-munity which wish to exchange bibliographicrecords with one another. Jt is intended parti-cularly for those agencies which have records ofmonographs and serials, library type materials,as well as records of journal articles, contri-butions in proceedings and other parts of physi-cal documents which constitute intellectuallyin~e endent entities; these records tend to beere ted by abstracting and indexing services.. An exchange format consists of three com-

ponents:

(a) Rules for the arrangement on a computerstorage medium of data to be exchanged(including rules for the size of the physicalstorage medium).

(b) Codes to identify the different data ele-ments in-the record (e.g. author, title,scale of map, starting date of journal).

(c) Rules for the formation of different dataelements, very closely tied up with (b).The data elements separately identified bythe code in the exchange format have tobe defined, not only in terms of contentbut also in form, if the records are to besuitable for use by another agency.

Vol 32 Nos 1-2 March-j une 1985

ALAN HOPKINSONBritish Library Bibliographic Services DivisionStore Street, London WC lC 70CUnited Kingdom

Effective exchange among .,cncics can beaccomplished only if the reco~ of the agenciesconform to (a), (b) and (c).

In respect of the first component, thereexists a standard format for the exchange ofdata on magnetic tape which has been estab-lished by the International Organization forStandardization, ISO 2709 [2}. Universal accept-ance of this standard has helped the informationcommunity enormously. It is accepted for theexchange of data on magnetic tape, and it 'isalso being used for the formating of biblio-graphic data sent on-line down the telephone.

The second component relates to the tags,indicators and subfield codes, in short, codeswhich define the different data elements in therecord. There is no universally accepted standardfor these. One reason why they vary amongdifferent implementations is that agencies havedifferent requirements in respect of the datathey wish to exchange. Most countries, forexample, have found it necessary to developtheir own national MARC formats, and thereare also formats developed by or specificallyintended for abstracting and indexing services.They all embody different schemes of tags andother identifiers.

The last component, the form and contentof the data elements, varies according to thecataloguing rules used as well as according to theway the different data elements prescribed bythe rules are divided up and separately identi-fied.

, The Common Communication Format im-plements ISO 2709 in full. It has its own systemof tags and other identifiers for the identifica-tion of data clements, but it docs not assumeany particular cataloguing code. This point willbe elaborated latcr.,/

HOPKINSON

HISTOR Y AND DEVELOPMENT Unesco cornrmssion of data element directorytaking into account tL' data elements of various

The Common Communication Format has been international exchange formats, including UNI-developed by a group of experts convened by MARC[ 5], USSR/lJS Common CommunicationUnesco and working in the context of the Format, MEKOF (the format of the CMEAUnesco General Information Programme. In countries[6], the format of the International1978, the International Symposium on Biblio- Serials Data System [7] , and the Referencegraphic Exchange Formats[3], sponsored by Manual. A consultant prepared a KWOC indexUnesco took place. Unesco was interested in to the data elements in the formats and anthis topic, because they had received a large abbreviated data element definition for eachnumber of requests from agencies around the data element. These became the basis for theworld for advice on setting up nacional or discussions which revolved around the defi-regional bibliographies. When recommending nition of a mandatory core set of data elements.which format a system should use, expert!>were Optional data elements were then added toencountering difficulties, since format were make possible the carrying of complete biblic-divided, roughly speaking, into two categories: graphic records in the format. However, itthose used by national libraries, the MARC quickly became clear that for the format to befarriily of formats, and those used by ab- accepted it would not be possible to prescribestracting and indexing services, which, though

d exactly the form and content of each elementsmore iverse, were to a certain extent exempli-fied by the UNISIST reference Manual format although it was possible to be precise in some

data elements such as those whose forrri and[4] . This format had been developed jointly byUnesco and the International Council of Scienti- content were prescribed by other standards,f such as the standard numbers (ISBN andie Unions Abstracting Board (ICSU-AB), becausethe need, was felt for standards and guidelines ISSN) and key title, Sufficient consensus

could not be obtained in the case of otherfor secondary services considering the automa-, f hei d b Th' . I data elements. That could never be achiev-tion 0 t elf ata ases. e mternatrona sym- . , . 11 d al '

posium addressed itself to the problem of the ed wltho~t mterna~lOna y a~cepte cat ogumgexistence of the two categories of format which ru!Ys, which are still somethmg for the future.

fl d h diff 'r. ti ~ ..$'E>, when the Common Communication Formatre ecte t e two 1 rerent mror'rna Ion co - .munities, libraries and secondary services- It was published by U~esco in the middle ofwas clear that in the future, at the level of 1984, the represe~~~tton of ,ma~?, of the datastandardization of bibliographic records, there elements was specified as bemg m acc~rdancehad to be more cooperation if the needs of the with the particles of the agency preparmg theend user, for example, a research worker want- ~ecord', which! me~ns eaclh agenhcyaccording to, . 1 '1 bi d h Its own cata ogumg ru es. T us an agency111garttc es on a parttcll.ar su ~ect, an - tt en . . d' h ' ld . dh ials i hi I h b f d recelvmg a recor m t e CCF wou nee tot e seri s in w IC1 t cy were to e oun , were k hid b h .

, . f d h f d c now t e ru es use y t e source agency mto be sans le . Researc ers 111 rerercnces tv ,. .. 1 ' h d t b f b t t' 1 order to determine whether It was worth takmg

:Udt1L~Sin t. e b<ia hases to a tS racl,1bng antO these records. That is not to say that records

tn exmg services, ut aye 0 go 0 a 1 rarv 0 d'ff' I ld bfi d h iate iournal 1" created according to 1 erent ru es wou em t e approprtate Journ or monograp 1. fall' , B '

o no use at to a recelvmg agency. emgThe symposium resolved to attempt to in the same Common Communication Format

break down the barriers between the two infor- and therefore having the same identifiers, re-mation communities. One way to contribute cords can easily be interpreted by the computertowards this was the devising of a format which programs of a receiving agency. They can bewould be directed specifically to neither corn- used to find out what an organization has inmunity and could be used by any agency which terms of its book stock, assuming that the re-was providing records to both the communities. cords originate from a library, or they can beThe Ad-hoc Group on the Establishment of a used to produce current contents lists of jour-Common Communication Format was con, nals, for example, if the records originate fromvened. The members firstly requested that a secondary information service. They can be

2 Ann Lib Sci Doc

COMMON COMMUNICATION FORMAT

added to on-line databases, though the absence tion manual is needed to enable the creationof common rules for access points may make of bibliographic records for entry into theinformation r'!y.ieval a little more difficut and system. This manual has to make data entry asless efficieutz" it is when such records from easy as possible, which is very important if itdifferent source; prepared according to different is to be used by cataloguers far removed fromcataloguing codes are merged into a database the help of fellow professionals. Most impor-with a view to producing a printed product tantly , the data entered must be capable ofsuch as a ctalogue that: problems arise. Records being transformed into the output as prescribedcreated under different rules cannot be easily by the exchange format, for it to be useful tofiled together in a printed product. There do recipient organizations. At one extreme, dataexist the International Standard Bibliographic could be entered in the form exactly as requiredDescriptions[8] which ensure uniformity in the by the exchange format. This would causedescriptive area of the record, but we are still difficulties in data entry. As mentioned above,IQ~king forward to the day when t~ere ycv-the Common Communica.tion Format is inde-universal standards for the form of headings. pendent of any cataloguing code. To a cata-

, loguer, however a cataloguing code is a veryimportant element in record creation, and sothe format must be related to the code. There-fore, an implementation manual aimed at cata-loguers has to be related as closely to the cata-loguing rules as it is to the exchange format.UK MARC [9], far example, contains againsteach data element the relevant AACR[10]sect;'J'l number. The first implementationmanual using the CCF will take AACR for itscatalo£:ui1tg rules. since AACR is one of themost widely used codes, However, AACR doesnot address all the conditions encounteredespecially by seconda y services ((:hapter 13is not -.'(;ry detailed '1 nd is ver/ much III thelibrary tradition L Thercfore. extra g'!1;d~"11CC has

.' L

to be gi'JC11.so that the manual C;Jn be used tocreate records which are compatible withAACR. but which will be acceptable in abstract-ing and indexing publications.

THE CCF AND SOFTWARE

Th:.; Common Communication Format is nO~fcourse an end Vi itself. Uncsc o was in teres edin its establishment as d means to an end. her ;'";r.]r has to enable <he exchange of recordsan:!):)t- ,-ijfferel1t agencies throughout the worldal~(; b,;t.ween developed and. developing court-trie-, fo be useful. what is required is a pool ofrecords which can be drawn on by agencies toavoid their having to catalogue of their OWIl re-cords themselves, working on the same principles.as the cooperative catah.;ui.ng projects found insome countries. For this to be effective. It needsa number of agencies, all provided with compati-ble software for data entry an.i informat ion rc-trieval, and using a common format, at least f'orexchange. In an ideal situation, they would,!;ethe same cataloguing rules, but as yet this is notfeasible. The Institut fur Maschinelle Dok \.1-

mentation in Graz, Austria, is developing asoftware package known as iV+V. Unesco isinterested in this software package, since it canbe applied to bibliographic information retrievaland has offered advice on the requirements tomake it suitable for use in developing countriesfor organizations wishing to exchange recordswith one another. Therefore, one of theserequirements is the Facility to provide dataoutput on a computer-readable medium for-matted to the provisions of international ex-change formats. Exchange formats do notusually specify how data are to be input and theCCF is no exception. Therefore. an irnplcrnenta-,

~"1..:

Vol 32 Nos 1-2 March-June] 985

DESIGN OF THE IMPLEMENTATIONMANUAL

Tb implementation manual enabling the entryof data in a format compatible with the CCFhas required a different approach from thattaken in data entry manuals for systems u3ingother formats.

One reason for this is that most exchangeformats such as national MARC formats havebeen developed simultaneously with their inputformat. and with particular cataloguinr; rulesin mind, Therefore, the forr;,at correspondsvery ..loselv to the requirements or the cata-

loguing rules. The same goes for the exchangeformats of abstracting and indexing serviceslike AGRIS[ 111 and INIS[12]. They too areclosely related to the input format and thecataloguing rules used. Users of the CCF arenot, however, using a format designed with anyregard for their data entry procedures. Thiscauses problems which can be overcome by thesoftware being designed to convert the data.The manual must, therefore, be directed towardsthe cataloguing rules and the input proceduresrather than the exchange format.

In practice, cataloguers do not enter biblio-graphic data in the form in which it appearson an exchange tape. They usually enter dataon a worksheet next to the tag to which theyrelate. On the ISO 2709 exchange tape, tagsare held apart from the data to which they re-late. The tags are associated with their res-pective data not by location but by means ofpointers, strings of numbers held in a part of therecord called the directory! 13]. Although thesenumbers are easily calculated by the computer,they are difficult for a human being to cal-culate. Therefore, they are never input by thecataloguer. Nevertheless, in many formats, thecataloguer finds the codes on the worksheet,the tags, in numerical order, because the y,unlike the pointers in the directory, weredesigned with an eye on input.

The order of the tags in the CCF is notbased on the order in which a cataloguer usingAACR enters data. The development of theCCF concentrated on making the format com-patible with other exchange formats. perhapsat the expense of making the format suitablefor data entry as it stands. The CCF also in-cludes a sophisticated record-linking techniquewhich requires complex codes which the cornputer rather than the cataloguer shou. 1 enter.Its record structure was devised t o enablerecord-linking; this involves the analr-.i- ofrecords into their different bibliographi. \ "dsto a much greater extent than cataloguer : r: etaken the analysis in more traditional c.u 'f~i -

ing systems. This kind of record struct.u ' : ,j,

only recently become feasiblc within ri'" \text of ISO 2709, since an extenciondirectory incorporated in the second ,i I ••

enabled pointers to group together parts,

4

HOPKINSON

cords into what the CCF calls 'segments'. Thisgreater conceptual gap between the exchangeformat and the cataloguer's practice has madethe development of an implementation manualfor data entry amount to the development ofa new format. Incidentally, had it not beengeared to just one cataloguing code, the re-sulting manual would have been unwieldy andcumbersome to use.

THE CCF AND OTHER FORMATS

wh y do we need another exchange format whenwe already have UNIMARC and. the RM, not tomention national exchange formats, is a notinfrequent criticism levelled at the CCF. How-ever, both these formats cater really well foronly half the information community. UNI-MARC is ideally suited to serve the nationallibraries with their own national formats whichwish to exchange records among themselves.Were it not for UNIMARC, exchange amongnational libraries would have to take place inevery instance on a bilateral basis, with conver-sion programs being written between every pairof formats among w'uch exchange was takingplace. An alternative would be for everyoneto use one national format, for example, usMARC, but if that method were chosen, therewould be an international outcry every time thenational format chosen was changed, since itwould mean adjustments to everyone else'sprograms. UNIMARC is sufficiently close to themajority of, if not to all, MARC formats toavoid complicated conversions between it andthe national MARC formats, and it can be chan-ged only by international agreement. The maindisadvantage of UNIMARC is that it is biased,naturally, towards library material and there isno easy way to link the record of an article tothe record of the serial containing it, for ex-ample. AACR-type analytics which consistusually of an access point for a work pub-ished elsewhere, even when it is contained in aolurne with other works, are no problem, but

. 'ything more complex becomes a little un-v :c1d v, The Reference Manual specificallya, lrcsscs this problem, but finds the solutiont.. it in a way which belongs to the days whental c were nor only the sole medium for ex-

Ann Lib Sci Doc

COMMONCOMMUNICATIONFORMAT

change but were also much used in textual dataprocessing for storing data where disks wouldbe used today. I'ach bibliographic level consti-tuting an item which someone might wish toretrieve has its own record, and within thesame record are all the details as to the mono-graph, collection or serial in which it is found.Thus, there will be an abbreviated serial recordin every record of a contribution within thatserial. The data elements in the total record areidentified by tags which indicate the biblio-graphic level of that data element. TJlUs, the titleof a monograph has a different tag from thetitle of a serial and the same goes for author,collation details, etc. In a database organized inthis way, there would be a great deal of repeti-tion; on an exchange tape it does not matter somuch; indeed knowing that every record is com-plete makes processing easier.

The CCF on the other hand does have amechanism to take care of record linking[14].it can link records at separate bibliographic'levels which when added together constitutethe recc.rd of a bibliographic item; or it can linkrecords with. different kinds of relationships,such as a serial title to its former title and viceversa, or a work to its translations. The linkingmechanism has been so devised as to be flexible.It had to be flexible to be able to be compatiblewith the complex mechanism of UNIMARCand the 'flat' record structure of the ReferenceManual, which assigns a different tag to a dataelement, depending on whether it is at theanalytic, monographic serial or collection level.Conversion between the CCF and MEKOF wasalso taken into account: although the recordstructure of MEKOF is 'flat', it uses a five-digittag to enable one digit to indicate consistentlythe bibliographic level of the data element.However, the flexibility with which the CCFis endowed means that users have to beware:they must ensure that: they themselves use themechanism in a consistent way.or else recipientsof the records may not have built into theirprograms the correct mechanisms to decodethe particular mechanisms used. Moreover, therecords are split into 'segments' which point toother records, so it is necessary to ensure thatall the little bits of a record are exchangedtogether. Thps, housekeeping procedures have to

Vol 32 Nos 1-2 March-june 1985

be developed between parties to an exchangebefore exchange can take place and at themoment there are no guidelines on these.

Consequently, there is still room for allthree international exchange formats whichhave been mentioned in this paragraph - MEKOFis regarded as regional rather than truly inter-national; and there is no question that AGRISand IN IS should give up their formats whichhave been devised with a particular system inmind.

THE FUTURE

The important aim of all exchange formats isto facilitate the free flow of information, inthis case bibliographic references. For thisto be achieved, probably the most importantthing is for computers to be able to talk to eachother, which is something to be achieved byexperts outside the library and information com-munity. After that what is required are inter-nationally-agreed forms of headings, or autho-rity files. At the same time, more effort shouldbe put into making conversion among existingexchange formats (and here the national formatsare included) and making them readily avail-able. Then the CCF would indeed be an effectivebridge between the library community and theinformation community and would certainlygain popularity as a format on which to base aninternal format for the databases of both librarysystems and secondary services.

REFERENCES

1. CCF: the common communication format. Paris:Unesco, 1984.

2. Internarion.d Organization for Standardization .•Docurncucatio.r : format for bibliographic infor-mation interchange on magnetic tape. 2nd ed.Geneva: ISO, 1981 (ISO 2709-1981).

3. International Symposium on Bibliographic Ex-change Formats: Towards a common biblio-graphie exchange format: proceedings. Budapest:

OMKDK, 1978.

4. Martin M D: Reference manual for machine.-read-able bibliographic descriptions. Paris: Unesco,1974, (SC. 74/WS/20). 2nd ed., edited by rlDierickx and A Hopkinson. Paris: Unesco, 1981.(PGI/81/WS/22).

5

HOPKINSON

5. UNIMARC: Universal MARC Format. London:IFLA International Office for UBC, 1977.

6. International interchange format - MEKOF.2:specification of data elements. Moscow: ICSTI,1977 (Official English translation).

7. ISDS manual. Paris: Unesco, 1984.

8. International Standard Bibliographie Descriptions,published by the IFLA International Office forUBC covering monographic publications, serials,non-book materials, cartographic materials, anti-quarian materials and printed music; there is alsoan ISBD general, a framework which governs thestructure of the other ISBD's.

9. UK MARC manual. 2nd ed. London: BritishLibrary Bibliographie ServicesDivision, 1980.

10. Anglo-American Cataloguing Rules. 2nd ed.London: Library Association, 1978.

11. Most exchange format documentation containsat least one illustration of a record in ISO 2709structure. See for example reference manual.op cit, pA.27.

6

12. CCF. op dt. p.19-:!4.

For background inforrnation to the other biblio-graphie exchange formats mentioned see:

(a) Long, Anthony: UK MARC and US/MARC: abrief history and comparison. Journal of Docu-mentation 1984,40(1), 1-12.

(b) Hopkinson, Alan: International access to biblio-graphie data: MARC and MARC-related activities.Journal of Documentation 1984,40(1), l3-24.

ABBREVIATIONS

AGRIS - Agricultural Information System of the Foodand Agricultural Organization of the United Nations.

INIS - International Nuclear Information Systems of theInternational Atomic Energy Agency of the UN.

Ann Lib Sci Doc