Download - Unicode Conversion
SAP NetWeaver BIUnicode Compliance
Product Management SAP NetWeaver BI
Version 7.0December, 2008
© SAP 2007 / Page 2
Agenda
1. Unicode in General2. Excursus: MDMP3. Unicode support of SAP NetWeaver BI4. Interfaces to SAP Systems5. Interfaces to non-SAP Systems
Unicode Essentials
What is Unicode?Character encoding schema for (nearly) all characters used world wide
Each character has a unique number („Unicode code point“)Notation U+nnnn (where nnnn are hexadecimal digits)
See http://www.unicode.org for complete code charts
E3 91 B979 3434 79U+3479
CE B1B1 0303 B1U+03B1
C3 A4E4 0000 E4U+00E4ä
6161 0000 61U+0061a
UTF-8UTF-16little endian
UTF-16big endian
Unicodecode point
Character
Representation of Unicode Characters
UTF-16 – universal transformation format, 16 bit encodingFixed length, 1 character = 16bit. Surrogates using 2*16 bitPlatform dependent byte order16 bit alignment restriction (needs even addresses)
UTF-8 – Unicode transformation format, 8 bit encodingVariable length, 1 character = 1...4 bytesPlatform independentNo alignment restriction
Character expansion modelSeparate Unicode and non-Unicode versions of SAP ERP
No explicit Unicode data type in ABAPSingle ABAP source for Unicode and non-Unicode systemsAutomatic conversion of character data for communication between Unicode and non-Unicode systems
1 character = 2 bytes (UTF16),(types C, N, D, T, STRING)
Unicode kernelUnicode database
1 character = 1 byte(types C, N, D, T, STRING)
Non-Unicode kernelNon-Unicode database
ABAPsource
Non-Unicode
R/3
UnicodeERP
ABAPsourceABAPsource
Transparent Unicode Enabling of SAPBusiness Suite
Implications
You are not interested in Unicode?You are not interested in Unicode?
No customer is forced to convert to UnicodePossibly minor changes for not Unicode enabled programs(more restrictive syntax check as of WebAS 6.10 on)
You are interested in Unicode?You are interested in Unicode?
Major part of ABAP coding is ready for Unicode without anychanges
Minor part of ABAP coding, i.e. some customer-specific ABAP coding,has to be adapted to comply with Unicode restrictions
SAP will deliver powerful tools to convert your existing system
Resource Requirements for Unicode-compliantBI
What are the additional resource requirements after aWhat are the additional resource requirements after aUnicode conversion?Unicode conversion?Text fields are usually bigger in a Unicode environment than in a Non-Unicode environment. Enhancedfunctionality requires additional resources, which strongly depend on the given customer scenario. In thefollowing, we give a rough estimation what a Unicode conversion could mean for the resources.
CPU’sWe expect additional SAP NetWeaver BI requirements of roughly 30% more CPU power - likethe SAP ERP requirements.
Main MemoryWe expect additional SAP NetWeaver BI requirements of roughly 50% more memory - like theSAP ERP requirements.
Disk StorageDisk Storage depends strongly on the underlying DBMS and the given data model/volume. Fora significant share of InfoCube data (only numeric keys!), there might not be a significantincrease of the DB size.For SAP ERP on ORACLE, tests have resulted in roughly 35% additional disk space. For SAPBI, we expect – depending on the scenario – less additional disk space.Note that after a conversion, the disk size may even decrease because of DB reorganisation.
The Unicode Upgrade Project
PreparationPreparationPreparation
Set up the UnicodeSet up the Unicodeupgrade projectupgrade project
Analyze data toAnalyze data tominimize downtimeminimize downtime
ConversionConversionConversion
To be done during systemTo be done during systemdowntimedowntime
Unload /reload process forUnload /reload process forsmall databasessmall databases
Minimum downtime tool forMinimum downtime tool forlarge databaseslarge databases
(Incremental Migration(Incremental MigrationIMIG)IMIG)
Post-ConversionPostPost--ConversionConversion
Set up the systemSet up the systemfor Unicodefor Unicode
Start up the UnicodeStart up the Unicodesystemsystem
© SAP 2007 / Page 9
Agenda
1. Unicode in General2. Excursus: MDMP3. Unicode support of SAP NetWeaver BI4. Interfaces to SAP Systems5. Interfaces to non-SAP Systems
MDMP – Definition
What does MDMP mean?What does MDMP mean?
In the past, SAP supported multiple languages by MDMPMDMP = Multiple Display – Multiple ProcessingMixed code page technology depending on logon language
Several 100 satisfied customersMDMP was an SAP workaround for accessing multiple code pages
Defined users where accessing the solution using SAPgui frontendOnly little integration into the world wide web
MDMP should not be used anymore for new installationsMDMP is not supported for SAP NetWeaver BI
MDMP – Connection
How can I connect SAP BI to an SAP MDMP source system?How can I connect SAP BI to an SAP MDMP source system?
Language-dependent source tablesMost text tables include the language key. In an SAP system, the code page can be derivedfrom the language key. This enables the correct conversion of the texts from the sourcesystem to SAP NetWeaver BI.
Language-independent source tableIf a text table does not include the language key (e.g. customer texts), these texts have to beconverted “manually”. The code page (or language) has to be defined for EACH record inthese language-independent tables either by appending a new column into the table or bydefining the extraction job in the corresponding codepage.If there is no language information given, the source tables will be converted based on thelogon language of the user.
MDMP – Project Based Solution
Why is there no generic solution for the connectionWhy is there no generic solution for the connectionof MDMP source systems?of MDMP source systems?
All language-independent source tables must be checked and for each record, the codepage which it was created in must be determinedIn order to derive this code page information, attribute values, organizationalassignments or heuristics may help, but there is no generic approachExamples:
in order to determine the code page of a customer name, the country where she/helives in may be used usually not correct, if e.g. a Japanese customer lives inRussiaAlternatively, the assignment to a sales organization may help
For more information see How-To Guide in SDN
Design Principles for Source Systems
How can I ensure a smooth Unicode conversion?How can I ensure a smooth Unicode conversion?SAP NetWeaver BI receives the following information from source systems
Date and timestamp informationKey figuresCharacteristics by keyTexts
How should I design the data in the source system?Date and timestamp information as well as key figures are just figures; the conversion is notcriticalCharacteristic keys (e.g. material number, customer number, brand key, country key) must bedesigned as 7-bit-US-ASCII characters (common characters). Don’t use other characters askeys or attributes.
As a benefit, the permitted character setting in the BW customizing need not be maintained. See SAPnote 173241 for more information.Note that maintaining the permitted characters makes you code page-dependent. There might beproblems with the conversion to Unicode (e.g. characters like ‘µ’ are a lower case letter in Unicode – andthus not allowed)
Texts are ONLY loaded into the text tables of the InfoObjects; transaction data and master dataattributes must NOT contain texts. Texts are usually language-dependent and must be designedin the source system with an associated language key (even if e.g. the SAP ERP data model isnot providing this).
© SAP 2007 / Page 14
Agenda
1. Unicode in General2. Excursus: MDMP3. Unicode support of SAP NetWeaver BI4. Interfaces to SAP Systems5. Interfaces to non-SAP Systems
Unicode-Compliant SAP NetWeaver BI
UnicodeUnicode--compliance means:compliance means:
SAP NetWeaver BI can interpret and display Unicode charactersUser interface will be shown in local languageBusiness data can use all languages in parallel
SAP NetWeaver BI can extract data from source systems with specific codepages (Non-Unicode or Unicode)
SAP NetWeaver BI can extract data from SAP source system running mixedcode pages (MDMP)
Interfaces to 3rd party systems support correct code page conversion
Unicode Platform Availability
Which platforms support Unicode?Which platforms support Unicode?Database Management Systems
Available DB management systemsMS SQL-ServerDB2/UDB V8maxDB 7.5+ORACLE 9.2DB2/AS400 V5 R2
Delayed DB management systemsDB2/zSeries, available as of 2004Informix: no Unicode support planned
Operating Systems restrictionsReliant Unix is not Unicode-enabledOS/390 available as of 2004
Unicode in SAP NetWeaver BI 7.0 and later
SAP NetWeaver BI 7.0 is fully Unicode compliantSAP NetWeaver BI 7.0 is fully Unicode compliantSAP NetWeaver BI Server
Fully Unicode compliantNote: when loading from a double-byte (i.e. Non-Unicode) SAP ERP source system(release <= 4.6D) into Unicode SAP NetWeaver BI, so-called container misalignmentsmay occur (see SAP notes 673533 and 510882 for more information).
BEx Web AnalyzerFully Unicode compliant (including right-to-left languages)
BEx Analyzer (.NET tools)Fully Unicode compliantException: Right-to-left languages are not supported by all controls
SAP ERP<= 4.6D NW BI
Non-Unicode Unicode SAP NetWeaver BIProject Solution
Supported Scenarios for SAP BW 3.5
Unicode supprt for SAP BW 3.5/SAP NetWeaverUnicode supprt for SAP BW 3.5/SAP NetWeaver ‘‘0404SAP NetWeaver BI Server (backend)
Load of data in different code pagesDisplay of all data in SAP Gui environment
Unicode Web front endDisplay of all data if front end is configured in Unicode code page
Restrictions for MS Windows-based client tools (BEx Excel front end, WebApplication Designer, Query Designer) – see also SAP note 588480
Object keys must be in US-7-bit-ASCII to ensure access to keys in all logon languages / code pagesThe restrictions only apply to texts, transactional data is not affected.The client is configured in one Non-Unicode code page. Note that also the Windows default code page mustbe switched to this Non-Unicode code page.Only the data in the logon code page are displayed correctly: I.e. a Russian employee can see all Russiantexts, but not the German ones (‘#’ characters will be displayed for special German characters), whereas theGerman employee can see the German texts (but not the Russian ones)English texts (US-7-bit-ASCII) are displayed correctlyA user must not change and save queries, work books, query elements or web templates with Non-US-7-bit-ASCII (English) texts that have been originally created in a different code page. For example, a querycontaining Russian text elements should only be changed in Russian code page.
Front end Restrictions SAP BW 3.5 in Detail– 1 –
Restrictions for MS WindowsRestrictions for MS Windows--based client toolsbased client tools ––Example: BEx Excel FrontendExample: BEx Excel FrontendLogon Language Russian
German characters corrupted
Logon Language GermanRussian characters corrupted
No restrictions in Web FrontendCorrect characters from both code pages – independent from logon language
Front end Restrictions SAP BW 3.5 in Detail– 2 –
BEx Analyzer Excel front endNote that the corrupted text is a pure display matter; the data in the database is alwayscorrect!
Query Designer / Web Application DesignerIn order to avoid corrupted text written back to the database, organizational means must beinstalled. There are basically two options
Queries, query elements, work books, web templates must only contain English texts (US-7-bit-ASCII)Queries, query elements, work books, web templates containing texts (also hidden ones ine.g. restricted key figures or variables) in a specific language / code page must not bechanged by users in a different code page
RTL (Right-to-Left) LanguagesThe Windows frontend components (BEx Analyzer, Query Designer, Web ApplicationDesigner) do not support RTL languages (like Hebrew, Arabian). However BEx Web doessupport them.
Keys in US-7-bit-ASCIIKeys (of any InfoObject) must be in US-7-bit-ASCII to enable users in every code page toenter the key in a search criterion; if Russian characters were allowed in keys, a German usercould never enter these and, hence, never select these
Supplementation
Business ScenarioYou run your SAP NetWeaver BI in different languages, but the availability of translations forall languages cannot be guaranteed. Hence, you want to define a standard language. Thisstandard language is used to generate all missing translations.Example: you want to use English material texts in case the Japanese ones are not available.
SupplementationThe text tables can be supplemented with a standard language.
UsageStart in transaction SMLT and check by double-clicking on the (target) language (e.g.Japanese), if the supplementation language has been maintained (e.g. English).Then use the menu path Language Special Actions Supplementation (Expert) in order toselect all affected tables (usually all language-dependent /BIC/T* and /BI0/T*-tables – thosetables are cross-client class A tables)If you want to run supplementation periodically, you can use reportRSTLAN_SUPPLEMENT_PERIODIC (parameter: preceding supplementation run) and includeit into your process chain.Check SAP note 111750 for supplementation of German language
Unicode Installation and Conversion
How can I make SAP NetWeaver BI UnicodeHow can I make SAP NetWeaver BI Unicode--compliant?compliant?Delivery
You can choose between Non-Unicode and Unicode installationNote: Unicode installation requires more hardware resources (depending on DB platform)
Installation modesNew InstallConversion of an existing SAP NetWeaver BI
Before the conversion, upgrade your SAP NetWeaver BI to SAP BW 3.5 (or later release)R3LOAD converts an existing SAP BW 3.5 (or later release) automatically by exporting theDB, realigning the DB and importing the DB againNote that the Unicode Conversion is a pilot project as part of a BW System Copy (see note543715)Customer-developed programs (variable exits, virtual characteristics/key figures,transformation rules, table interface etc.) must be in line with the Unicode rulesThe duration of a conversion depends on the size of the existing database
© SAP 2007 / Page 23
Agenda
1. Unicode in General2. Excursus: MDMP3. Unicode support of SAP NetWeaver BI4. Interfaces to SAP Systems5. Interfaces to non-SAP Systems
Interfaces to SAP Source Systems
Which SAP Source System Types exist ?Which SAP Source System Types exist ?Single code page systems
One single code page can include several languages (e.g., Latin-1 contains German, French,Spanish, Italian etc.)A Unicode source system is also a single code page system
Mixed code page systemsSAP source systems with several code pages (usually: MDMP)
How does the conversion work?How does the conversion work?RFC-Calls are enhanced by source and target code page parameters; they transformcharacters „on the fly“ to the correct code pageThe RFC call derives the source code page information from the language keyNote: English texts (US-7-bit-ASCII) is converted correctly in ALL code pagesFor mixed code page systems, language information on RECORD level defines the conversion
PROJECT based solution
Connection Scenarios – 1 –
Which systems can I connect to ?Which systems can I connect to ?Overview on the supported connections
SAPNetWeaverBI Unicode
SAP BW 3.xNon-Unicode
SAP BW 2.xNon-Unicode
SAP sourceUnicode
Ok ok, but someinformationmight be lost
notrecommended
Project! Someinformationmight be lost
notrecommended
SAP sourcenon-Unicode
Ok (fromdouble-bytesources: onprojectbasis) (*)
ok ok
SAP sourceMDMP
Project! ok, but someinformationmight be lost
notrecommended
ok, but someinformationmight be lost
notrecommended
(*) seeslide 17
Connection Scenarios – 2 –
What are the prerequisites for a proper connection ?What are the prerequisites for a proper connection ?Unicode SAP NetWeaver BI systems and Unicode / Non-Unicode SAP
source systemsSAP BI-Service API 3.0B SP3 resp. PI_BASIS 2002_1_620 SP3 is required to connect theUnicode source system with SAP BI
e.g. R/3 BI
Unicode / Non-Unicode
Service API 3.0B SP3 or PI_BASIS2002_1_620 SP3
Unicode BI
See slide 17 for restrictions fordouble-byte languages from
SAP ERP <= 4.6D. No issuesas of release 4.7.
Connection Scenarios – 3 –
Non-Unicode SAP BW 3.x systems and Unicode SAP source systemsDuring conversion some information might be lost. The following restrictions apply
Characters might be corrupted as the target system does not know all Unicode charactersIn multi-byte code pages (Asian languages), suffix characters could get cut off during theextraction, as the bigger Unicode containers must be mapped to the target non-Unicodecontainers
Non-Unicode SAP BW 2.x systems and Unicode SAP source systemsSAP BW 2.x as non-Unicode BW system is not generally released to extract data from Unicodesource systems; in special scenarios, this connection might be possible on a project basis
e.g. R/3 BI
Unicode Non-Unicode SAP BW 3.x
e.g. R/3 BI
Unicode Non-Unicode SAP BW 2.xProject Solution
Restrictions apply
Connection Scenarios – 4 –
What about other interfaces to SAP source systems ?What about other interfaces to SAP source systems ?The same connection scenarios apply for transient interfaces to SAP
source systemsReport-Report-InterfaceRemoteCubesVirtual InfoProviders to SAP source systems
Technical Settings for Unicode
Be sure to set up your system according to the followingBe sure to set up your system according to the followingrecommendationsrecommendations
LanguagesYou can only extract languages to SAP NetWeaver BI, which are defined in the basis parameterZCSA/INSTALLED_LANGUAGES (language vector)
© SAP 2007 / Page 30
Agenda
1. Unicode in General2. Excursus: MDMP3. Unicode support of SAP NetWeaver BI4. Interfaces to SAP Systems5. Interfaces to non-SAP Systems
Unicode-compliance of Interfaces
Which source system code pages are supported for NonWhich source system code pages are supported for Non--SAPSAPsources?sources?
For Non-SAP sources, it is difficult to determine the source code page.In most cases, only English texts (7-bit-US-ASCII) or Unicode texts are converted correctlyExamples
Flat FileDB Connect (Same restrictions apply like for all MultiConnect connections – independentfrom Unicode)UD ConnectVirtual InfoProviders to Non-SAP sources
Unicode-compliance of BAPIs and 3rd PartyTools
What about Unicode with respect to BWWhat about Unicode with respect to BW--BAPIBAPI’’s and 3s and 3rdrd PartyPartyTools?Tools?
Loading data into BW via Staging-BAPI (and 3rd party ETL-tools)This interface is released for the following sources: (see SAP note 765543)
Single-byte code pagesUnicode UTF-16All other code pages, but only for transfer structure of size 250 characters and smaller
This interface is not released for the following sources:Multi-byte code pages with variable byte length (I.e. Asian code pages and UTF-8) ANDTransfer structure is larger than 250 characters
Displaying data in a 3rd party front end via OLAP BAPI, OLE DB forOLAP or XMLA
These interfaces are currently released as a pilots for Unicode
ETL or frontend partnersAsk our partners for Unicode-compliance of their products
© SAP 2007 / Page 33
Thank you!