ar8

43
New AR: View AR | AR without Proprietary History | States | Assignment | Timetracking Add Attachment AR Text Search | More | CARES Home Assistance Request 1-5874139 Customer Ticket na Short PS sudden traffic reduction on 3G NW Description Current Aug 20: No issue from RNC side, closure recieved Summary State Closed : Solution/Service Provided Outage No Severity 1 Priority 1 Service Request Remote Technical Support Request Type Support Sub-Type Diagnose Category Software Sub-Category No Fault Found Internal No Assignment Events

Upload: yasser-el-sammak

Post on 05-Dec-2015

6 views

Category:

Documents


1 download

DESCRIPTION

AR8

TRANSCRIPT

Page 1: AR8

New AR:  View AR |  AR without Proprietary

History |  States |  Assignment |  Timetracking

Add Attachment

AR Text Search  |  More  |  CARES HomeAssistance Request  1-5874139

Customer Ticket na

Short PS sudden traffic reduction on 3G NW

Description

Current Aug 20: No issue from RNC side, closure recieved

Summary

State Closed : Solution/Service Provided

Outage No

Severity 1

Priority 1

Service Request Remote Technical Support

Request Type Support

Sub-Type Diagnose

Category Software

Sub-Category No Fault Found

Internal No

Assignment Events

Ask Alcatel-Lucent

Page 2: AR8

Product

Product 9370 RNC

Model 9370 RNC 1500

Version LR13.3.W

No Change Request is linked to this AR

Product Location

Site ETISALAT ABU DHABI

Instance 9370RNC-Etisalat : In Service

Site Company Emirates Telecommunications Corporation

City Abu Dhabi

Country United Arab Emirates

Contact

Name Karim IBRAHIM ABD EL NABY

Company Alcatel-Lucent : Open on behalf of Customer

Phone 971 56 354 4568 Email [email protected]

Request Method Email-Eng

Dates

Occurred 13-Aug-2015 16:28 Time now 29-Aug-2015 09:36 (GMT+4)

Reported 13-Aug-2015 16:28 AR Created 13-Aug-2015 16:33

Service Start 13-Aug-2015 16:28

Next Customer Contact 27-Aug-2015 10:30

Responded 13-Aug-2015 16:58 Respond Target 13-Aug-2015 16:58 SA - Calculated

Restored 14-Aug-2015 13:30 Restore Target 21-Aug-2015 07:09 SA - Calculated

Resolved 24-Aug-2015 11:23 Resolve Target 19-Sep-2015 07:09 SA - Calculated

Targets for Restore & Resolve were extended by 6 days 15 hours

to account for Pending time

Page 3: AR8

Closed 24-Aug-2015 11:24

Last Modified 24-Aug-2015 11:24 Modified By gauravarora

Entitlement

Agreement 242895 (OXIA 627085)

Covered Service TS 24x7 (Gold, Wireless Mobile Access)

Script This request is entitled to service.

People

Owner TSCd-WLS-WCDMA-IN : mohmadi

Assignee CloQ-WLS-WCDMA-vGlobal : gauravarora

Referred 1 NorP-WLS-WCDMA-vGlobal : gauravarora

Referred 2 TEC-WLS-WCDMA-RNC : ayounan

Resolve Group NorP-WLS-WCDMA-vGlobal

Submitter ggenckan

Description

Attachments

[empty]

Showing Investigation and Proprietary logs together in time order Show separately

Investigation Log and Proprietary Log

1. 13-Aug-2015 16:43 ALCATEL-LUCENT PROPRIETARY ggenckan

Outgoing E-mail to Karim (Karim)** CTR ** IBRAHIM ABD EL NABY

Workflow Status: Queued

Subject: Re: Etisalat UAE// PS sudden traffic reduction on 3G NW all

RNCs//1-5874139

Contact: Karim (Karim)** CTR ** IBRAHIM ABD EL NABY

Page 4: AR8

Agent: ggenckan

From: [email protected]

To: "IBRAHIM ABD EL NABY, Karim (Karim)** CTR **"

<[email protected]>

Cc: "MERAD, ABDESSAMAD (ABDESSAMAD)" <[email protected]>,

"ABDEL-HALIM, SAYED (SAYED)" <[email protected]>, "KHEDR,

MAHMOUD (MAHMOUD)" <[email protected]>, "EL-MIDANY, AHMED

(AHMED)" <[email protected]>, "NABIL, EMAN (EMAN)"

<[email protected]>, "ELHAKIM, Mohamed (Mohamed)** CTR **"

<[email protected]>, "AMMAR, Omar (Omar)"

<[email protected]>

Bcc:

Sent on:

Dear Karim,

Thank you for contacting the Alcatel-Lucent Welcome Center.

Your request has been processed and the ticket number for your reference is AR

1-5874139

Please reference that number when contacting Alcatel-Lucent for follow-up.

An Alcatel-Lucent representative will contact you regarding your request.

Kind Regards,

Gencay Genckan

Page 5: AR8

ALCATEL-LUCENT

GLOBAL WELCOME CENTER

Visit OnLine Customer Support for Contact details

Incoming E-mail from Karim (Karim)** CTR ** IBRAHIM ABD EL NABY

Workflow Status: Handled by an agent

Subject: Etisalat UAE// PS sudden traffic reduction on 3G NW all RNCs

Contact: Karim (Karim)** CTR ** IBRAHIM ABD EL NABY

From: "IBRAHIM ABD EL NABY, Karim (Karim)** CTR **"

<[email protected]>

To: ALU Support <[email protected]>

Cc: "MERAD, ABDESSAMAD (ABDESSAMAD)" <[email protected]>,

"ABDEL-HALIM, SAYED (SAYED)" <[email protected]>, "KHEDR,

MAHMOUD (MAHMOUD)" <[email protected]>, "EL-MIDANY, AHMED

(AHMED)" <[email protected]>, "NABIL, EMAN (EMAN)"

<[email protected]>, "ELHAKIM, Mohamed (Mohamed)** CTR **"

<[email protected]>, "AMMAR, Omar (Omar)"

<[email protected]>

Sent on: 13 sierpien 2015 14:28:12 CEST

Hello

PROBLEM DESCRIPTION: PS sudden traffic reduction on 3G NW

PROBLEM IMPACT:

Page 6: AR8

Sudden traffic reduction on all RNCs observed on daily based since 12 –Aug 00:01am

, without any clear reason or alarms, also CS ,RRC, RAB is stable,

We checked with PS core and no action taken from their side, and also they are not

observing any traffic reduction from there, which is abnormal behavior

Please refer and check the AR , AR which may have any correlation, as we had NPO

auto reset yesterday,

There are another AR running to check if any issue on WMS or NPO, and we need from

AR to make sure that no issue on RNCs Level

EXPECTED RESULTS:

RCA and Soln for this issue as the customer is escalating us due to this issue,

Contact Surname : Karim

Contact Given Name : Ibrahim

Contact Phone :+971 56 354 4568

Contact Company : Alcatel-Lucent

Page 7: AR8

Company : Emirates Telecommunications Corporation

Contract Number : 242895

Region : EMEA

Country : United Arab Emirates

City : Abu Dhabi

Site : ETISALAT ABU DHABI

Product Instance : 9370 RNC

Severity : 1

Priority : 1

Product Version : LR13.3.1

Thanks,

BR,

Karim Ibrahim

LTE/3G Integration Professional

Page 8: AR8

Alcatel-Lucent, Dubai, Etisalat Program

M: +971(0)563544568| ONNET: 22341798

Every Success has its network

image001.gif (1,78 KB)

2. 13-Aug-2015 17:44 mohmadi

Update to Current Summary: Aug 13: Under Investigation

3. 13-Aug-2015 17:45 mohmadi

Update to Current Summary: Aug 13: Awaiting Receive Log Files / Technical Info

4. 13-Aug-2015 18:04 ALCATEL-LUCENT PROPRIETARY mohmadi

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Thursday, August 13, 2015 7:33 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE

Subject: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications Corporation/PS

sudden traffic reduction on 3G NW

Page 9: AR8

Hello Karim,

Please share following details for the issue:

CONFIGURATION DETAILS :

1. Hardware configuration:

2. Software configuration:

>OAM:

> RNC:

> NodeB:

HISTORY:

1. Which & how many NEs impacted ?( How many RNC's)

2. When the issue has started.

3. What actions were performed before or in time when the issue begun (feature

activation, upgrade, 3rd party actions...)

4. What recovery actions have been done & with what results

5 MOP/Procedure that was being used during planned event.

PROBLEM IMPACT:

1. Frequency of the problem (once, x times, sporadic, systematic):

2. Impact (drop call, board reset, KPI degradation, redundancy, None):

3. What is PS CSR KPI %.

4. What is the % of PS KPI degradation with PS KPI details?

5. How much is PS KPI sudden traffic reduction.

6. Was the impact on all node B's connected to impacted RNC’s?

7) Any observation from Core side KPI for PS drop?

Page 10: AR8

LOGS REQUIREMENT:

1) HFB of the impacted RNC before 2 days from issue to till date.

2) RNC Log collection full for the impacted RNC

3) RNC Health check of the impacted RNC.

4) KPI reports of degradation for the impacted RNC.

5) Was drive test performed and was PS call drop observed?

6) Was any CTG/CTB trace taken during the duration of the problem?

Please upload the logs on china ftp server path:

/home/wcdma/Etisalat_UAE/1-5874139

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

5. 13-Aug-2015 20:14 ALCATEL-LUCENT PROPRIETARY mohmadi

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Thursday, August 13, 2015 9:43 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Page 11: AR8

Hello Karim,

We are collecting the RNC log-collection for all 6 RNC’s and will update you by

accordingly as we are on data card and due to slow connectivity uploading and

downloading will take time.

+---------------------------------------------------------------------

| Please, select the RNC from which you wish to collect logs

+---------------------------------------------------------------------

|

| 1. RNC239 (UMAINSERV1)

| 2. RNC235 (UMAINSERV1)

| 3. RNC232 (UMAINSERV1)

| 4. RNC231 (UMAINSERV1)

| 5. RNCWCE242 (UMAINSERV1)

| 6. RNCWCE241 (UMAINSERV1)

|---------------------------------------------------------------------

|---> q : quit the tool

+---------------------------------------------------------------------

Please enter your choice: 4

###-- Command File "RNC1500LogCommands.cfg" is specified --###

Page 12: AR8

INFO : Getting Info For RNC : RNC231

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

6. 13-Aug-2015 20:16 ALCATEL-LUCENT PROPRIETARY mohmadi

From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Sent: Thursday, August 13, 2015 8:33 PM

To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Dear Mohamdi

First part we already provided lot of times,

Please involve the TEC as the issue is very urgent,

Below my comments

Page 13: AR8

Fill free to collect all your required logs from the WMS on ramses and share with

us the your session log,

Thanks,

BR,

Karim Ibrahim

LTE/3G Integration Professional

Alcatel-Lucent, Dubai, Etisalat Program

M: +971(0)563544568| ONNET: 22341798

Every Success has its network

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Thursday, August 13, 2015 6:03 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE

Subject: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications Corporation/PS

sudden traffic reduction on 3G NW

Hello Karim,

Please share following details for the issue:

CONFIGURATION DETAILS :

1. Hardware configuration:

2. Software configuration:

Page 14: AR8

>OAM:

> RNC:

> NodeB:

HISTORY:

1. Which & how many NEs impacted ?( How many RNC's) All RNCs, escpailly

RNC231,242,

2. When the issue has started. 12 Aug 00:01Am

3. What actions were performed before or in time when the issue begun (feature

activation, upgrade, 3rd party actions...) , No actions, only migration from

legacy to WCE RNCs

4. What recovery actions have been done & with what results No action taken

5 MOP/Procedure that was being used during planned event.NO

PROBLEM IMPACT:

1. Frequency of the problem (once, x times, sporadic, systematic): Continues

since the 12 Augn 00:01 am

2. Impact (drop call, board reset, KPI degradation, redundancy, None): PS

traffic reduction

3. What is PS CSR KPI %.nothing, attached detailed reports

4. What is the % of PS KPI degradation with PS KPI details? Check the reports

5. How much is PS KPI sudden traffic reduction. 500G

6. Was the impact on all node B's connected to impacted RNC’s?traffic

reducation

7) Any observation from Core side KPI for PS drop? No observation from core,

attached

Page 15: AR8

LOGS REQUIREMENT:

1) HFB of the impacted RNC before 2 days from issue to till date.

2) RNC Log collection full for the impacted RNC

3) RNC Health check of the impacted RNC.

4) KPI reports of degradation for the impacted RNC. Attached

5) Was drive test performed and was PS call drop observed?NO

6) Was any CTG/CTB trace taken during the duration of the problem? No

Please upload the logs on china ftp server path:

/home/wcdma/Etisalat_UAE/1-5874139

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Thursday, August 13, 2015 9:43 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Page 16: AR8

Hello Karim,

We are collecting the RNC log-collection for all 6 RNC’s and will update you by

accordingly as we are on data card and due to slow connectivity uploading and

downloading will take time.

+---------------------------------------------------------------------

| Please, select the RNC from which you wish to collect logs

+---------------------------------------------------------------------

|

| 1. RNC239 (UMAINSERV1)

| 2. RNC235 (UMAINSERV1)

| 3. RNC232 (UMAINSERV1)

| 4. RNC231 (UMAINSERV1)

| 5. RNCWCE242 (UMAINSERV1)

| 6. RNCWCE241 (UMAINSERV1)

|---------------------------------------------------------------------

|---> q : quit the tool

+---------------------------------------------------------------------

Please enter your choice: 4

###-- Command File "RNC1500LogCommands.cfg" is specified --###

Page 17: AR8

INFO : Getting Info For RNC : RNC231

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

7. 14-Aug-2015 08:41 ALCATEL-LUCENT PROPRIETARY mohmadi

Time Tracking Entry Added - IR01-TSA3

8. 14-Aug-2015 11:14 mohmadi

Update to Current Summary: Aug 14: Awaiting Receive Log Files / Technical Info

9. 14-Aug-2015 11:14 ALCATEL-LUCENT PROPRIETARY mohmadi

From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Sent: Thursday, August 13, 2015 10:47 PM

To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Page 18: AR8

Corporation/PS sudden traffic reduction on 3G NW

Hello Mohamdi

Thanks a lot for your help, please note the libe RNCs are

RNC235,RNC231,RNCWCE241,RNCWCE242,

Thanks,

BR,

Karim Ibrahim

LTE/3G Integration Professional

Alcatel-Lucent, Dubai, Etisalat Program

M: +971(0)563544568| ONNET: 22341798

Every Success has its network

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Friday, August 14, 2015 12:43 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Page 19: AR8

Hi Karim,

Let us know as KPI are fine now then can we close this AR.

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

10. 14-Aug-2015 12:02 mohmadi

Update to Current Summary: Aug 14: KPI are ok , awaiting closure

11. 14-Aug-2015 12:02 ALCATEL-LUCENT PROPRIETARY mohmadi

Time Tracking Entry Added - IR01-TSA3

12. 14-Aug-2015 12:03 mohmadi

Update to Current Summary: Aug 14: KPI are ok after WMS process stop/start

performed (1-5873971) , awaiting closure

13. 14-Aug-2015 13:28 ALCATEL-LUCENT PROPRIETARY mohmadi

From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Sent: Friday, August 14, 2015 2:46 PM

Page 20: AR8

To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hello Mohamdi

Issue still not solved totally, Please did you find something from the rnclogs?

Thanks,

BR,

Karim Ibrahim

LTE/3G Integration Professional

Alcatel-Lucent, Dubai, Etisalat Program

M: +971(0)563544568| ONNET: 22341798

Every Success has its network

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Friday, August 14, 2015 2:57 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

Page 21: AR8

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hi Karim,

At present nothing abnormal found from Healthcheck and RNC logcollection.

As this issue is seen in all RNC so it is suspected that the common connectivity

should be checked properly, and I see that you are working on WMS part so first

kindly make WMS error free and monitor the KPI and let us know the outcome, so

then once again we can check from RNC side.

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

14. 14-Aug-2015 13:35 ALCATEL-LUCENT PROPRIETARY mohmadi

From: ARORA, Gaurav (Gaurav)** CTR **

Sent: Friday, August 14, 2015 3:02 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **; CUTLERYWALA, Mohmadi (Mohmadi)**

CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

Page 22: AR8

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hi Karim ,

The issue is being tracked on WMS AR . As per the attached mail it was advised

to put a fix earlier as well . Any reason for not updating it .

Thanks and regards

Gaurav Arora

+91 9503055691

[email protected]

15. 14-Aug-2015 13:41 ALCATEL-LUCENT PROPRIETARY mohmadi

From: ARORA, Gaurav (Gaurav)** CTR **

Sent: Friday, August 14, 2015 3:11 PM

To: ARORA, Gaurav (Gaurav)** CTR **; IBRAHIM ABD EL NABY, Karim (Karim)** CTR **;

CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar);

Page 23: AR8

INDIA-WCDMA-WMS-NPO-BE

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

+

Thanks and regards

Gaurav Arora

+91 9503055691

[email protected]

16. 14-Aug-2015 15:20 ALCATEL-LUCENT PROPRIETARY mohmadi

Time Tracking Entry Added - IR01-TSA3

17. 15-Aug-2015 09:43 mohmadi

Update to Current Summary: Aug 15: KPI are ok after WMS process stop/start

performed (1-5873971) , awaiting closure

18. 15-Aug-2015 09:43 ALCATEL-LUCENT PROPRIETARY mohmadi

Hello TEC, Escalating the issue for your help.

Product: 9370 RNC

SW Version: LR13.3.W

Page 24: AR8

Customer: Emirates Telecommunications Corporation

Problem Short Description

PS sudden traffic reduction on 3G NW (1-5874139)

1. Detailed Description

Site team is observing PS sudden traffic reduction on 3G NW for all RNC's (4 RNC's

are 9370 and two RNC's are WCE RNC).Issue seen satrted Continues since the 12 Augn

00:01 am. There was PS traffic reduction of about 500G.

They have also raised AR at WMS side (AR 1-5873971) and they have found some

errors and they have already asked customer to install the patch for it since

long, but still that patch installation is pending..

As Site team have critical decision to take early next week and as all RNC are

involved..

All Logs are available on FTP Server 135.240.25.160 wftp/wftp under

/home/wcdma/Etisalat_UAE/1-5874139

2. All the steps taken before AR escalated to ALU

As above.

3. All the steps taken before the issue observed by the engineer, who reported the

AR

N/A

4. Activities After AR was escalated to L2 Technical Support.

Page 25: AR8

As checked, RNC health check and RNC log collection and no suspected abnormal

found.

Escalating the issue as Site team have critical decision to take early next week.

Required solution Root cause for the issue.

Just for

5. Current Status

Required solution and RCA.

6. Severity of this issue and its impact on the customer's organization

• Critical

7. Attachments

• Escalation Note.

8. Expectations from the TEC:

Required solution and RCA.

Thanks,

Mohmadi

19. 15-Aug-2015 09:49 mohmadi

Update to Current Summary: Aug 15: Information/Action Requested

20. 15-Aug-2015 09:56 ALCATEL-LUCENT PROPRIETARY mohmadi

Page 26: AR8

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Saturday, August 15, 2015 11:17 AM

To: ROY, Paul (Paul)** CTR **; DAS, Sunil (Sunil)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE

Subject: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications Corporation/PS

sudden traffic reduction on 3G NW

Hello Paul/Sunil,

This is for your information that we have escalated this AR to your workgroup

(TEC-WLS-WCDMA-RNC). Please assign someone to have support on this.

Escalation note attached for your reference.

All Logs are available on FTP Server 135.240.25.160 wftp/wftp under

/home/wcdma/Etisalat_UAE/1-5874139

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

21. 17-Aug-2015 12:17 mohmadi

Update to Current Summary: Aug 17: Information/Action Requested

Page 27: AR8

22. 17-Aug-2015 12:17 ALCATEL-LUCENT PROPRIETARY mohmadi

Hi TEC, just to share more update on it.:

As WMS team have found some alarms on only two WCE RNC, please check is it related

to this issue?

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrCNodeMediationContext.cc:965 |

Thr:35 | ERR | component-name | class-name | CId:60022649

Wrong instance of class:R/EM/IpIf/BwPool

observation file name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150814150000.gz

ne Unique Id:RNCWCE241

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrCNodeDNHandler.cc:723 | Thr:35 |

ERR | component-name | class-name | CId:60022649

Can NOT get the userLabel for : [RNC/RNCWCE241 RNCEquipment/0 INode/0]

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrDNHandler.cc:284 | Thr:35 | ERR |

component-name | class-name | CId:60022649

Failed to convert RAN DN: RNC/1 RNCEquipment/0 INode/0 EM/0 IpIf/180 BwPool/1 to

Operator DN

2015/08/14-19:02:15,613+04:00 | CP_APM_UTRAN_xdrMediationImpl.cc:335 | Thr:38 |

Page 28: AR8

ERR | component-name | class-name | CId:60022093

Some counters under _classIdx_n 37 can't be processed successfully

observation file name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150815023000.gz

Line 5493: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150815023000.gz

Line 5505: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150815023000.gz

Line 5517: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150815023000.gz

Line 5539: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE242/20150815023000.gz

Line 5551: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE242/20150815023000.gz

Line 5563: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE242/20150815023000.gz

Line 5575: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE242/20150815023000.gz

Line 5587: observation file

name:/opt/nortel/data/utran/gpo/IN/RNCWCE242/20150815023000.gz

Thanks,

Mohmadi

23. 18-Aug-2015 09:37 mohmadi

Update to Current Summary: Aug 18: Information/Action Requested

Page 29: AR8

24. 18-Aug-2015 09:37 ALCATEL-LUCENT PROPRIETARY mohmadi

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Tuesday, August 18, 2015 11:07 AM

To: ROY, Paul (Paul)** CTR **; DAS, Sunil (Sunil)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hello Paul/Sunil,

Any update on assignment of this AR please.

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

25. 18-Aug-2015 19:15 ALCATEL-LUCENT PROPRIETARY ayounan

TEC Update:

==========

Hi Mohmadi,

i'm not sure what is requested from RNC TEC in this AR.

Page 30: AR8

in your last update when ruting the AR to our queue, youare asking about the below

error seen in WMS

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrCNodeMediationContext.cc:965 |

Thr:35 | ERR | component-name | class-name | CId:60022649

Wrong instance of class:R/EM/IpIf/BwPool

observation file name:/opt/nortel/data/utran/gpo/IN/RNCWCE241/20150814150000.gz

ne Unique Id:RNCWCE241

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrCNodeDNHandler.cc:723 | Thr:35 |

ERR | component-name | class-name | CId:60022649

Can NOT get the userLabel for : [RNC/RNCWCE241 RNCEquipment/0 INode/0]

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrDNHandler.cc:284 | Thr:35 | ERR |

component-name | class-name | CId:60022649

Failed to convert RAN DN: RNC/1 RNCEquipment/0 INode/0 EM/0 IpIf/180 BwPool/1 to

Operator DN

2015/08/14-19:02:15,613+04:00 | CP_APM_UTRAN_xdrMediationImpl.cc:335 | Thr:38 |

ERR | component-name | class-name | CId:60022093

Some counters under _classIdx_n 37 can't be processed successfully

+ AR 1-5878714: Repeated errors on RNCWCE241&RNCWCE242---> is tracking the

above error already

Page 31: AR8

+ AR 1-5873971: PS sudden traffic reduction on 3G NW ---> is tracking the Error

occurred: Exception DataInfo_Exception from package WISCI_DATAINFO ("Out of

bound exception ---> fixed by WMS CR 01545832

so please clarify what is needed from RNC TEC to investigate in this AR if it is

not the error of your last update which is already tracked by AR 1-5878714

Regards,

Amir

26. 18-Aug-2015 19:16 ALCATEL-LUCENT PROPRIETARY ayounan

Time Tracking Entry Added - TSA3 diagnosis

27. 18-Aug-2015 23:04 ALCATEL-LUCENT PROPRIETARY ayounan

TEC Update:

===========

below is the latest update from AR 1-5878714: Repeated errors on

RNCWCE241&RNCWCE242---> is tracking the above error already

if this is what is requested from this AR as well , then this AR can be closed

if anything else is requested from this AR please specify.

and i'll put the AR in pending till i know wht is requested to track with this AR.

Page 32: AR8

11. 18-Aug-2015 13:42 ALCATEL-LUCENT PROPRIETARY gserise

Hello Sudhakar,

i looked at the traces in FMK_PM_server-20023.tra and the error you are referring

to is purely related xdr format PM files decoding by WMS process

FMK_PM_server-20023.

It seems the FMK_PM_server-20023 is trying to decode Inode counters in WCE RNC

counter files.

Therefore it is trying to map the WCE RNC Inode object with what it has in its

database.

2015/08/14-19:05:16,959+04:00 | CP_APM_UTRAN_xdrCNodeDNHandler.cc:723 | Thr:35 |

ERR | component-name | class-name | CId:60022649

Can NOT get the userLabel for : [RNC/RNCWCE241 RNCEquipment/0 INode/0]

the problem is Inode is a 9370 RNC object (Legacy RNC). In the WCE 9771 RNC, the

INode was replaced with another object called WNode.

On 9370 , 2 Stats file are generated, one for Cnode and one for INode.

On WCE only one is generated containing counters of both CNode and WNode.

This is a non sense to look for object Inode in a WCE type RNC.

However, despite the presence of error messages in the processing of WCE stats

Page 33: AR8

file in FMK_PM_server-20023.tra, there is absolutely no impact whatsoever to the

management of WCE RNC stats/FM and CM by WMS server.

This is a WMS bug. This AR should be escalated to WMS team. this issue can be

easily reproduced in a lab.

Wrong Product, Please escalate to WMS Product team.

Best Regards,

Ghislain Serise

28. 19-Aug-2015 09:11 mohmadi

Update to Current Summary: Aug 19: Information/Action Requested

29. 19-Aug-2015 09:11 ALCATEL-LUCENT PROPRIETARY mohmadi

Hello Amir,

Actually Site team raised two ARs for this same subject line:

1) AR from Utran side AR 1-5874139 and another 2) from WMS side AR 1-5873971.

From Utran side we were asked to investigate why there is sudden PS traffic

Page 34: AR8

reduction on 3G NW for all RNC's. From L2 perspective from Utran side we checked

and found no such issue from RNC side.

The issue is mostly from WMS side as its impacting all the RNC's and they found

some alarms on it.

The reason for this AR escalation is, as there is a huge pressure from customer on

this issue and to just verify and check once from your side.

So let me know your view on it.

Thanks,

Mohmadi.

30. 19-Aug-2015 19:36 ALCATEL-LUCENT PROPRIETARY ayounan

Hi Mohmadi,

i still dont see why we still have this AR

i see issue is already handled with the 2 other ARs (1-5873971 & 1-5878714 )

and the latest update from Karim is as below in AR 1-5873971

so i think at this point you can request a closure for this AR.

From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Sent: Tuesday, August 18, 2015 2:48 PM

Page 35: AR8

To: VAIDYA, Deepak (Deepak)** CTR **

Cc: SHAKAR, Pravin Kumar (Pravin Kumar)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD);

BERIDY, AHMED (AHMED); WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; WAGEEH, WAEL

(WAEL)** CTR **; ELHAKIM, Mohamed (Mohamed)** CTR **; CHOWDARY, Ajay (Ajay)** CTR

**; NADIR, HAMMAD (HAMMAD); INDIA-WCDMA-WMS-NPO-BE; NABIL, EMAN (EMAN);

INDIA-WCDMA-UTRAN-BE; ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); REDA,

RAMY (RAMY); DAS, Sunil (Sunil)** CTR **; ARORA, Gaurav (Gaurav)** CTR **;

CHATTERJEE, Nilanjan (Nilanjan)** CTR **; KHEDR, MAHMOUD (MAHMOUD)

Subject: RE: RE : RE : AR // 1-5873971 // Major // 9959 NPO // Emirates telecomm

// PS sudden traffic reduction on 3G NW

Hello Deepack

Please note the traffic start to come back to normal after core team action,

Please all logs before/after still valid on WMS , thanks to FTP them directly from

ramses,

Thanks to share with us your analysis and findings from the requested logs,

Thanks,

BR,

Karim Ibrahim

LTE/3G Integration Professional

Page 36: AR8

Alcatel-Lucent, Dubai, Etisalat Program

M: +971(0)563544568| ONNET: 22341798

Every Success has its network

Time Tracking Entry Added - TSA3 diagnosis

31. 20-Aug-2015 09:16 mohmadi

Update to Current Summary: Aug 20: No issue from RNC side, awaiting closure.

32. 20-Aug-2015 09:16 ALCATEL-LUCENT PROPRIETARY mohmadi

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Tuesday, August 18, 2015 3:29 PM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hello Karim,

Any update, Please let us know if we can close the AR.

Page 37: AR8

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

From: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Sent: Thursday, August 20, 2015 10:46 AM

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hello Karim,

Can we close this AR , as TEC also confirmed no issue from RNC side.

Thanks,

Mohmadi Cutlerywala,

Mobile:+91 9870327804.

Email ID: [email protected]

33. 24-Aug-2015 11:19 gauravarora

Update to Current Summary: Aug 20: No issue from RNC side, closure recieved

Page 38: AR8

34. 24-Aug-2015 11:19 ALCATEL-LUCENT PROPRIETARY gauravarora

From: [email protected]

[mailto:[email protected]] On Behalf Of IBRAHIM

ABD EL NABY, Karim (Karim)** CTR **

Sent: Monday, August 24, 2015 12:39 PM

To: ARORA, Gaurav (Gaurav)** CTR **; CUTLERYWALA, Mohmadi (Mohmadi)** CTR **

Cc: INDIA-WCDMA-UTRAN-BE; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD

(MAHMOUD); ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED); BERIDY, AHMED

(AHMED); NABIL, EMAN (EMAN); MERAD, ABDESSAMAD (ABDESSAMAD); AMMAR, Omar (Omar)

Subject: RE: AR 1-5874139/Priority/9370 RNC/Emirates Telecommunications

Corporation/PS sudden traffic reduction on 3G NW

Hello Arora,

You can close it.

Thanks,

35. 24-Aug-2015 11:20 ALCATEL-LUCENT PROPRIETARY gauravarora

Time Tracking Entry Added - IR01-TSA1

Add to log

Resolution

Page 39: AR8

NO issues seen from RNC end . Huawei Core action on site was the main cause of

the PS degradation .

End of Assistance Request  1-5874139

New AR:  View AR |  AR without Proprietary |  OLCS

History |  States |  Assignment |  Timetracking

View Attachments |  Add Attachment

AR Text Search  |  More  |  CARES Home