Tuesday, November 22, 2016

adop cutover phase status showing failed in the cloned instance.


Run the following steps:


Node Name       Node Type       Phase           Status          Started                        Finished                       Elapsed
--------------- --------------- --------------- --------------- ------------------------------ ------------------------------ ------------
xxxx12          master          PREPARE         COMPLETED       22-NOV-16 04:00:33 +08:00      22-NOV-16 04:06:20 +08:00      0:05:47
                                APPLY           COMPLETED       22-NOV-16 04:44:02 +08:00      22-NOV-16 05:01:25 +08:00      0:17:23
                                FINALIZE        COMPLETED       22-NOV-16 01:11:06 +08:00      22-NOV-16 01:11:17 +08:00      0:00:11
                                CUTOVER         FAILED          22-NOV-16 01:26:09 +08:00      22-NOV-16 03:19:33 +08:00      1:53:24
                                CLEANUP         NOT STARTED

File System Synchronization Used in this Patching Cycle: Full

File System Information:
========================

File System: SINGLE NODE INSTANCE

Node Name   Node Type   Current Base                   Other Base                     Non Editioned Base             Inst Top
----------- ----------- ------------------------------ ------------------------------ ------------------------------ ------------------------------
xxxx12      master      /u01/appluat/appluat/fs2            /u01/appluat/appluat/fs1            /u01/appluat/appluat/fs_ne          /u01/appluat/appluat/fs2/inst/apps/
                                                                                                                     appluat_xxxx
Custom File System Synchronization script: /u01/appluat/appluat/fs_ne/EBSapps/appl/ad/custom/adop_sync.drv


Solution:
[appluat@XXXXXX PATCH]$ adop phase=cutover mtrestart=no

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

Please wait. Validating credentials...
RUN file system context file: /u01/appluat/UAT/fs1/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml

PATCH file system context file: /u01/appluat/UAT/fs2/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml
Execute SYSTEM command : df /u01/appluat/UAT/fs2

Worker count determination...

Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.
[EVENT]     [START 2016/02/23 22:58:56] Performing database sanity checks
[EVENT]     [END   2016/02/23 22:58:57] Finished performing database sanity checks
  There is already a session which is incomplete. Details are:
        Session Id            :   12
        Prepare phase status  :   COMPLETED
        Apply phase status    :   COMPLETED
        Cutover  phase status :   RUNNING
        Abort phase status    :   NOT APPLICABLE
        Session status        :   FAILED
  Will continue with previous session
  [START 2016/02/23 22:58:58] adzdoptl.pl run
    ADOP Session ID: 12
    Phase: cutover
    Log file: /u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/adop_20160223_225831.log
    [EVENT]     [START 2016/02/23 22:58:58] Calling task: "Executing txkADOPValidation script on XXXXXX"; script: "/u01/appluat/UAT/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl"; args: " -contextfile=/u01/appluat/UAT/fs1/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml -patchctxfile=/u01/appluat/UAT/fs2/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml -phase=cutover -logloc=/u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/cutover_20160223_225831/UAT_XXXXXX -promptmsg=hide"
      [EVENT]     [START 2016/02/23 22:58:58] Executing txkADOPValidation script on XXXXXX
        [EVENT]     Calling: /u01/appluat/UAT/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl
Oracle E-Business Suite Online Patching
RUN edition environment variables have been set.


calling: /u01/appluat/UAT/fs1/FMW_Home/webtier/perl/bin/perl /u01/appluat/UAT/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkValidateSetup.pl -contextfile=/u01/appluat/UAT/fs1/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml -patchctxfile=/u01/appluat/UAT/fs2/inst/apps/UAT_XXXXXX/appl/admin/UAT_XXXXXX.xml -logloc=/u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/cutover_20160223_225831/UAT_XXXXXX -phase=cutover -promptmsg=hide

Program : txkValidateSetup.pl started @ Tue Feb 23 22:59:01 2016

*** Log File = /u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/cutover_20160223_225831/UAT_XXXXXX/ADOPValidations_Tue_Feb_23_22_59_01_2016.log

SETUP VALIDATION is in progress. This may take few minutes to complete.

Completed execution of all tests.

=========================== START OF VALIDATION REPORT ====================


Validation Results for Node: XXXXXX
---------------------------------------
All validations were successful, please check /u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/cutover_20160223_225831/UAT_XXXXXX/ADOPValidations_Tue_Feb_23_22_59_01_2016.log

=========================== END OF VALIDATION REPORT ======================
      [EVENT]     [END   2016/02/23 22:59:05] Executing txkADOPValidation script on XXXXXX
    [EVENT]     [END   2016/02/23 22:59:05] Executing txkADOPValidation script on XXXXXX
    [EVENT]     [START 2016/02/23 22:59:05] Checking if all nodes were synchronized in the previous apply phase
    [EVENT]     [END   2016/02/23 22:59:05] Checking if all nodes were synchronized in the previous apply phase
    [START 2016/02/23 22:59:05] cutover phase
      [EVENT]     [START 2016/02/23 22:59:07] Performing Cutover Phase steps
        User specified mtrestart=no, hence middle tier service startup will be skipped.
        User would need to bring up the services manually.
      [EVENT]     [END   2016/02/23 22:59:08] Performing Cutover Phase steps
    [END   2016/02/23 22:59:08] cutover phase
    [START 2016/02/23 22:59:08] Generating Post Cutover Reports
    [END   2016/02/23 22:59:08] Generating Post Cutover Reports
    [EVENT]     [START 2016/02/23 22:59:09] Generating AD_ZD_LOGS Report
      [EVENT]     Report: /u01/appluat/UAT/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql


      [EVENT]     Output: /u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/cutover_20160223_225831/UAT_XXXXXX/adzdshowlog.out
    [EVENT]     [END   2016/02/23 22:59:09] Generating AD_ZD_LOGS Report

Broadcast message from appluat@XXXXXX.myoracle.com (Tue Feb 23 22:59:09 2016):

UAT environment has changed.
All users must re-source the environment using below command:
source /u01/appluat/UAT/EBSapps.env run|patch
  [END   2016/02/23 22:59:09] adzdoptl.pl run
  adop phase=cutover - Completed Successfully

  Log file: /u01/appluat/UAT/fs_ne/EBSapps/log/adop/12/adop_20160223_225831.log

adop exiting with status = 0 (Success)
==============================


Case2:

ADOP Cutover Failed 

Problem Description 
===========
We were trying to apply hrms patches on our test environment, where cutover phase status showing failed. 
However we noticed fs1 & fs2 got swapped but stauts showing FAILED. Please suggest. 

Solution:
=====

1. Ensure that you have taken a backup of your system before applying the recommended steps. 

2. Please take backup of context files from both run fs and patch fs from application tier. 

3. Edit both context files and for line: 

oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-d64 -Xms512m -Xmx512m -XX:CompileThreshold=8000 -XX:PermSize=128m 

replace it with: 

oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-d64 -Xms512m -Xmx2048m -XX:CompileThreshold=8000 -XX:PermSize=128m 

save. 

4. Reboot the server. When the server is rebooted, the changes will be activated at system level and the oacore server will take the new Xmx settings. 
 Obs: there is no need to run autoconfig -- as these are OS system settings for OACORE startup. 

5. Restart cutover phase with option: 

adop phase=cutover mtrestart=no 

Obs: this will continue the cutover phase that previously failed, until completion 

6. After the cutover completes, you can restart application tier services manually on the new run fs. 

cd $ADMIN_SCRIPTS_HOME 

adstrtal.sh 

No comments:

Post a Comment