Java VM: Java HotSpot(TM) while running autoconfig on the database tier 10g

Issue: While running the autoconfig on the database tier below issue occurred.

Context Value Management will now update the Context file

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  SIGSEGV (0xb) at pc=0xf7a8172a, pid=8033, tid=4160080576

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)

# Problematic frame:

# V  [libjvm.so+0x4eb72a]

#

# An error report file with more information is saved as hs_err_pid8033.log

#

Log details.

Stack: [0xffb43000,0xffd43000),  sp=0xffd3eda4,  free space=2031k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

V  [libjvm.so+0x5056fa]

V  [libjvm.so+0x4320f8]

V  [libjvm.so+0x26a6a6]

V  [libjvm.so+0x28b95f]

C  [libnjni10.so+0x5e4c]  Java_oracle_net_common_NetGetEnv_getLocalHostName+0xd6

j  oracle.net.common.NetGetEnv.getLocalHostName()Ljava/lang/String;+0

j  oracle.net.config.Config.systemName()Ljava/lang/String;+36

j  oracle.net.config.DirectoryService.getSystemObjectPath(Loracle/net/config/Config;)Ljava/lang/String;+6

j  oracle.net.config.DirectoryService.qualifyObjectName(Loracle/net/config/Config;Ljava/lang/String;Z)Ljava/lang/String;+36

j  oracle.net.config.Listener.<init>(Loracle/net/config/Config;Ljava/lang/String;)V+37

v  ~StubRoutines::call_stub

V  [libjvm.so+0x26a17c]

V  [libjvm.so+0x43b3f8]

V  [libjvm.so+0x269faf]

V  [libjvm.so+0x4877cc]

"hs_err_pid11679.log" 375L, 23491C 

Cause: In this case in our server the hostname command was returning null, unfortunately the hostname was disappeared in the linux prompt.

Solution:

=======

[oradr@- ~]$ printenv |less

NLS_SORT=binary

ADJREOPTS=-Xms128M -Xmx512M

HOSTNAME=-

SHELL=/bin/bash

TERM=xterm

HISTSIZE=1000

Added the hostname to the environment as below.

[root@- ~]# /bin/hostname localhost.domain

[root@- ~]# hostname

localhost.domain

[root@- ~]#

[oradr@drdb PRODSTBY_drdb]$ printenv |less

NLS_SORT=binary

ADJREOPTS=-Xms128M -Xmx512M

HOSTNAME=localhost.domain

SHELL=/bin/bash

TERM=xterm

HISTSIZE=1000

Run the autoconfig it should complete.

Grant privileges on Directory to Users in Oracle 11g

SQL> grant all on directory <directory name> to user;

Grant succeeded.

SQL>

Backup archivelogs to the new location in Oracle 11g

run{
allocate channel d1 type disk;
allocate channel d2 type disk;
allocate channel d3 type disk;
backup tag 'DISK_ARCH' as compressed backupset archivelog all format '/data02/oracle/24052015/rman_arch_%s_%d_%D_%U.arc';
release channel d1;
release channel d2;
release channel d3;
}

ORA-01157: ORA-01111: ORA-01110:

There are many reasons for a file being created as UNNAMED or MISSING in the standby database, including insufficient disk space on standby site (or) Improper parameter settings related to file management.

STANDBY_FILE_MANAGEMENT enables or disables automatic standby file management. When automatic standby file management is enabled, operating system file additions and deletions on the primary database are replicated on the standby database.

For example if we add a data file on the Primary when parameter STANDBY_FILE_MANAGEMENT on standby set to MANUAL , While recovery process(MRP) is trying to apply archives, Due to that parameter setting it will create an Unnamed file in $ORACLE_HOME/dbs and it will cause to kill MRP process and Errors will be as below.

1. Check for the files needs to be recovered.

select * from v$recover_file where error like '%FILE%';

Identify on primary of data file 124(Primary Database)

select file#,name from v$datafile where file#='124'

Identify on primary of data file 124(Standby Database)

select file#,name from v$datafile where file#='124'

Crosscheck that no MRP is running and STANDBY_FILE_MANAGEMENT can be enabled once after creating file on standby

ENABLE STANDBY_FILE_MANAGEMENT to MANUAL on the STANDBY server

alter system set standby_file_management=MANUAL scope=both;

show parameter standby_file_management

Create missing datafile on the Standby Server

alter database create datafile '/drhome/oradr/STANDBY/db/tech_st/10.2.0/dbs/UNNAMED00124' as '/drdata/oradr/db/apps_st/data/switchover01.dbf';

ENABLE STANDBY_FILE_MANAGEMENT to MANUAL on the STANDBY server

alter system set standby_file_management=AUTO scope=both;

show parameter standby_file_management

alter database recover managed standby database disconnect from session;

After creating the file, MRP will start applying archives on standby database.

Note: Setting STANDBY_FILE_MANAGEMENT to AUTO causes Oracle to automatically create files on the standby database and, in some cases, overwrite existing files. Care must be taken when setting STANDBY_FILE_MANAGEMENT and DB_FILE_NAME_CONVERT so that existing standby files will not be accidentally overwritten.

PING[ARCk]: Heartbeat failed to connect to standby

Issue: Unable to ship the logs.

Troubleshooting Steps:

1. Verify the listener and tnsnames configuration on PRIMARY and STANDBY.

2. Check the tnsping from the both nodes vice-versa.

3. Connect from primary as below

#sqlplus sys@standby

password: 

the above command should connect to the standby database.

verify using the below command.

select name, open_mode from v$database;

4. Repeat the step 3 from standby node using the connect identifier as primary.

5. If step 1 to 4 are successfully executed.

6. Bounce the services as below.

Bounce Standby Database.

1. stop MRP (Managed Recovery Process) if it is running.

2. Stop listener.

3. Stop database.

4. start listener.

5. Start database.

6. Alter system register

Bounce Primary Database.

1. Disable log shipping

2. Stop listener.

3. Stop database.

4. start listener.

5. Start database 

6. Alter system register.

7. Enable log shipping.

8. Start MRP on the standby node.

DGMGRL> Object "xxxTEST" was not found

Object "prodtest" was not found

sometimes single quotes count and sometime they don't

Issue as below it worked with single quotes.

DGMGRL> show database prodtest;
Object "prodtest" was not found
DGMGRL>
DGMGRL>
DGMGRL> show database 'PRODTEST'

Database
  Name:            PRODTEST
  Role:            PRIMARY
  Enabled:         YES
  Intended State:  ONLINE
  Instance(s):
    PRODTEST

Current status for "PRODTEST":
SUCCESS


DGMGRL>

ORA-16014: log 2 sequence# 26 not archived, no available destinations

SQL> startup pfile='/drdb/oradr/db/tech_st/10.2.0/dbs/initPRODTEST.ora';
ORACLE instance started.

Total System Global Area 1073741824 bytes
Fixed Size                  2089336 bytes
Variable Size             432017032 bytes
Database Buffers          624951296 bytes
Redo Buffers               14684160 bytes
Database mounted.
ORA-16014: log 2 sequence# 26 not archived, no available destinations
ORA-00312: online log 2 thread 1: '/drdb/oradr/db/apps_st/data/log02a.dbf'
ORA-00312: online log 2 thread 1: '/drdb/oradr/db/apps_st/data/log02b.dbf'


SQL> alter database clear unarchived logfile group 2;

Database altered.

SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.
SQL> startup pfile=initPRODTEST.ora
ORACLE instance started.

Total System Global Area 1073741824 bytes
Fixed Size                  2089336 bytes
Variable Size             432017032 bytes
Database Buffers          624951296 bytes
Redo Buffers               14684160 bytes
Database mounted.
Database opened.
SQL> create spfile from pfile;

File created.

SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oradr@crpdmz dbs]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Sun May 17 01:51:14 2015

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>

Discoverer 10g Cloning

Pre-clone Steps:
============

Assuming the EBS is already restored. 

1) Login on Source Discoverer (Solaris) Machine as discoverer user, 

There is no need to shutdown the services

$ export ORACLE_HOME=/u01/product/10gR2

$ export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH

$ cd $ORACLE_HOME/clone/bin

$ chmod 755 prepare_clone.pl

$ ./prepare_clone.pl

Clone log file location: /u01/product/10gR2/clone/logs/clone.log

Error log file location: /u01/product/10gR2/clone/logs/error.log

2. Copy the Discoverer directory to the target machine.

3. Get the ias_admin_old_pwd as it is necessary as input for the clone

Clone Steps

=========

1) Login to target machine as root and set the inventory locations as:

mkdir /etc/oraInventory

chmod -R 777 /etc/oraInventory

chown -R applmgr:dba /etc/oraInventory

touch /etc/oratab

chmod a+rw /etc/oratab

chown applmgr:dba /etc/oratab

echo 'inventory_loc=/etc/oraInventory' > /etc/oraInst.loc

chown applmgr:dba /etc/oraInst.loc

2) Rename the old inventory directory, which is copied from source.

3) Start the clone activity

$ export ORACLE_HOME=/u01/product/10gR2

$ export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH

$ cd $ORACLE_HOME/clone/bin

$ perl clone.pl ORACLE_HOME=$ORACLE_HOME ORACLE_HOME_NAME=discodr -instance TESTDR -ias_admin_old_pwd welcome -ias_admin_new_pwd secret

Note: If the error occurs during the clone, clean the oraInventory directory and restart the clone process, otherwise the clone process will fail.

Post Clone Steps

============

1) Remove Old dbc files from $ORACLE_HOME/discoverer/secure

2) Copy new dbc file from the application node to $ORACLE_HOME/discoverer/secure

3) Change the old servername to new server name in httpd.conf

4) Stop the Discoverer services 

    $ORACLE_HOME/opmn/bin/opmnctl stopall

5) Start the Dicoverer services.

    $ORACLE_HOME/opmn/bin/opmnctl startall

Where are Attachment files held for AGIS transactions ?

Hi All,

Attachments are held in fnd_attached_documents table. 

If you wanted to use sql to find the attachment details then you could use the following sql:- 

select * from fun_trx_batches where batch_number = << batch number>> 

select * from fun_trx_headers where batch_id = << from the first query>> 

select * from fnd_attached_documents where pk1_value = <<header id from the second 

query>> 

Thanks

Ravi