Sunday, June 15, 2014

ORA-01139: RESETLOGS option only valid after an incomplete database recovery

Error :

sql statement: alter database open resetlogs
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of sql command on default channel at 05/13/2004 09:18:54
RMAN-11003: failure during parse/execution of SQL statement: alter database open
resetlogs
ORA-01139: RESETLOGS option only valid after an incomplete database recovery

RMAN> exit



SOLUTION

Two solutions for this problem
=======================
1.Restore the old controlfile from rman , mount the database with old controlfile
and run following script.

RMAN> run {
restore database;
recover database noredo;
alter database open resetlogs;
}

2.Do the incomplete recovery using sqlplus

SQL>connect / as sysdba
SQL>recover database until cancel using backup controlfile;

Type 'cancel' here.

Then open the database with resetlogs

SQL>alter database open resetlogs;

JVM: Guidelines to setup the Java Virtual Machine in Apps Ebusiness Suite 11i and R12 (Doc ID 362851.1)

JDK Version:
It is recommended to use JDK 1.6, because of better performance and additional features. On Solaris and Linux, Sun's JVM will automatically detect whether the Hotspot Server Compiler or the Client Compiler should be used based on the number of CPUs and memory available on the machine.  For JDK 7.0, please reference Using JDK 7.0 Latest Update with Oracle E-Business Suite Release 12.0 and 12.1 (Document 1467892.1)
It is recommended also to use always the latest version for the specific JDK release.
For information about upgrading the JDK, refer to:
Note 401561.1 Using J2SE Version 6.0 with Oracle E-Business Suite 11i
Note 455492.1 Using Latest Update of Java 6.0 with Oracle E-Business Suite Release 12. 
Note 418664.1 Overview of Using Java with Oracle E-Business Suite Release 12
Number of JVMs:
1) For OACoreGroup:
Assumption: eBusiness Suite  is running in a 2-tier environment (middle tier in a separate machine)
- Start witn no more than 100 active users per JVM/OC4J instance.
- Start with 1 active JVM/OC4J instance per CPU core, though newer/faster chip families may support more than 1 JVM/OC4J instance per CPU core, depending upon threading, memory, I/O capabilities, etc.  Confirm with appropriate hardware vendor for best results. 
- Watch out for available memory (make sure that you have enough memory to run all the configured JVMs/OC4J instances without swapping)
Note: Each 'core' in a multi-core system is considered to be a separate CPU. For example, a dual quad-core chip would be the same as stating 2 x 4 x CPU  or 8 CPUs.
In regards to hyper-threading cores, “virtual CPUs,” etc., in theory the same thing applies, but in some cases it may not equate to the exact same as a separate core.  One could tune greater then a single CPU, but not treating the same as two CPUs, four CPUs, etc., depending upon the chip architecture.  Confirm with appropriate hardware vendor for best results. 



2) For  
FormsGroup:
For the forms servlet, the actual JVM/servlet usage is minimal, and a single JVM can normally handle up to 100-250 forms users.
Note: that depends on the type of forms. For more complex forms, those numbers are likely to be substantially lower
Notes: 
a) The number of jvms (oc4j in R12) is configured by the autoconfig variables s_oacore_nprocs, s_disco_nprocs, s_forms_servlet_nprocs (11i), s_forms_nprocs (R12) and s_xmlsvcs_nprocs. 
     (these variable should be updated in the $CONTEXT_FILE using the autoconfig editor from OAM)
b) In 11i, those changes are made in the $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.conf file, while in R12 in the $ORA_CONFIG_HOME/10.1.3/opmn/conf/opmn.xml
Heap Configuration:
1) For OACore, start with the following configuration:
For R12.0 and 11i:
       -Xmx512M -Xms256M -XX:MaxPermSize=128M 
       -XX:NewRatio=2 -XX:+PrintGCTimeStamps
For R12.1 and higher:
      -Xmx1024M -Xms512M -XX:MaxPermSize=256M
      -XX:NewRatio=2 -XX:+PrintGCTimeStamps

     Also, ensure you add the following parameter to the DBC file:

     JDBC\:oracle.jdbc.maxCachedBufferSize=262144
    
Adjust the heap sizes accordingly to minimize garbage collection frequency and user pause times due to full garbage collections.
In JDK 1.6, the JVM detects that you have a server class machine (2 or more CPUs with 2GB or more memory), and will automatically enable Parallel Throughput Garbage Collector. The number of GC threads defaults to the number of CPUs on the machine. If you are running multiple JVMs on the same machine, or if your machine has more than 2 CPUs, to avoid the GC threads to be overly active you should reduce the number of GC threads by using:

      -XX:+UseParallelGC -XX:ParallelGCThreads=2

In most cases, using 2 GC threads should be sufficient. If you are using heap size of over 1GB, you can experiment with 4 GC threads and see if it gives you better performance. 
2) If using Forms in Servlet mode, the following configuration for FormsGroup should be enough for most of the cases:
      -Xmx256M -Xms128M -XX:MaxPermSize=128M -XX:NewRatio=2
Notes: 
a) The heap is configured by the following autoconfig variables:
    11i: s_jvm_options, s_forms_jvm_options
    R12: s_oacore_jvm_start_option, s_forms_jvm_start_options, 
    (these variable should be updated in the $CONTEXT_FILE using the autoconfig editor from OAM)
b) In 11i, those changes are made in the $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.properties file, while in R12 in the $ORA_CONFIG_HOME/10.1.3/opmn/conf/opmn.xml
Additional Recommendations:

1) R12 comes with Forms configured in servlet mode by default, because it is easier to setup if you need features like SSL. However, Forms in servlet mode may have some performance degradation, depending on the network capacity. For this reason, if performance is a concern, you may consider run Forms in socket mode, as it is more efficient than servlet mode.

2) Although the setup of those parameters can be temporarily done manually changing (jserv.conf and jserv.properties in 11i; opmn.xml in R12), the correct way to update those parameters is using the context editor, update the variables s_oacore_nprocs and s_forms_servlet_nprocs, and then run autoconfig.

3) Oracle Development recommends to be on the last ATG rollup patch, which is currently ATG-rollup7 (Note.783600.1).

4) Make sure to have the following setting in the jserv.properties (11i) or oc4j.properties file (R12) file:
wrapper.bin.parameters=-DLONG_RUNNING_JVM=true

If there is any entry in jserv.properties setting -DCACHEMODE, either comment it out or remove that entry:
#wrapper.bin.parameters=-DCACHEMODE=LOCAL

5) Check also recommended patches for Performance issues in:
Note 244040.1  - Oracle E-Business Suite Recommended Performance Patches

6) AIX accepts only the parameters "-Xmx" -Xms". The reminder of the parameters should not be used on those platforms. More 
More info about JDK parameters on AIX can be found at 
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.60_26%2Fvm626%2FJ9%2FGC%2Fxmxoption.html

Saturday, June 7, 2014

Oracle data block corruption errors have been found in the alert log.

Error :
ORA-01578: ORACLE data block corrupted (file # 14, block # 406221)
ORA-01110: data file 14: '/u02/oracle/SVRQAT/db/apps_st/data/a_txn_data04.dbf'
ORA-26040: Data block was loaded using the NOLOGGING option
( OR)
REP-0069: Internal error
REP-57054: In-process job terminated:Terminated with error:
REP-300: ORACLE data block corrupted (file # 14, block # 406221)
ORA-01110: data file 14: '/u02/oracle/SVRQAT/db/apps_st/data/a_txn_data04.dbf'
ORA-26040: Data block was loaded using the NOLOGGING option

Solution  :

STEP 1: TO FIND THE CORRUPTION  SEGMENT NAME/TYPE

Connect / as sysdba

SQL> SELECT tablespace_name, segment_type, owner, segment_name FROM dba_extents WHERE file_id = 14 and 406221 between block_id AND block_id + blocks - 1;

SQL> set pagesize 2000
set linesize 250
SELECT e.owner, e.segment_type, e.segment_name, e.partition_name, c.file#
     , greatest(e.block_id, c.block#) corr_start_block#
     , least(e.block_id+e.blocks-1, c.block#+c.blocks-1) corr_end_block#
     , least(e.block_id+e.blocks-1, c.block#+c.blocks-1)
       - greatest(e.block_id, c.block#) + 1 blocks_corrupted
     , null description
  FROM dba_extents e, v$database_block_corruption c
 WHERE e.file_id = c.file#
   AND e.block_id <= c.block# + c.blocks - 1
   AND e.block_id + e.blocks - 1 >= c.block#
UNION
SELECT s.owner, s.segment_type, s.segment_name, s.partition_name, c.file#
     , header_block corr_start_block#
     , header_block corr_end_block#
     , 1 blocks_corrupted
     , 'Segment Header' description
  FROM dba_segments s, v$database_block_corruption c
 WHERE s.header_file = c.file#
   AND s.header_block between c.block# and c.block# + c.blocks - 1
UNION
SELECT null owner, null segment_type, null segment_name, null partition_name, c.file#
     , greatest(f.block_id, c.block#) corr_start_block#
     , least(f.block_id+f.blocks-1, c.block#+c.blocks-1) corr_end_block#
     , least(f.block_id+f.blocks-1, c.block#+c.blocks-1)
       - greatest(f.block_id, c.block#) + 1 blocks_corrupted
     , 'Free Block' description
  FROM dba_free_space f, v$database_block_corruption c
 WHERE f.file_id = c.file#
   AND f.block_id <= c.block# + c.blocks - 1
   AND f.block_id + f.blocks - 1 >= c.block#
order by file#, corr_start_block#;

SQL> select TABLESPACE_NAME,RELATIVE_FNO,BYTES from DBA_FREE_SPACE where FILE_ID=14 and 406221 between BLOCK_ID AND BLOCK_ID + BLOCKS -1;

SQL> SELECT owner, segment_name, segment_type FROM dba_extents WHERE file_id = 14 AND 406221 BETWEEN block_id AND block_id + blocks - 1;
OWNER          SEGMENT_NAME              SEGMENT_TYPE
------------    ------------------    --------------------
APPLSYS      WF_LOCAL_USER_ROLES     TABLE PARTITION

STEP2:

ALTER TABLE WF_LOCAL_USER_ROLES ENABLE ROW MOVEMENT;
ALTER TABLE APPLSYS.WF_LOCAL_USER_ROLES SHRINK SPACE COMPACT;
ALTER TABLE APPLSYS.WF_LOCAL_USER_ROLES SHRINK SPACE;
ALTER TABLE APPLSYS.WF_LOCAL_USER_ROLES  SHRINK SPACE CASCADE;

Error :
SQL> ALTER TABLE APPLSYS.WF_LOCAL_USER_ROLES SHRINK SPACE COMPACT;
ALTER TABLE APPLSYS.WF_LOCAL_USER_ROLES SHRINK SPACE COMPACT
*
ERROR at line 1:
ORA-01578: ORACLE data block corrupted (file # 14, block # 421578)
ORA-01110: data file 14: '/u02/oracle/SVRQAT/db/apps_st/data/a_txn_data04.dbf'
ORA-26040: Data block was loaded using the NOLOGGING option

STEP 3:EXPORT TABLE FROM SOURCE< PROD >

SQL> select directory_name, directory_path from dba_directories where directory_name=' WF_LOCAL_USER' ;

expdp \"/ as sysdba\"  directory=WF_LOCAL_USER  dumpfile=WF_LOCAL_USER_ROLES.dmp  TABLES=APPLSYS.WF_LOCAL_USER_ROLES  logfile=WF_LOCAL_USER.log

STEP 4: DROP TEH TABLE FROM TARGET

SQL> drop TABLE APPLSYS.WF_LOCAL_USER_ROLES;
SQL> exit

STEP 5: COPY THE EXPORT DUMPFILE FROM PROD TO TARGET

STEP6: IMPORT THE TABLE FORM TARGET
SQL> select directory_name, directory_path from dba_directories where directory_name=' WF_LOCAL_USER' ;

impdp  \"/ as sysdba\"  directory=WF_LOCAL_USER  dumpfile=WF_LOCAL_USER_ROLES.dmp  TABLES=APPLSYS.WF_LOCAL_USER_ROLES  logfile=WF_LOCAL_USER.log

STEP 6:

SQL> SELECT owner, segment_name, segment_type FROM dba_extents WHERE file_id = 14 AND 406221 BETWEEN block_id AND block_id + blocks - 1;

no rows selected
 ++++++++++++++Manjunatha++++++++++++++
Use RMAN to format corrupt data block which is not part of any object (Doc ID 1459778.1)
Doc ID 214369.1,472231.1,28814.1


Master Note (Doc ID 1578.1)

Saturday, May 10, 2014

Error :

RMAN-03002: failure of restore command at 05/10/2014 08:24:51
RMAN-06026: some targets not found - aborting restore
RMAN-06100: no channel to restore a backup or copy of datafile 17
RMAN-06100: no channel to restore a backup or copy of datafile 16
.....
.....

Solution :

rman target /

RMAN> list backupset
RMAN> crosscheck backup;

RMAN> delete expired backup;
It should Fix the above error .....

RMAN> crosscheck backup;

+++++++++++++++++++++++++++++++
Error :


SQL> alter database mount EXCLUSIVE;
alter database mount EXCLUSIVE
*
ERROR at line 1:
ORA-01102: cannot mount database in EXCLUSIVE mode



Solution :
Step 1:
ps -ef | grep ora_

Note: Please kill all ora processess if nay other db is running ... kill it .

step 2: start db using pfile
SQL> startup nomount pfile='$ORACLE_HOME/dbs/initPROD.init

SQL> alter database mount;


Wednesday, April 30, 2014

Error:
Autoconfig could not process the following files:
    Directory: /u01/app/oracle/inst/apps/SVRPRD_svrprdebsal1/ora/10.1.3/Apache/Apache/bin
      .apachectl              BINCPY
      .apachectl              BINCPY

AutoConfig is exiting with status 2

(Or)

AC-00429: Binary copy failed for source file: .../apps/tech_st/10.1.3/Apache/Apache/bin/.apachectl
to destination file: .../ora/10.1.3/Apache/Apache/bin/.apachectl

Raised by oracle.apps.ad.autoconfig.InstantiateProcess
setting permissions: .../ora/10.1.3/Apache/Apache/bin/.apachectl 700

AC-00425: Setting permissions failed for file: .../ora/10.1.3/Apache/Apache/bin/.apachectl

Raised by oracle.apps.ad.autoconfig.InstantiateProcess...

Solution :

The workaround for this is simply to change the permissions on .apachectl, run AutoConfig, and then put them back:

1. As applmgr, stop the web tier services (adopmnctl.sh stop).

2. Switch to the root user (su root)

3. Run the following commands (assumes applmgr environment and that "applmgr" is your actual applmgr user name):
    a.  ls -l $ORA_CONFIG_HOME/10.1.3/Apache/Apache/bin/.apachectl
    b.  chown applmgr $ORA_CONFIG_HOME/10.1.3/Apache/Apache/bin/.apachectl
    c.  chmod 0700 $ORA_CONFIG_HOME/10.1.3/Apache/Apache/bin/.apachectl
    d.  ls -l $ORA_CONFIG_HOME/10.1.3/Apache/Apache/bin/.apachectl

4. Switch back to the applmgr user (exit)

5. Run AutoConfig

6. Reverse step 3 so that the result looks like step 3a:
    a. su root
    b. chown root .apachectl
    c. chmod 6750 .apachectl
    d. exit


Monday, April 28, 2014

Autoconfig Failed with Error: FsCtxFileException: Start of Root Element Expected


Error :

ERROR: FsCtxFile.XMLParseException
oracle.xml.parser.v2.XMLParseException: Start of root element expected.
at oracle.xml.parser.v2.XMLError.flushErrors1(XMLError.java:205)
at oracle.xml.parser.v2.NonValidatingParser.parseRootElement(NonValidatingParser.java:306)
at oracle.xml.parser.v2.NonValidatingParser.parseDocument(NonValidatingParser.java:268)

( or )

ERROR: FsCtxFile.XMLParseException
oracle.xml.parser.v2.XMLParseException: Unexpected EOF
…….....

Or
Caused by: oracle.apps.fnd.oam.em.sdk.targetData.discovery.AppsDiscoveryException

Solution :

To implement the solution, please execute the following steps:

1. Backup the fnd_oam_context_files table in the EBS env:

sqlplus applsys/pwd

SQL> create table fnd_oam_context_files_bkp as select * from fnd_oam_context_files;

2. Truncate this table:

SQL> truncate table fnd_oam_context_files;

3. Run autoconfig in the DB tiers to populate fnd_oam_context_files table with the database information;

4. Run autoconfig in each Apps tier to populate fnd_oam_context_files with the application tier information;

5. From the EBS agent home, run:

./emctl upload

This will upload the information from the EBS system to Grid Control.

Note:
By running the following sql query, it will help you identify if there are duplicate node_name entries in the FND_OAM_CONTEXT_FILES table.
  If duplicates are listed, the older entry should be removed only leaving the most current entry.

SQL> select * from FND_OAM_CONTEXT_FILES where status in ('S','F') and name not in
('METADATA','TEMPLATE') and ctx_type='A';

++++++++++++Manjunatha+++++++++++++

Sunday, April 27, 2014

RC-00110: Fatal: Error occurred while relinking of ApplyDBTechStack

Error :

[oracle@svrtrnebsdl1 bin]$ perl adcfgclone.pl dbTechStack

 0% completed       RC-00110: Fatal: Error occurred while relinking of ApplyDBTechStack

ERROR while running Apply...
Wed Apr 16 10:29:58 2014

 ERROR: Failed to execute /d02/oracle/svrtrndb/11.2.0.3/appsutil/clone/bin/adclone.pl

 Please check logfile......

Solution :
[oracle@svrtrnebsdl1 bin]$ perl /d02/oracle/svrtrndb/11.2.0.3/appsutil/clone/bin/adclone.pl java=/d02/oracle/svrtrndb/11.2.0.3/appsutil/clone/bin/../jre mode=apply stage=/d02/oracle/svrtrndb/11.2.0.3/appsutil/clone component=dbTechStack method=CUSTOM dbctxtg=/d02/oracle/svrtrndb/11.2.0.3/appsutil/SVRTRN_svrtrnebsdl1.xml showProgress contextValidated=true

APPS Password : apps

Note: The above will solve the RC-00110 Fatal error ...