Request URI:/OA_HTML/AppsLocalLogin.jsp java.lang.NoClassDefFoundError

I'm using 11i(11.5.10.2) EBS in our environment. This is maintained for historical data and not in regular use. So the routine health check or any other formalities are not followed for this instance as we would do for any other normal EBS Instance.

Often I'll receive mails from users stating 'Cannot access 11i Instance'. On checking mostly the error will be

There are two ways to solve this issue.

Method 1:

Stop the apache, clear the cache and start the apache server as follows.

Once the apache is started, clear the browser cache and try to login. In most cases this workaround will solve the issue. If still you are facing issue even after this workaround, then follow the below given method 2.

Method 2:

Stop the application services, clear the cache, compile all the jsps, start the application services and try again.

Once the jsp compilation is finished, check any application related services are running. If so kill them and start the application services. 

Note: Here I killed all the specific user (applmgr --> My application OS user) related process, since it is not regular use and no business is running on this.

Now start the application services. Clear the browser cache and try to login. This time you can login.

****************************நன்றி****************************

ORA-20100: ORA-20100: tmp creation failed

 After we cloned our TEST instance, some of the reports(seeded and custom) completed with error. On checking the log file found the below error

After a short search found some metalink documents which shed light on this issue. The reason is UTL_FILE_DIR and the environment variable $APPLPTMP are pointing to the same directory but that mount is already used by the grid software in my environment.

Insight on the Issue

1. The EBS environment variable $APPLPTMP is set to temporary directory path on the 'Database' node.
2. The $APPLPTMP path is needed only on the Database node and not required on the Application node
3. The $APPLPTMP path needs to be the first path in the UTL_FILE_DIR database parameter.
4. If multiple databases runs on the same host, then use a unique value for this temporary directory(i.e. All DB should not use the same directory and also the directory should not be owned by a different user. Normally the owner for this directory will be root.)
5. If there are more than one database(i.e. RAC) then the path must be a shared point
6.  The environment variable $APPLPTMP / UTL_FILE_DIR database parameter is used to process concurrent requests that use
1. Database tools
2. Programs of the execution method: PL/SQL, Stored Procedure, SQL*Plus, SQL*Loader, etc.
7. Therefore this directory must have sufficient disk space and privileges to allow reports to be created and allow the applmgr(APPS OS User) to move the output to $APPLCSF/$APPLOUT
8. The sysadmin concurrent report "Generate concurrent processing environment information" can be used to check $APPLPTMP. 
1. No parameter is required to run this program. 
2. This program is safe to run as it does not change or create anything on the instance.
3. It only reports a listing of environment variables visible to the concurrent managers.

 Below are the steps, I followed to overcome this issue.

1. Take backup of pfile, spfile, context file on the DB Tier
2. Take backup of context file on the APPS Tier
3. Check the existing entries from pfile and context file on DB and APPS Tiers
4. Run the report "Generate concurrent processing environment information" to account the environment variables
5. Create your temporary folder at the desired location with appropriate ownership and permissions
6. Change the entries to your desired location on pfile and context file on DB and APPS Tiers
7. Run auto-config on both DB and APPS tiers
8. Check the entries from pfile and context file on DB and APPS Tiers
9. Re-test the issue.

Take backup of pfile, spfile, context file on the DB Tier

Take backup of context file on the APPS Tier

Check the existing entries from pfile and context file on DB and APPS Tiers

Run the report "Generate concurrent processing environment information" to account the environment variables

Create your temporary folder at the desired location with appropriate ownership and permissions

Change the entries to your desired location on pfile and context file on DB and APPS Tiers

Run auto-config on both DB and APPS tiers

Now shutdown the DB, startup with the modified pfile, check the entries for to be proper and run autoconfig on the DB Node.

Check the entries from pfile and context file on DB and APPS Tiers

Now Re-test the issue. This time the concurrent program(OAM Applications Dashboard Collection) has completed successfully without any issues.

Reference:

12.2 E-Business Suite Concurrent Processing Reports Fails With ORA-06512: at "APPS.FND_FILE" and "ORA-20100: ORA-20100: File O0047353.tmp Creation Failed" Errors (Doc ID 2329514.1)

How To Set APPLPTMP and UTL_FILE_DIR (Doc ID 2141438.1)

Concurrent Requests Fail With Error ORA-20100 (Doc ID 2723960.1)

How To Change Temporary Directories in Apps? (Doc ID 469377.1)

****************************நன்றி****************************

Generating AWR Report for PDB

While tried to generate AWR report for a PDB database, faced an issue. On searching the net and gone through some oracle documents came to know some pre steps needs to done at PDB level to generate a successful AWR report from a PDB database.

Issue Faced:  AWR_PDB_DATABASE_INSTANCE     ORA-06512: at line 27

After some search, found the issue was raised as there was no PDB database snapshot was available for AWR report to run. So decided to create the snapshot for PDB. Now the snapshot can be created manually or automatically.

1. Creating PDB Snapshot Manually
2. Creating PDB Snapshot Automatically

Creating PDB Snapshot Manually

Connect to the PDB database and execute the below procedure to create a snapshot manually.

Once created, check the snapshot information.

Note: In a normal database snapshot information will be available in the table DBA_HIST_SNAPSHOT but in a PDB database you need to check the table AWR_PDB_SNAPSHOT

Now we've created one snapshot but it'll not be sufficient to run an AWR report as the report needs begin and end snaps. So now you can wait for some time and again create a snapshot manually to use as end snap.

Creating PDB Snapshot Automatically

To avoid the inconvenience of  creating snapshots manually, we can tune the database to create the snapshots automatically at regular intervals.

Now by following below steps we've enable to create the snapshots automatically. 

If more than 1 PDB is running, you can change the parameter 'AWR_SNAPSHOT_TIME_OFFSET' to avoid performance issue when AWR report is running parallelly from multiple PDB's.

alter system set AWR_SNAPSHOT_TIME_OFFSET=1000000

Next we need to set the time interval for the snapshot creation. 

Now snapshots will be generated automatically for every 1 hour. This interval timing can be changed as per the need.

After waited for 1 hour new snapshot is created. 

Now we can run AWR report to capture the data.

Ref: https://wikidba.com/how-to-create-awr-for-pdb-level-on-12-2-step-by-steps/

****************************நன்றி****************************

ORA-03113: end-of-file on communication channel

Recently I shut down one of my test database. After 10 mins there is no progress so I decided to check the alert log for any info there. I was surprised to see that the recovery path is filled almost 100%. My TEST database runs in archive mode and the db_recovery_file_dest_size is 20 GB.

So I manually deleted some files under the archive location to check if space is freed up so that my DB can go down. 

It is not so, since I need to crosscheck the archive logs via RMAN after manual deletion of the same. Now I was trapped with no other option left except for SHUT ABORT.

I cancelled my shut immediate and initiated shut abort for the database. DB went down, when I tried to bring up the DB, I was in trouble... facing the ORA-03113 error.

Here the thing is my DB can go upto mount state, which means the control file is still intact. Now it is clear that that due to unavailability of space for db_recovery_file_dest the issue persists. So obvious solutions would be 

1. Clearing the archive logs through RMAN
2. Clearing the archive logs manually and then crosscheck and delete the same via RMAN
3. Change the size of db_recovery_file_dest_size using the alter system command

Since I already made my DB down and trapped myself in an inconsistent state the above normal methods cannot help me out.

 I followed the below steps to overcome the situation.

• Mount the database
• Cleared the unarchived log files to free up some space
• Shutdown and start the database
• Change the size of recovery destination

Finally I connected to RMAN and delete the expired archive logs.

****************************நன்றி****************************