Reference
--------
Note:182154.1 Subject:Summary of Possible Reasons and Solutions for the Problem Where All Concurrent Requests Stuck in Pending Phase
Scope
--------
Oracle Application 11i,Not Test in R12
Possible Reasons and Solutions
--------
1.Check /etc/hosts file to verify it has correct syntax:
<IP address> <hostname.domain> <hostname>
and for one hostname don't have multiple entries.
2.Before starting the concurrent managers using adcmctl.sh,Make sure all concurrent managers process FNDLIBR have been shutdown at the OS level.
for example:
>ps -ef|grep $USER|grep FNDL
if there still have FDNLIBR processes still running.
>kill -9 PID
PID form output of the command ps in previous step.
3.Changing status of Concurrent Requests from Pending to Inactive.
At form [Administer Concurrent Managers(FNDCPVCM)],select the concurrent manager which pending rrquests is not zero and click button [Requests] in the lower right corner.
in the pop up windows,recording the request IDs,and then using specific Request in Find Request window search these request,(notice the field [Select the number of Days to view]
should be set a properly value.),in Requests window,click [Hold Request] to change the phase of these request to [Inactive] and status to [On Hold] .
if ALL concurrent managers been shown in form [Administer Concurrent Managers(FNDCPVCM)] with zero or null pending requests,then restart CM,after the CM processes have been started normally,
then in request windows,you can click [Move Hold] to change the phase of these request back to [pending].
4.Check the value of system profile [Concurrent:GSM Enabled] at Site level.
if the value of [Concurrent:GSM Enabled] is Y,changing it to N and restart concurrent managers,if concurrent managers can start normally,changing it back to Y and
restart CM again.
5.Make sure customize env file has been added in env file which used in Concurrent Manager start/stop scripts adcmctl.sh.
>cat $COMMON_TOP/admin/scripts/$CONTEXT_NAME/adcmctl.sh|grep ENVFILE=
if your environment have user defined customize env files,please sure they have been added in env file used in adcmctl.sh.
6. Relink executable program under $FND_TOP/bin using adadmin.
adadmin-->2. Maintain Applications Files menu-->1. Relink Applications programs
when ask specific products,input:FND
Enter list of products to link ('all' for all products) [all] : FND
7.Rebuild the concurrent manager views. As applmgr run the following from the OS:
This is non-destructive.
Concurrent Manager views can be rebuilt by running the following command at the command line:
Ensure that concurrent managers are shutdown.
FNDLIBR FND FNDCPBWV apps/apps SYSADMIN 'System Administrator' SYSADMIN
Restart the concurrent managers.
8.The Profile Option 'Concurrent: OPS Request Partitioning' may be set incorrectly. This profile option should always be set to
OFF, regardless of whether you are running OPS(RAC) or not, because the profile is obsolete.
9. The System Profile Option: Concurrent Active Requests is possibly to 0.
a. Log into Oracle Applications as SYSADMIN.
b. Select System Administrator responsibility.
c. Navigate to PROFILE > SYSTEM.
d. Query for %CONC%ACTIVE%.
e. Change the profile option for 'Concurrent: Active Request Limit' to Null (blank).
f. Exit Oracle Applications and log in again for the change to take affect.
g. Run a new concurrent request.
10. The Concurrent managers were brought down, while an outstanding request was still running in the background. In which case, update the FND_CONCURRENT_REQUESTS table as follows:
sql>
update fnd_concurrent_requests
set status_code='X', phase_code='C'
where status_code='T';
sql> commit;
STATUS_CODE
SQL> select lookup_code,meaning
2 from fnd_lookups
3 where lookup_type = 'CP_STATUS_CODE'
4 order by lookup_code;
LOOKUP_CODE MEANING
--------------- --------------------
A Waiting
B Resuming
C Normal
D Cancelled
E Error
G Warning
H On Hold
I Normal
M No Manager
P Scheduled
Q Standby
R Normal
S Suspended
T Terminating
U Disabled
W Paused
X Terminated
Z Waiting
PHASE_CODE
SQL> select lookup_code,meaning
2 from fnd_lookups
3 where lookup_type = 'CP_PHASE_CODE';
LOOKUP_CODE MEANING
-------------- --------------------
C Completed
I Inactive
P Pending
R Running
11. The control_code for concurrent_queue_name = 'FNDCRM' is 'N' in the FND_CONCURRENT_QUEUES table, which means 'Target node/queue unavailable'.
This value should be NULL (CRM is running; target and actual process amount are the same), or 'A' ('Activate concurrent manager' control status).
Set the control_code to 'A' in fnd_concurrent_queues for the Conflict Resolution Manager:
a. Logon to Oracle Applications database server as 'applmgr'.
b. Verify the Applications environment is setup correctly ($ORACLE_HOME and $ORACLE_SID).
c. Logon to SQL*Plus as 'APPS' and run the following SQL statement:
update fnd_concurrent_queues
set control_code = 'A'
where concurrent_queue_name = 'FNDCRM';
commit;
d. Verify the status of the concurrent managers through the Concurrent -> Manager -> Administer form.
If the CRM is still not active, bounce (deactivate, activate) the Internal Concurrent Manager. This is done through the Concurrent > Manager > Administer form
from the 'System Administrator' responsibility. It can also be done through the CONCSUB command at the command level.
Setting the control_code to 'A' in the fnd_concurrent_queues table for the Conflict Resolution Manager indicates that this concurrent manager
is to be activated with the parameter values specified through this table for this manager (MAX_PROCESSES, CACHE_SIZE, etc).
Concurrent Manager [Conflict Resolution Manager] and [Scheduler/Prereleaser Manager] cannot be Restart or Active in the form [Administer Concurrent Managers(FNDCPVCM)],
so we need update control_code to do this job.
queue name of Conflict Resolution Manager is FNDCRM.
queue name of Scheduler/Prereleaser Manager is FNDSCH.
Control_code
SQL> select lookup_code,meaning
2 from fnd_lookups
3 where lookup_type = 'CP_CONTROL_CODE'
4 order by lookup_code;
LOOKUP_CODE MEANING
--------------- --------------------
A Activating
B Activated
D Deactivating
E Deactivated
N Target node/queue unavailable
O Suspending concurrent manager
P Suspended
Q Resuming concurrent manager
R Restarting
T Terminating
U Updating environment information
V Verifying
X Terminated
12. What is the cache size? Try increasing the cache size then stop/restart the concurrent managers.
If concurrent requests are rarely prioritized and there are managers that service short-running requests, consider setting the cache size to
equal at least twice the number of target processes. This increases the throughput of the concurrent managers by attempting to avoid any sleep time.
For example:
If more than one manager or worker processes the same type of requests with only a small cache size, it may be unable to process any jobs in a
single processing cycle, because other processes have already run the cached requests.
When this happens, it is important to note that the manager will sleep before refreshing its cache. To increase manager throughput where there
are sufficient requests of the required type in the queue, increase the cache size to improve the chance of the manager finding work to process
and thus avoid having to enter a sleep phase.
TIP: Ensure that the system is not resource-constrained before attempting to increase the rate of concurrent processing in this way, otherwise,
these changes may actually reduce concurrent processing throughput because jobs take longer to run.
Help of Cache size
Cache Size (Concurrent Manager only)
Enter the number of requests your manager remembers each time it reads which requests to run. For example, if a manager's workshift has 1 target process and a cache value of 3, it will read three requests and wait until these three requests have been run before reading new requests.
In reading requests, the manager will only put requests it is allowed to run into its cache. For example, if you have defined your manager to run only Order Entry reports then the manager will put only Order Entry requests into its cache.
If you enter 1, the concurrent manager must look at its requests list each time it is ready to process another request.
By setting the cache size at a higher number, the concurrent manager does not have to read its requests list each time it runs a request. However, the manager does not recognize any priority changes you make for a particular request if it has already read that request into its cache. Further, even if you give a higher priority to a new request, that new request must wait until the buffer is empty and the manager returns to look at the requests list. That request may have to wait a long time if you set the buffer size to a high number.
You should use cache size to tune your concurrent managers to work most efficiently for you site's needs. If your organization tends to reprioritize jobs going to a certain manager, that manager should have its buffer size set fairly low.
Suggestion: Enter a value of 1 when defining a manager that runs long, time-consuming jobs, and a value of 3 or 4 for managers that run small, quick jobs.
13.Concurrent Manager [Output Post Processor] can not be started with Actual is 0 and Service Info %%service threads running.
Select [Output Post Processor] in Administer and click [Processes] button in lower right corner,verify [Start Date],if it early than restart date,you should Terminate it first in main windows and the restart it.
14.OAM Generic Collection Service:HOSTNAME is showing status:Deactivated in FNDCPVCM and "The target node/queue unavailable" in OAM.
OAM access path:OAM-->Site Map -->Application Services -->Generic Services --> Select OAM Generic Collection Service -->View Details.
using below sql to verify TARGET_NODE=<Right Node> and CONTROL_CODE=NULL,if not,update.
SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE, TARGET_NODE, NODE_NAME from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';
15.if "The target node/queue unavailable" had been shown after cloning or Adding new node.
>sqlplus apps/appsuat
SQL*Plus: Release 10.1.0.5.0 - Production on Mon Mar 23 15:48:04 2009
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
PL/SQL procedure successfully completed
SQL> update icx_parameters set session_cookie_domain=NULL;
1 row updated
SQL> commit;
Commit complete
SQL> exit
after execute FND_CONC_CLONE.SETUP_CLEAN,firstly run Autoconfig on ALL DB tier and then all APP tier.
16.from note:738405.1,if the Internal Concurrent Manager starts, but all others managers have a status of 'Target queue/node unavailable'.
and the log file of internal Concurrent Manager shows the following errors:
Internal Concurrent Manager found node to be down .
you should check if ping command can be used in OS level.
>ping hostname
17.Note:104541.1 Troubleshooting (Concurrent Manager Generic)
18.Note:105133.1 Concurrent Manager Questions and Answers Relating to Generic Platform
19.Note:134007.1 CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
-EOF-
转自pccw的同事小马哥
