EM 13c, 12c: Listener Target Shows a Status of Down in the EM Cloud Control; Response Metric shows: TNS-01189: The listener could not authenticate the user ( Doc ID 2009573.1 ) LSNRCTL Commands May Yield Frequent TNS-1189 Errors ( Doc ID 285439.1 ) Oracle Support - 17+ hours ago [Collaborations Notes] Collaboration Notes -------------------------------------- Zoom Summary -------------------------- Explained the user this alerts were not triggered from the OEM , Customer asked to create a SPIN off SR with the database team Oracle Support - 18+ hours ago [Web Conference] Oracle Web Conferencing ---------------------------------------------- Hi, I would like to collaborate with you in a Web Conference in order to progress your issue. Please join the conference as soon as possible via the following link: https://oracle.zoom.us/j/95398042847?pwd=aHM4UDFGRWxTRmxhRC9NSnBuaWl2UT09 Thank you, Global Customer Support .WONGSO@.MU - 18+ hours ago [Update from Customer] do we have the call today or I go through the documents first? Oracle Support - 18+ hours ago [ODM Research] How to Supress Alerts on TNS-1189 error for Listener targets in EM Console ( Doc ID 2509691.1 ) EM 13c, 12c: Listener Target Shows a Status of Down in the EM Cloud Control; Response Metric shows: TNS-01189: The listener could not authenticate the user ( Doc ID 2009573.1 ) Please dont take any action based on the above documents this is only for the Refernce .WONGSO@.MU - 18+ hours ago [Update from Customer] cannot access the document you suggested Document cannot be displayed. Possible reasons are: The document id was entered incorrectly. Please check and try again. The document id does not exist (was referenced incorrectly). The document is not classified as publicly accessible ("non-public"). The content is being updated and it is temporarily unavailable but will be made available again soon. If you still have questions about why you could not access this article or bug, please use Contact Us. .WONGSO@.MU - 20+ hours ago [Update from Customer] Hi Sudheer, apologies, please schedule for 1:30 PM IST today instead. Thanks .WONGSO@.MU - 21+ hours ago [Update from Customer] Hi Sudheer, Ok for 1PM IST. Thanks Oracle Support - 21+ hours ago [ODM Action Plan] Hello Prakash - Had an scheduled call at 11:30 can we have call at 01:00 PM IST. Regards, Sudheer .WONGSO@.MU - 21+ hours ago [Update from Customer] Hi Sudheer, pls let's have the call at 11:30IST today, that is in 45 mins Oracle Support - Tuesday [ODM Action Plan] Sure Prakash .WONGSO@.MU - Tuesday [Update from Customer] apologies , we had some urgent production issue . let's schedule the call tomorrow morning around 11:30 IST. will update the SR tomorrow morning and confirm the time Oracle Support - Tuesday [ODM Action Plan] Hello Prakash - Ended the call now, Kept open for 30 minutes no one joined the call. Global Customer Support | Enterprise Manager Team My Working hours: Mon - Fri: 6:00 am to 2:00 pm IST/5:30 pm -2:00 am Mountain Time/7:30 pm – 4:00 am Eastern Time Regards, Sudheer Oracle Support - Tuesday [Web Conference] Oracle Web Conferencing ---------------------------------------------- Hi, I would like to collaborate with you in a Web Conference in order to progress your issue. Please join the conference as soon as possible via the following link: https://oracle.zoom.us/j/91585216653?pwd=M1B2WllYRlJXNlcrdW9DY1JGM29iZz09 Thank you, Global Customer Support .WONGSO@.MU - Tuesday [Update from Customer] Hi , can we have the call now? Oracle Support - Tuesday [ODM Action Plan] Hello Prakash - My Working hours: Mon - Fri: 6:00 am to 2:00 pm IST/5:30 pm -2:00 am Mountain Time/7:30 pm – 4:00 am Eastern Time Please let me know when will be feasible timing for you. Regards, Sudheer .WONGSO@.MU - Tuesday [Update from Customer] Hi please advise when we can have the call today .WONGSO@.MU - Friday [Update from Customer] ok noted thanks Oracle Support - Friday [Notes] Hi Prakash, Sudheer in on leave and he will he will update you on Tuesday. Regards Sabir .WONGSO@.MU - Friday [Update from Customer] Hi, we can have the call now, or in the next 5 hours, please advise Oracle Support - 13 days ago [ODM Action Plan] Hello Prakash - Please update me the feasible time for you. Best Regards, Sudheer Global Customer Support | Enterprise Manager Team My Working hours: Mon - Fri: 6:00 am to 2:00 pm IST/5:30 pm -2:00 am Mountain Time/7:30 pm – 4:00 am Eastern Time If you need immediate assistance outside these working hours, please call to Oracle number and ask for immediate assistance. http://www.oracle.com/us/support/contact/index.html .WONGSO@.MU - 13 days ago [Update from Customer] Hi, we missed the web conf. can you please advise if we can have it tomorrow 6 Jan and at what time? Oracle Support - 19 days ago [Web Conference] Oracle Web Conferencing ---------------------------------------------- Hi, I would like to collaborate with you in a Web Conference in order to progress your issue. Please join the conference as soon as possible via the following link: https://oracle.zoom.us/j/94159668859?pwd=RkthSTlYNkVIZXlwOXM1bHlmbkZUdz09 Thank you, Global Customer Support .WONGSO@.MU - 20 days ago [Update from Customer] Hi let's plan for the zoom session for tomorrow morning, noon IST Time if possible, thanks .WONGSO@.MU - 20 days ago [Update from Customer] [oracle@exdprd0dbadm03 bin]$ ps -ef | grep lsnr grid 189573 1 0 Nov17 ? 00:23:18 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit grid 189633 1 0 Nov17 ? 00:10:27 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_APP -no_crs_notify -inherit grid 189685 1 0 Nov17 ? 00:05:08 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_DG -no_crs_notify -inherit grid 247089 1 0 Nov08 ? 00:19:40 /u01/app/19.0.0.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit grid 262890 1 0 Nov08 ? 02:03:19 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_SCAN1 -no_crs_notify -inherit oracle 399079 397659 0 14:54 pts/2 00:00:00 grep --color=auto lsnr [oracle@exdprd0dbadm03 bin]$ ./emctl getmetric agent LISTENER_SCAN1,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 5 Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved. EMD getmetric error: Unable to connect to the agent at https://exdprd0dbadm03..local:3872/emd/main/ [Connection refused (Connection refused)] [oracle@exdprd0dbadm03 bin]$ ./emctl get metric agent LISTENER_exdprd0dbadm03..local,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 5 Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved. Oracle Enterprise Manager Cloud Control 13c Release 5 Agent Commands: Lifecycle Commands Usage: emctl start agent Starts the agent. emctl stop agent Stops the agent. Agent Operations Usage: emctl setproperty agent [-allow_new] -name PROPERTY_NAME -value PROPERTY_VALUE Sets the specified property name and value in agent configuration file -allow_new is an optional flag that inserts a new property in Agent configuration file if it does not exist. -allow_new must be the first option provided to the command emdctl run_autotune agent This command auto tunes the agent. Autotuning should be attempted when the agent is down. The tuning parameters are dependent on the configuration of the agent. emctl stop agent This command stops the agent. emctl clearsudoprops Clears public, private key pair used for trusted remote operations .WONGSO@.MU - 20 days ago [Update from Customer] Hi Sudheer, ok for now, please share zoom link Oracle Support - 20 days ago [ODM Action Plan] Hello Prakash - can we have a quick Zoom session to check and fix this issue. Best Regards, Sudheer Global Customer Support | Enterprise Manager Team My Working hours: Mon - Fri: 6:00 am to 2:00 pm IST/5:30 pm -2:00 am Mountain Time/7:30 pm – 4:00 am Eastern Time If you need immediate assistance outside these working hours, please call to Oracle number and ask for immediate assistance. http://www.oracle.com/us/support/contact/index.html .WONGSO@.MU - 20 days ago [Update from Customer] still having error, please advise urgently .WONGSO@.MU - 20 days ago [Update from Customer] [orarom@exdprd0dbadm02 bin]$ ./emctl get metric agent LISTENER_exdprd0dbadm03..local,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 4 Copyright (c) 1996, 2020 Oracle Corporation. All rights reserved. Oracle Enterprise Manager Cloud Control 13c Release 4 Agent Commands: Lifecycle Commands Usage: emctl start agent Starts the agent. emctl stop agent Stops the agent. Agent Operations Usage: emctl upload agent Request an upload of collected metric data and severity emctl control agent runCollection TARGETNAME:TARGETTYPE COLLECTIONITEM ... Runs the collections item(s) for the specified target instance emctl stop agent This command stops the agent. emctl clearsudoprops Clears public, private key pair used for trusted remote operations emctl setproperty agent [-allow_new] -name PROPERTY_NAME -value PROPERTY_VALUE Sets the specified property name and value in agent configuration file -allow_new is an optional flag that inserts a new property in Agent configuration file if it does not exist. -allow_new must be the first option provided to the command emctl stopschedule agent [-type TARGETTYPE | TARGETNAME,TARGETTYPE ] [-disable_response] [-disable_agent_schedule] Stops data collection for a specific target instance or for all targets of a specific type. If no target instance is specified or no type is specified then this command will stop all collections for a target except for the target response and collections for the agent target. -disable_response can be used to specify that collection of target response should also be stopped. -disable_agent_schedule can be used to specify that agent collections should also be stopped. emctl verifykey Verifies that the ssl handshake succeeds with the agent key and also sends a request to the agent to ping the OMS emctl control agent unsuspend_metric TARGETNAME,TARGETTYPE COLLECTIONITEM METRIC_COLLNAME This command can be used to unsuspend metrics for a given target. TARGETNAME,TARGETTYPE: Target is specified by specifying the target name and type separated by a ',' COLLECTIONITEM: Collection item name METRIC_COLLNAME: Name of the metric collection that needs to be unsuspended. emctl getproperty agent {-name NAME_1 [... NAME_N] | -category NAME_1 } Gets the specified properties or a category of properties from agent configuration files Note: -name takes a list of property names separated by spaces. Currently this command does not support spaces in the name. emctl dumpstate agent [-dump_full_state] [COMPONENTNAME ...] Dumps the internal state of the specified list of components. emctl status agent [TIMEOUT] Returns the status of the agent TIMEOUT: timeout value in seconds emctl getmetric agent TARGETNAME,TARGETTYPE,METRICNAME [-force] Performs a realtime evaluation of the metric and displays the results for the specified metric -force : (optional) if specified, evaluate the metric even if the target is in blackout emdctl run_autotune agent This command auto tunes the agent. Autotuning should be attempted when the agent is down. The tuning parameters are dependent on the configuration of the agent. emctl set credentials TARGET_NAME:TARGET_TYPE Interactively sets the credential properties for the specified target. emctl clearstate agent [TARGETNAME,TARGETTYPE ...] | [-incrementalconfig TARGETNAME,TARGETTYPE[,COLLECTIONITEM] ... ] 1. Clear all severity state of the agent All target states can be cleared by running: emctl clearstate agent 2. Clear the severity state of a specific set of targets Specific target state can be cleared by specifying: emctl clearstate agent TARGETNAME,TARGETTYPE ... 3. Clear just the incremental state of config metrics. Incremental config state can be cleared by specifying: emctl clearstate agent -incrementalconfig TARGETNAME,TARGETTYPE[,COLLECTIONITEM] ... emctl reload [agent dynamicproperties [-upload_timeout TIMEOUT] TARGETNAME:TARGETTYPE ...] reload Oracle Support - 22 days ago [ODM Action Plan] Hello Prakash - Error you got is due to the space between the listener name and the target type. emctl get metric agent LISTENER_exdprd0dbadm03..local,oracle_listener,Response Regards, Sudheer .WONGSO@.MU - 22 days ago [Update from Customer] Upload to TDS successful for the file 20221227_emctl_get_metric_agent_LISTENER_exdprd0dbadm03.txt. .WONGSO@.MU - 22 days ago [Update from Customer] pls check output uploaded for the emctl get metric agent command you sent Oracle Support - 23 days ago [ODM Action Plan] Hello Prakash - Could you please share the below commnad output from the Agent. emctl get metric agent LISTENER_exdprd0dbadm03..local ,oracle_listener,Response LISTENER_exdprd0dbadm03..local Regards, Sudheer .WONGSO@.MU - 23 days ago [Update from Customer] Please advise what to check Oracle Support - 27 days ago [ODM Action Plan] Hello Prakash - Issue is happening for the below target , could you please check for this target on both the new nodes. LISTENER_exdprd0dbadm03..local Regards, Sudheer .WONGSO@.MU - 27 days ago [Update from Customer] Please advise on issue, script is run on agent server and listener LISTENER_SCAN3 is running on the server [orarom@exdprd0dbadm02 ~]$ ps -ef | grep lsnr grid 9136 1 0 Nov17 ? 03:17:45 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit grid 9322 1 0 Nov17 ? 00:15:30 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_DG -no_crs_notify -inherit orarom 19986 19633 0 09:41 pts/0 00:00:00 grep --color=auto lsnr grid 71326 1 0 Aug19 ? 00:45:43 /u01/app/19.0.0.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit grid 387546 1 0 Aug19 ? 03:45:37 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_SCAN3 -no_crs_notify -inherit [orarom@exdprd0dbadm02 ~]$ cd /opt/OracleHomes/agent_home/agent_13.4.0.0.0/bin [orarom@exdprd0dbadm02 bin]$ ./emctl getmetric agent LISTENER_SCAN3,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 4 Copyright (c) 1996, 2020 Oracle Corporation. All rights reserved. EMD getmetric error: the oracle_listener target "LISTENER_SCAN3" does not exist [orarom@exdprd0dbadm02 bin]$ Oracle Support - 27 days ago [ODM Action Plan] Hello Prakash - On the Agent server. Regards, Sudheer .WONGSO@.MU - 28 days ago [Update from Customer] Hi , should this command be done on the EM server or Database server? Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - you can get the targets list from the below command. cd $AGENT_HOME/bin./emctl config agent listtargets. Regards, Sudheer .WONGSO@.MU - 1+ month ago [Update from Customer] [orarom@exdprd0dbadm02 bin]$ ./emctl getmetric agent LISTENER_SCAN3,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 4 Copyright (c) 1996, 2020 Oracle Corporation. All rights reserved. EMD getmetric error: the oracle_listener target "LISTENER_SCAN3" does not exist [orarom@exdprd0dbadm02 bin]$ exit .WONGSO@.MU - 1+ month ago [Update from Customer] Hi , can you please advise how to run the emctl script? [orarom@exdprd0dbadm02 bin]$ ps -ef | grep lsnr grid 9136 1 0 Nov17 ? 03:00:03 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit grid 9322 1 0 Nov17 ? 00:14:11 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_DG -no_crs_notify -inherit grid 71326 1 0 Aug19 ? 00:44:40 /u01/app/19.0.0.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit orarom 287542 178025 0 14:41 pts/0 00:00:00 grep --color=auto lsnr grid 387546 1 0 Aug19 ? 03:37:35 /u01/app/19.0.0.0/grid/bin/tnslsnr LISTENER_SCAN3 -no_crs_notify -inherit [orarom@exdprd0dbadm02 bin]$ hostname exdprd0dbadm02..local [orarom@exdprd0dbadm02 bin]$ cd /opt/OracleHomes/agent_home/agent_13.4.0.0.0/bin [orarom@exdprd0dbadm02 bin]$ [orarom@exdprd0dbadm02 bin]$ [orarom@exdprd0dbadm02 bin]$ ./emctl getmetric agent LISTENER_SCAN3,oracle_listener,Response Oracle Enterprise Manager Cloud Control 13c Release 4 Copyright (c) 1996, 2020 Oracle Corporation. All rights reserved. Oracle Support - 1+ month ago [ODM Action Plan] Hi Team, As you would appreciate the fact that we cannot have this SR open for a long without progressive business updates We haven't heard back from you on this issue for some time now .So I assume either the issue is resolved or no more exists thus I am inactivating this SR. In case of any concerns in the future, request to log a new service request. Best Regards, Global Customer Support Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - Is this listener correct as there were no services connected to this listener. Regards, Sudheer STEVEN.LEUNG@.MU - 1+ month ago [Update from Customer] ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production Start Date 05-DEC-2022 11:02:50 Uptime 0 days 1 hr. 39 min. 14 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Log File /em/oracle/em13c/middleware/network/log/listener.log Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=emsprod..local)(PORT=1521))) The listener supports no services The command completed successfully Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - Please share the lsnrctl status output. of the listener Regards, Sudheer STEVEN.LEUNG@.MU - 1+ month ago [Update from Customer] Upload to TDS successful for the file emoutput.txt. STEVEN.LEUNG@.MU - 1+ month ago [Update from Customer] Hello Sudheer, command is failing with error attached. Regards, Roshan Oracle Support - 1+ month ago [ODM Research] EM 13c, 12c: Listener Target Shows a Status of Down in the EM Cloud Control; Response Metric shows: TNS-01189: The listener could not authenticate the user ( Doc ID 2009573.1 ) Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - could you please share the below command output. emctl get metric agent ,oracle_listener,Response Regards, Sudheer PRAKASH.BUCKTOWAR@.MU - 1+ month ago [Update from Customer] Hello Sudheer, The alert is being raised by OEM. We need to check if this is a genuine listener error or is it a bug at OEM level since the listener is always up when receiving the alerts. So, we are not sure where the issue lies. Is it at OEM or at Database level itself. Regards, Prakash Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - Could you please clarify on the below question. Do you want to fix this error at the DB level or you need to fix the alert in OEM ? If it is for the Database fix, please raise another SR with the Database team if it is for the fix in OEM we can check it. EM will trigger the alert based on the alert triggered in database Regards, Sudheer PRAKASH.BUCKTOWAR@.MU - 1+ month ago [Update from Customer] Hello Sudheer, We are running on Oracle 19c on Exadata. Is the solution application to us? Regards, Prakash Oracle Support - 1+ month ago [ODM Action Plan] Hello Prakash - Please go through the below document. TNS-01189 During Listener Monitoring Using Enterprise Manager ( Doc ID 2009044.1 ) Regards, Sudheer Oracle Support - 1+ month ago [ODM Question] Why TNS-1189. error got triggered for the Listener from OEM STEVEN.LEUNG@.MU - 1+ month ago [Customer Problem Description] Customer Problem Description --------------------------------------------------- Problem Summary --------------------------------------------------- Enterprise Manager Cloud error Problem Description --------------------------------------------------- Dear Support, could you please advise on error below? EM Event: Critical:LISTENER_exdprd0dbadm03..local - TNS-1189. Please check log for details. Host=exdprd0dbadm03..local Target type=Listener Target name=LISTENER_exdprd0dbadm03..local Message=TNS-1189. Please check log for details. Severity=Critical Event reported time=Nov 25, 2022 5:33:16 AM MUT Operating System=Linux Platform=x86_64 Associated Incident Id=9734 Associated Incident Status=New Associated Incident Owner= Associated Incident Acknowledged By Owner=No Associated Incident Priority=None Associated Incident Escalation Level=0 Event Type=Metric Alert Event name=TNS_ERRORS:tnserrmsg Total Occurrences=207 Occurrences in this event=9 First Occurred in this event=Nov 25, 2022 4:44:07 AM MUT Last Occurred in this event=Nov 25, 2022 5:16:16 AM MUT Metric Group=TNS Errors Metric=TNSMsg Metric value=TNS-1189(version) Key Value=TNS-1189 Key Column 1=TNSError Rule Name=Incident management rule set for all targets,Create incident for critical metric alerts Rule Owner=System Generated Update Details: TNS-1189. Please check log for details. Please let me know if any other details required. Regards, Roshan Error Codes --------------------------------------------------- Problem Category/Subcategory --------------------------------------------------- DB Control 10g and 11g (dbconsole)/Console Access (Browser, User Interface) Uploaded Files --------------------------------------------------- Template Question Responses --------------------------------------------------- 1) === Additional Problem Description === EM Event: Critical:LISTENER_exdprd0dbadm03..local - TNS-1189. Please check log for details. Host=exdprd0dbadm03..local Target type=Listener Target name=LISTENER_exdprd0dbadm03..local Message=TNS-1189. Please check log for details. Severity=Critical Event reported time=Nov 25, 2022 5:33:16 AM MUT Operating System=Linux Platform=x86_64 Associated Incident Id=9734 Associated Incident Status=New Associated Incident Owner= Associated Incident Acknowledged By Owner=No Associated Incident Priority=None Associated Incident Escalation Level=0 Event Type=Metric Alert Event name=TNS_ERRORS:tnserrmsg Total Occurrences=207 Occurrences in this event=9 First Occurred in this event=Nov 25, 2022 4:44:07 AM MUT Last Occurred in this event=Nov 25, 2022 5:16:16 AM MUT Metric Group=TNS Errors Metric=TNSMsg Metric value=TNS-1189(version) Key Value=TNS-1189 Key Column 1=TNSError Rule Name=Incident management rule set for all targets,Create incident for critical metric alerts Rule Owner=System Generated Update Details: TNS-1189. Please check log for details. 2) ###Business impact### Getting alerts frequently and wanted to know what's the real issue. Is this a bug or an issue that needs to be addressed immediately. No impact