We are ingesting Microsoft tracelogs on Splunk using a Splunk Addon - "Splunk Addon for Microsoft Office 365 Reporting Web Service" . Addon link -https://splunkbase.splunk.com/app/3720 The Addon leverages the credentials from an Application registered on Azure AD and tries to connect to below URL to pull the trace logs from Microsoft reporting webservices. It polls the API every 5 mins. - URL -: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2023-04-13T11%3A40%3A35.692020Z'%20and%20EndDate%20eq%20datetime'2023-04-13T12%3A40%3A35.692020Z'&$skiptoken=1999 Since Apr,6th we observed a sigificant reduction in the log volume on Splunk and to give some stats the app used to pull 50-60 lakhs of events per day which has come down to 1-2 lakh events per day. On further investigation we found some error(401 Client Error) in the logs on Splunk App and it looks like its failing to connect to the URL intermittently leading to drop in the events . PFB error details – requests.exceptions.HTTPError: 401 Client Error: for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2023-04-13T11%3A40%3A35.692020Z'%20and%20EndDate%20eq%20datetime'2023-04-13T12%3A40%3A35.692020Z'&$skiptoken=1999 We have also tried to configure the messagetrace inputs on another Addon - "Splunk Add-on for Microsoft Office 365" and have seen the same errors (401 Client Error) . We have also validated the credentials (Tenant id , Client id and Secret Key) and all looks good . It looks like this issue was identified by other people as well and we can see this was added as known issue on Splunk doc- Splunk Add-on for Microsoft Office 365 . Has anyone identified a fix to it or know a workaround. ? Any help would be appreciated.  ... View more

HI All, We are facing and issue with Splunk Addon for AWS where the Configuration and Inputs page on UI isn't loading and it keeps on loading with a circle without getting the page.  We are observing it after we did Splunk enterprise upgrade recently to 8.2.3.3 . We also updated the add-on to latest version - 5.2.1 Under Splunkd log we can see some REST errors . PFB  12-23-2021 07:40:25.606 -0500 ERROR AdminManagerExternal [11556 TcpChannelThread] - Unexpected error "<class 'splunklib.binding.HTTPError'>" from python handler: "HTTP 500 Internal Server E rror -- Unexpected error "<class 'splunktaucclib.rest_handler.error.RestError'>" from python handler: "REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/urllib3/connectionpool.py", line 667, in urlopen\n self._prepare_proxy(conn)\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bi n/3rdparty/python3/urllib3/connectionpool.py", line 930, in _prepare_proxy\n conn.connect()\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/urllib3/connection.py", li ne 316, in connect\n self._tunnel()\n File "/opt/splunk/hf/lib/python3.7/http/client.py", line 931, in _tunnel\n message.strip()))\nOSError: Tunnel connection failed: 403 Forbidden\n \nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/packa ges/requests/adapters.py", line 449, in send\n timeout=timeout\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/urllib3/connectionpool.py", line 725, in urlopen\n m ethod, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/urllib3/util/retry.py", line 439, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='0', port=8089): Max retries exceeded with url: /servicesNS/nobody/Splun k_TA_aws/configs/conf-aws_sqs_tasks/_reload (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/splunktaucclib/rest_handler/handler.   Error from python.log Traceback (most recent call last): File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/splunk_rest_client.py", line 145, in request verify=verify, proxies=proxies, cert=cert, **kwargs) File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/packages/requests/api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/opt/splunk/hf/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/packages/requests/adapters.py", line 510, in send raise ProxyError(e, request=request) solnlib.packages.requests.exceptions.ProxyError: HTTPSConnectionPool(host='0', port=8089): Max retries exceeded with url: /servicesNS/nobody/Splunk_TA_aws/configs/conf-aws_sqs_tasks/_reload (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))   It has just impacted the UI, the inputs continues to work in the background.  Any help with this would be appreciated.  Regards, Sumeet   ... View more

Hi Community, I am stuck in preparing a dashboard where i have a text input with token name as "uname" . I want to use that token value(uname) in the search string i am writing under the dropdown input to be able to get all the JsessionIds against that userid. <input type="text" token="uname">   <label>Username</label> </input>   <input type="dropdown" token="session">   <label>Jsessionid</label>   <fieldForLabel>Jsessionid</fieldForLabel>   <fieldForValue>Jsessionid</fieldForValue>   <search>     <query>index=middleware_logs userName=$uname$|stats count by JSessionID </query>   </search> The token value doesnt get picked up under dropdown search query. Is there a way to make this work?\ Please help   ... View more