Deployment client resolving to old deployment server

[Poojitha]

Hi All,

I have deployed new deployment server  (aws ec2 instance) and updated the existing route53 dns entry to point to this new server. But I see the deployment clients are making connection to old server still.

I believe there is  old connection saved at deployment client. Does anyone of you know how to encounter this issue ? Your solution helps me lot please.

Regards,
PNV

[Poojitha]

@deepakc  I checked the  first command : 

/opt/splunkforwarder/bin/splunk list deploy-poll  --> It is pointing to the right dns record . Also, I tried " dig <dns record name>. It is showing IP address of new deployment server.

I tried your second command. I see the configuration is made in local directory.
/opt/splunk/etc/system/local/deploymentclient.conf

I tried set deploy-poll as well. Still it is pointing to old server.

Previously connection was fine. I removed and re-setup the deployment server on same instance. After this I am facing the issue.

Regards,
PNV

[deepakc]

From what your saying, something seems then to be overiding it, if its still taking the old setting, which could be another app.

Can you show me the output of this command on the UF NOT deployment server? (Obviously remove your hostname and ip for security reasons)
/opt/splunkforwarder/bin/splunk btool deploymentclient list --debug

Can you also check the log on the UF  (It may help further as to why - should show connection failures at this stage)
cat /opt/splunkforwarder/var/log/splunk/splunkd.log | grep DC:DeploymentClient

Can you confirm the UF can communicate to port 8089 which is the Deployment Server (telnet to it if you can ) temporarly disable the firewall if you can. 

Check the Deployment Server ports run the below on the Deployment Server
netstat -tuplna

[deepakc]

#On your Forwarders Check this to show what the target is?
/opt/splunkforwarder/bin/splunk show deploy-poll

On your Forwarders Check this to show what the config is?
/opt/splunkforwarder/bin/splunk btool deploymentclient list --debug

It might be that the configuration has been set into the below system local config (/opt/splunkforwarder/etc/system/local/deploymentclient.conf) or sometimes its in a custom app (the above Btool should show you this?)

If so then change it to the new address (Ensure firewalls and ports are accessable):

/opt/splunkforwarder/bin/splunk set deploy-poll <IP_address/hostname>:<management_port>
/opt/splunkforwarder/bin/splunk restart