7

I have a managed package deployed that is calling web services supplied by another party.

On Tuesday last week all calls to this web service started throwing a:

System.CalloutException: IO Exception: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 503 Service Unavailable"

It turns out the domain registration used by the web service expired and was subsequently renewed after the fact. Part of this process required the DNS entry to be updated. I'm not sure, but I believe this changed the IP address the domain was registered against.

I could successfully call the web service using SOAP UI, but calls from Salesforce were failing with the message above. I checked with the other party, they have no IP restrictions on the web services.

By Friday evening the web service calls started working again. I assume the DNS records had been refreshed. This may have been the result of the support cases that were raised or it may have been a general refresh by Salesforce of the DNS records.

In general, how often would Salesforce pickup on DNS changes?

There are a lot of assumptions about what was happening above as I didn't have access to the Salesforce support cases or the web service hosting details. I'll clear them up if I can get some more concrete details.

Improve this question
9

The value on the DNS records for the time to live is controlled by the Authoritative DNS host, not Salesforce. That said, each DNS relay server in the chain between the 'client' and the authoritative server can skew the actual TTL by a few seconds - but it should still be close to the original expiration time.

Most DNS relay servers don't deliberately modify TTL values on records which are retrieved so that they live longer, but anything is possible. Expiring a record and querying for new data is just a suggestion so that everything continues to work in the ever changing land of the internet.

Generally DNS Records have TTL values of 1 day. If you're performing High Availability and failover stuff, you can set the TTL as low as 30 seconds but in the day-to-day life of a DNS record, 24 hours is a fair amount of time for a record to live in the cache of DNS servers around the world at all of the different resolvers that previously looked it up.

The registrar, when the domain expired, could have had a TTL on their own records which were longer than what they have on their customer DNS records - but that's not terribly likely. The reason you were able to hit it with SOAPUI is most likely because your ISP's cache probably hadn't yet been 'polluted' with the bad value and you received a response with the proper host record(s), where the DNS servers at Salesforce were waiting for the TTL to expire on the bad records which it had already retrieved.

In any case, the location where the DNS records are defined is the source for the amount of time a DNS record lives. The host should be able to tell you what the value is and if you can't find it on their website, you could execute an nslookup against their servers/domains and see what the TTL is on the domain A and CNAME records.

(This won't tell you what the TTL was during the period when the domain registration was expired. Maybe you can find a parked / expired domain and check the TTL on those responses.)

Improve this answer
5
  • 2
    The TTL for the domain is currently coming back as 900 (seconds). This was probably set low due to the issues. The registrar appears to be networksolutions.com. They say they default to 7200 (2 hours). It's hard to determine how it took 3 days for the DNS changes to come through. A bit difficult to check in retrospect. – Daniel Ballinger Jun 9 '14 at 0:40
  • I've had confirmation that the TTL was always set at 900 seconds on the web services domain. Everyone using the web service outside of Salesforce was back up and running in 15 minutes. It would appear the Salesforce refresh was delayed for approximately 48 hours. – Daniel Ballinger Jun 9 '14 at 20:53
  • "It can take up to 48 hours for domain changes to become available on the Internet. However, you can reduce this time by lowering the time to live (TTL) value in the account management settings of your DNS provider." Adding Domains Using CNAME Records in Site.com – Daniel Ballinger Jun 9 '14 at 20:57
  • Odd! The statements such as "up to 48 hours" are usually to cover the "are we there yet?" as a general rule of thumb. The moment DNS records have been made visible by the Authoritative server for that domain they are available to any request. It is the modifications to preexisting DNS records and caching behavior of forwarding servers which can increase visibility delay in some cases. 48 hours is a safe estimate when DNS records usually only live in caches between 1 and 24 hours. That said, I know nothing about DNS infrastructure within the SFDC networks and can't speak to them specifically. – Mark Pond Jun 9 '14 at 21:41
  • 1
    It might have been negative DNS caching causing the delay. The last value from the SOA record for the web service HostName is 86400 (24 hours). This is much more into the OPs side of things that I usually need to handle. I think I'll stick to DEV :) – Daniel Ballinger Jun 9 '14 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.