getent and dig Take Different Routes
In one line
getent goes through the name service switch and dig does not. If the two results differ, the problem is not DNS but the stub resolver segment.
Why this exists
You received a report that "DNS isn't working". You log in and type dig api.example.com, and it comes out fine. Yet the application keeps saying it cannot find the name.
If you conclude here that "dig works, so DNS is fine", you lose hours. This is because dig does not take the path the application uses.
The application calls glibc's getaddrinfo(), and glibc looks into /etc/nsswitch.conf, at its hosts: line, to decide in which order to ask. Usually it is files dns, so it looks at /etc/hosts first and then DNS. dig, on the other hand, skips all of this and asks the nameserver in /etc/resolv.conf directly.
So if there is a wrong entry in /etc/hosts, dig is normal and only the app goes to the wrong place. Conversely, for a name that exists only in hosts, the app works but dig cannot find it. The very fact that the two tools can give different answers for the same name is itself a diagnostic tool.
How it works
Name resolution is not one system but a path made of five segments of different natures joined together.
- Application + stub resolver —
/etc/hosts,/etc/nsswitch.conf, the runtime's own cache - Resolver selection —
/etc/resolv.conf - Recursive resolver cache — most queries end here
- Delegation tracing — root → TLD → authoritative server
- Authoritative server response
The reason for dividing into segments is that you can ask about each segment separately. getent looks including segment 1, dig starts from segment 2, dig @<server> +norecurse looks only at that resolver's cache, and dig +trace walks segment 4 directly. If "+trace is normal but the app fails", the culprit is not the authoritative server but the resolver segment.
/etc/resolv.conf decides the arithmetic of latency.
- Only up to 3
nameserverentries are used options timeout:ndefaults to 5 seconds, andoptions attempts:ndefaults to 2 timesoptions ndots:n— the threshold for how many dots a name needs for it to be queried first as an absolute name without search domains
Two calculations come out of this. If the first nameserver is dead, the user feels the 5 seconds as they are with the defaults. And if ndots is large, several failed queries go out ahead of the resolution of a single external name — a typical cause of slow name resolution in containers. The solution is surprisingly simple: put a dot at the end of the name to make it an absolute name.
TTL and negative caching. DNS has no push. A change does not spread; caches expire. And the answer "it doesn't exist" is cached too. NXDOMAIN (the name itself does not exist) and NODATA (the name exists but only that type does not) are different events, and when you fix a typo in a record and it keeps failing for half a day, the cause is usually the negative cache time.
What it looks like in the field
Read the status of dig first. If it is NOERROR but the answer section is empty, it is NODATA. SERVFAIL means not "the name doesn't exist" but "it could not produce an answer", and indicates a failure to reach upstream or a DNSSEC validation failure. REFUSED is a refusal by policy.
The single letter aa in the flags. Without it, the answer is not authoritative but came from a cache. An authoritative server always gives the original TTL, and a resolver gives the remaining time. It is normal for the two to differ.
Split horizon. In a setup where the internal DNS and the external DNS give different answers to the same name, where you asked changes the answer. That is why, when you receive an outage report, you must first check "are you connected to the VPN".
One last principle. Once the name resolves normally, end the DNS investigation at that point and move on to the connection layer. Continuing to dig at DNS even though name resolution works is a typical form of wasted time.
The actual path by which names are resolved on Linux
It is common for dig to work while the program does not, because the two take different paths. dig asks the DNS server directly, and programs go through getaddrinfo().
What decides the order is /etc/nsswitch.conf.
hosts: files mdns4_minimal [NOTFOUND=return] dns
files is /etc/hosts, and it comes before DNS. So a single old entry left in the hosts file hides DNS entirely. After mdns4_minimal, the [NOTFOUND=return] means if it is not found there, do not go to DNS but fail, so .local domains are not passed on to DNS.
You see the answer a program sees with getent.
getent hosts api.example.com # follows nsswitch as-is
getent ahostsv4 api.example.com # IPv4 only
In /etc/resolv.conf, search multiplies queries. When looking for one short name, it asks once for each search domain. With ndots:5, a name with fewer than 5 dots is tried with search appended first, so even a name like api.example.com is looked up as is only after going through search. This is the reason external domain lookups are slow in Kubernetes Pods. If you put a dot at the end of the name (api.example.com.), that process is skipped.
If you use systemd-resolved, there is one more layer. /etc/resolv.conf points to 127.0.0.53, and the actual servers are behind it. In that case, look at the state separately.
resolvectl status
resolvectl query api.example.com
There are caches in several places. The resolver daemon, the application (especially the JVM's networkaddress.cache.ttl), and the browser. When you changed a record but only one side sees the new value, it is usually the application cache. In some cases the JVM's default is a permanent cache, so during a failover it keeps holding on to the old address.
Same symptom, different culprits: when a name goes to the wrong place
Beneath a report that "the name resolves to the wrong address" lie at least four different faults. The first thing to do is not to fix anything but to hold the answers of the two tools up against each other.
A stale line in hosts. When moving a server, it is common to put a line in /etc/hosts "temporarily" because you cannot wait for DNS to propagate, and that line usually stays rather than being removed. nsswitch looks at files first, and files stops at the first match, so however many times the deployment tool writes the new address below it, the old line wins. The evidence is simple: getent gives the old address, while dig @<server>, asked directly of the internal DNS, gives the new one. If the two differ, there is no reason to wait for the TTL; it is not that the record hasn't propagated, but that DNS was never asked in the first place.
Search domains and wildcards. Suppose the internal DNS has a wildcard (*.corp.test) for a maintenance page, and someone put search corp.test in resolv.conf. A name with fewer dots than ndots is asked with the search domain attached first, so a mistyped name or the name of a removed server becomes <name>.corp.test and gets caught by the wildcard. Here dig gives NXDOMAIN while getent gives an address, because dig does not attach search domains by default. If the search domain is attached at the end of the canonical name that getent returned, that is the decisive evidence. In the field, it really is common for a VPN client to add such a line when connecting and fail to remove it when disconnecting.
A flipped nsswitch order. If it changes to hosts: dns files, any name DNS answers ignores the fixed value in hosts. If getent and dig give the same answer and both differ from hosts, it is this one.
A local resolver. If resolv.conf points to 127.0.0.1 (a cache or a VPN proxy), that daemon's answer is everyone's answer. Even asking for the name as is gets a wrong answer, and dig's SERVER: line points to 127.0.0.1. Merely killing that daemon makes the symptom disappear, but resolv.conf still points to that address, so the same thing happens the next time someone takes that port.
There are two axes for judgment: whether getent and dig give the same answer (if different, it is hosts or search in the stub segment; if the same, it is the server side, a local resolver or the record), and whether the canonical name getent returned is the same as the name asked (if different, an alias or a search domain got in). After fixing, also check that the original configuration is still alive. Other names still depend on that order and that server, and a repair that only erases the symptom books the next fault in advance.
What you will do in the next lab
You will read /etc/resolv.conf and /etc/nsswitch.conf yourself to confirm how the stub resolver works, and add an entry to /etc/hosts to create yourself a situation in which getent finds it and dig does not. You will judge from search and ndots in what order names are asked, and write the same name on two lines to see which one wins. In the last two incidents, you go through the same symptom, "the name resolves to the wrong address", with different causes, and tell them apart by holding the answers of the two tools up against each other.