What came first: the CNAME or the A record?
Recorded: Jan. 20, 2026, 10:03 a.m.
| Original | Summarized |
What came first: the CNAME or the A record? Timeline All timestamps referenced are in Coordinated Universal Time (UTC).TimeDescription2025-12-02The record reordering is introduced to the 1.1.1.1 codebase2025-12-10The change is released to our testing environment2026-01-07 23:48A global release containing the change starts2026-01-08 17:40The release reaches 90% of servers2026-01-08 18:19Incident is declared2026-01-08 18:27The release is reverted2026-01-08 19:55Revert is completed. Impact ends What happened? While making some improvements to lower the memory usage of our cache implementation, we introduced a subtle change to CNAME record ordering. The change was introduced on December 2, 2025, released to our testing environment on December 10, and began deployment on January 7, 2026. How DNS CNAME chains work When you query for a domain like www.example.com, you might get a CNAME (Canonical Name) record that indicates one name is an alias for another name. It’s the job of public resolvers, such as 1.1.1.1, to follow this chain of aliases until it reaches a final response:www.example.com → cdn.example.com → server.cdn-provider.com → 198.51.100.1As 1.1.1.1 traverses this chain, it caches every intermediate record. Each record in the chain has its own TTL (Time-To-Live), indicating how long we can cache it. Not all the TTLs in a CNAME chain need to be the same:www.example.com → cdn.example.com (TTL: 3600 seconds) # Still cached The logic change The code that merges these two chains is where the change occurred. Previously, the code would create a new list, insert the existing CNAME chain, and then append the new records: However, to save some memory allocations and copies, the code was changed to instead append the CNAMEs to the existing answer list: As a result, the responses that 1.1.1.1 returned now sometimes had the CNAME records appearing at the bottom, after the final resolved answer. Why this caused impact When DNS clients receive a response with a CNAME chain in the answer section, they also need to follow this chain to find out that www.example.com points to 198.51.100.1. Some DNS client implementations handle this by keeping track of the expected name for the records as they’re iterated sequentially. When a CNAME is encountered, the expected name is updated: ;; ANSWER SECTION: Find records for www.example.comEncounter www.example.com. CNAME cdn.example.comFind records for cdn.example.comEncounter cdn.example.com. A 198.51.100.1When the CNAME suddenly appears at the bottom, this no longer works: ;; ANSWER SECTION: Find records for www.example.comIgnore cdn.example.com. A 198.51.100.1 as it doesn’t match the expected nameEncounter www.example.com. CNAME cdn.example.comFind records for cdn.example.comNo more records are present, so the response is considered emptyOne such implementation that broke is the getaddrinfo function in glibc, which is commonly used on Linux for DNS resolution. When looking at its getanswer_r implementation, we can indeed see it expects to find the CNAME records before any answers: Another notable affected implementation was the DNSC process in three models of Cisco ethernet switches. In the case where switches had been configured to use 1.1.1.1 these switches experienced spontaneous reboot loops when they received a response containing the reordered CNAMEs. Cisco has published a service document describing the issue. Not all implementations break Most DNS clients don’t have this issue. For example, systemd-resolved first parses the records into an ordered set: typedef struct DnsAnswer { When following a CNAME chain it can then search the entire answer set, even if the CNAME records don’t appear at the top. What the RFC says RFC 1034, published in 1987, defines much of the behavior of the DNS protocol, and should give us an answer on whether the order of CNAME records matters. Section 4.3.1 contains the following text:If recursive service is requested and available, the recursive response to a query will be one of the following:- The answer to the query, possibly preface by one or more CNAME RRs that specify aliases encountered on the way to an answer.While "possibly preface" can be interpreted as a requirement for CNAME records to appear before everything else, it does not use normative key words, such as MUST and SHOULD that modern RFCs use to express requirements. This isn’t a flaw in RFC 1034, but simply a result of its age. RFC 2119, which standardized these key words, was published in 1997, 10 years after RFC 1034.In our case, we did originally implement the specification so that CNAMEs appear first. However, we did not have any tests asserting the behavior remains consistent due to the ambiguous language in the RFC. The subtle distinction: RRsets vs RRs in message sections To understand why this ambiguity exists, we need to understand a subtle but important distinction in DNS terminology.RFC 1034 section 3.6 defines Resource Record Sets (RRsets) as collections of records with the same name, type, and class. For RRsets, the specification is clear about ordering:The order of RRs in a set is not significant, and need not be preserved by name servers, resolvers, or other parts of the DNS.However, RFC 1034 doesn’t clearly specify how message sections relate to RRsets. While modern DNS specifications have shown that message sections can indeed contain multiple RRsets (consider DNSSEC responses with signatures), RFC 1034 doesn’t describe message sections in those terms. Instead, it treats message sections as containing individual Resource Records (RRs).The problem is that the RFC primarily discusses ordering in the context of RRsets but doesn't specify the ordering of different RRsets relative to each other within a message section. This is where the ambiguity lives.RFC 1034 section 6.2.1 includes an example that demonstrates this ambiguity further. It mentions that the order of Resource Records (RRs) is not significant either:The difference in ordering of the RRs in the answer section is not significant.However, this example only shows two A records for the same name within the same RRset. It doesn't address whether this applies to different record types like CNAMEs and A records. CNAME chain ordering It turns out that this issue extends beyond putting CNAME records before other record types. Even when CNAMEs appear before other records, sequential parsing can still break if the CNAME chain itself is out of order. Consider the following response: ;; ANSWER SECTION: Each CNAME belongs to a different RRset, as they have different owners, so the statement about RRset order being insignificant doesn’t apply here.However, RFC 1034 doesn't specify that CNAME chains must appear in any particular order. There's no requirement that www.example.com. CNAME cdn.example.com. must appear before cdn.example.com. CNAME server.cdn-provider.com.. With sequential parsing, the same issue occurs:Find records for www.example.comIgnore cdn.example.com. CNAME server.cdn-provider.com. as it doesn’t match the expected nameEncounter www.example.com. CNAME cdn.example.comFind records for cdn.example.comIgnore server.cdn-provider.com. A 198.51.100.1 as it doesn’t match the expected name What should resolvers do? RFC 1034 section 5 describes resolver behavior. Section 5.2.2 specifically addresses how resolvers should handle aliases (CNAMEs): In most cases a resolver simply restarts the query at the new name when it encounters a CNAME.This suggests that resolvers should restart the query upon finding a CNAME, regardless of where it appears in the response. However, it's important to distinguish between different types of resolvers:Recursive resolvers, like 1.1.1.1, are full DNS resolvers that perform recursive resolution by querying authoritative nameserversStub resolvers, like glibc’s getaddrinfo, are simplified local interfaces that forward queries to recursive resolvers and process the responsesThe RFC sections on resolver behavior were primarily written with full resolvers in mind, not the simplified stub resolvers that most applications actually use. Some stub resolvers evidently don’t implement certain parts of the spec, such as the CNAME-restart logic described in the RFC. The DNSSEC specifications provide contrast Later DNS specifications demonstrate a different approach to defining record ordering. RFC 4035, which defines protocol modifications for DNSSEC, uses more explicit language:When placing a signed RRset in the Answer section, the name server MUST also place its RRSIG RRs in the Answer section. The RRSIG RRs have a higher priority for inclusion than any other RRsets that may have to be included.The specification uses "MUST" and explicitly defines "higher priority" for RRSIG records. However, "higher priority for inclusion" refers to whether RRSIGs should be included in the response, not where they should appear. This provides unambiguous guidance to implementers about record inclusion in DNSSEC contexts, while not mandating any particular behavior around record ordering.For unsigned zones, however, the ambiguity from RFC 1034 remains. The word "preface" has guided implementation behavior for nearly four decades, but it has never been formally specified as a requirement. Do CNAME records come first? While in our interpretation the RFCs do not require CNAMEs to appear in any particular order, it’s clear that at least some widely-deployed DNS clients rely on it. As some systems using these clients might be updated infrequently, or never updated at all, we believe it’s best to require CNAME records to appear in-order before any other records.Based on what we have learned during this incident, we have reverted the CNAME re-ordering and do not intend to change the order in the future.To prevent any future incidents or confusion, we have written a proposal in the form of an Internet-Draft to be discussed at the IETF. If consensus is reached on the clarified behavior, this would become an RFC that explicitly defines how to correctly handle CNAMEs in DNS responses, helping us and the wider DNS community navigate the protocol. The proposal can be found at https://datatracker.ietf.org/doc/draft-jabley-dnsop-ordered-answer-section. If you have suggestions or feedback we would love to hear your opinions, most usefully via the DNSOP working group at the IETF.Cloudflare's connectivity cloud protects entire corporate networks, helps customers build Internet-scale applications efficiently, accelerates any website or Internet application, wards off DDoS attacks, keeps hackers at bay, and can help you on your journey to Zero Trust.Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.To learn more about our mission to help build a better Internet, start here. If you're looking for a new career direction, check out our open positions. 1.1.1.1Post MortemDNSResolverStandardsBugsConsumer ServicesFollow on XCloudflare|@cloudflareRelated postsJanuary 13, 2026 12:00 AMWhat we know about Iran’s Internet shutdownCloudflare Radar data shows Internet traffic from Iran has effectively dropped to zero since January 8, signaling a complete shutdown in the country and disconnection from the global Internet. |
The blog post by Sebastiaan Neuteboom details a critical incident involving the order of DNS records—specifically CNAME and A records—that led to widespread resolution failures for users relying on Cloudflare’s 1.1.1.1 DNS service. On January 8, 2026, a routine update to reduce memory usage in the DNS resolver inadvertently altered the order in which CNAME records were returned relative to A records. This change, though seemingly minor, triggered a cascade of issues for DNS clients that expected CNAME records to appear first in responses. The incident underscores the longstanding ambiguity in DNS protocol specifications and highlights how even subtle implementation details can have far-reaching consequences. The root cause of the problem lay in a code change introduced to optimize memory usage within Cloudflare’s DNS resolver. Prior to the update, the resolver constructed DNS responses by first appending CNAME records and then adding A/AAAA records. This ensured that CNAMEs, which indicate domain aliases, were listed before the final IP addresses they pointed to. However, a refactoring aimed at reducing memory allocations and copies modified the logic so that CNAME records were appended to existing answer lists rather than prioritized. As a result, some DNS responses began returning CNAME records after the A/AAAA records they were meant to precede. While modern DNS clients typically ignore record order, certain implementations—particularly older or more rigid ones—relied on the assumption that CNAME records would appear first in responses. This expectation, though not formally mandated by DNS standards, exposed a critical vulnerability in the protocol’s design. The incident revealed how DNS CNAME chains operate and why their ordering matters. A typical CNAME chain involves a series of aliases that a resolver must traverse to reach the final IP address. For example, querying for *www.example.com* might return a CNAME pointing to *cdn.example.com*, which in turn points to an A record for *198.51.100.1*. Each step in this chain has its own Time-To-Live (TTL), and resolvers cache intermediate results to avoid redundant lookups. However, when a CNAME record in the chain expires, the resolver must re-resolve only that specific part of the chain. The original code ensured that CNAMEs were listed before A records, allowing clients to follow the chain sequentially. The revised code disrupted this process by placing CNAMEs at the end of responses, causing some clients to misinterpret or ignore the chain entirely. The impact was most pronounced on implementations that explicitly relied on the order of records in DNS responses. For instance, the *getaddrinfo* function in glibc, widely used on Linux systems, failed to resolve domains when CNAMEs appeared after A records. This was because *getaddrinfo* iterated through DNS responses sequentially, updating the expected name upon encountering a CNAME. When CNAMEs were placed after A records, the resolver would process the A record first, which did not match the expected name, and then encounter the CNAME later, leading to an incomplete or incorrect result. Similarly, Cisco switches configured to use 1.1.1.1 experienced spontaneous reboot loops when receiving reordered CNAMEs, as their internal DNS processing logic was similarly dependent on the order of records. These cases illustrate how legacy or specialized implementations can be disproportionately affected by deviations from assumed protocol behaviors. The ambiguity in DNS specifications further complicated the issue. RFC 1034, published in 1987, defines DNS behavior but does not explicitly require CNAME records to appear before other types of records. Instead, it states that recursive responses may include “one or more CNAME RRs that specify aliases encountered on the way to an answer.” The phrase “possibly preface” leaves room for interpretation, and the document does not use normative language like *MUST* or *SHOULD* to enforce specific ordering. This lack of clarity has allowed implementations to develop varying expectations about record order over time. Later specifications, such as RFC 2119 (1997), introduced stricter terminology for requirements, but RFC 1034’s outdated phrasing left room for ambiguity. Additionally, the distinction between Resource Record Sets (RRsets) and individual Resource Records (RRs) in DNS terminology contributed to the confusion. While RFC 1034 clarifies that the order of RRs within an RRset is insignificant, it does not address how different RRsets (e.g., CNAMEs and A records) should be ordered within a message section. This gap in the specification left room for implementations to assume that CNAMEs should be prioritized, even if no formal rule mandated it. The incident also highlighted a deeper challenge in DNS design: the tension between protocol flexibility and implementation consistency. While RFC 1034’s wording allowed for varying interpretations, the widespread reliance on CNAME ordering by certain clients created a de facto standard that was not formally codified. This situation mirrors broader issues in internet protocols, where informal practices can become entrenched even in the absence of explicit specifications. The Cloudflare team’s decision to revert the code change and propose a new Internet-Draft for the IETF reflects an effort to address this gap. The draft seeks to clarify how CNAMEs should be ordered in DNS responses, aiming to prevent similar issues in the future. However, achieving consensus on such changes requires balancing the needs of diverse stakeholders, including resolver developers, network operators, and application builders. The post also touches on the broader implications of this incident for DNS security and reliability. While the immediate issue was a software bug, it exposed vulnerabilities in how critical infrastructure relies on untested assumptions about protocol behavior. The DNSSEC specifications, which define stricter rules for record ordering in secured zones, offer a contrast to the ambiguity of RFC 1034. For example, RFC 4035 explicitly requires that RRSIG records (used for DNSSEC validation) be included in the Answer section when a signed RRset is present. However, even these specifications do not address the ordering of CNAMEs relative to other record types, leaving room for similar issues in unsigned zones. This highlights the need for ongoing refinement of DNS standards to account for evolving deployment realities and client behaviors. Ultimately, the incident underscores the importance of rigorous testing and documentation in DNS implementations. Cloudflare’s lack of tests to enforce CNAME ordering, despite its historical implementation, illustrates a common pitfall in software development: relying on undocumented assumptions. The team’s post-mortem analysis and proposed IETF draft represent steps toward greater transparency and standardization, but they also serve as a cautionary tale about the risks of protocol ambiguity. As DNS continues to evolve, with increasing emphasis on security, performance, and reliability, resolving such ambiguities will be critical to ensuring the stability of the global internet infrastructure. For developers and network engineers, this incident serves as a reminder that even seemingly minor implementation details can have significant consequences, and that adherence to well-defined standards is essential for interoperability. |