Edit 4 Jan 2019 – It has been pointed out to me that instead of packets, these are frames. You can read more about how I was mistaken here. The link to RFC 2131, Dynamic Host Configuration Protocol, can be found here. I’m not going to go through and change all the words, just replace them in your head as you read through!
Dynamic Host Configuration Protocol, or DHCP, is one of the first things you learn about with IP devices and the super basics of how they work; even before you learn binary and MAC addresses and layer 3’s and LAN vs WAN and, and, and, and…
Hopefully you get where I am going with this. DHCP is one of the building blocks of IP networking, and most people know just enough to survive. As WLAN Professionals maybe you have heard of DHCP Option 43 (Vendor Specific information) or Option 60 (Class ID) but did you know about Option 51? It’s OK, neither did I.
The point of this diatribe is to point out how much most people, including me, don’t know about DHCP and the different options that are defined within the protocol itself, and how this can affect clients on a WLAN. My interest in this was piqued by a conversation on the WLAN Pro’s Slack site (more on that at the end) and how Apple devices negotiated their DHCP lease. Didn’t know that there was a negotiation during the DHCP process? Neither did I. Let’s get at it.
Some of the basic options that most people already know about, but probably didn’t know their defined option numbers are:
- Option 3 – Router
- Option 1 – Subnet Mask.
- Option 4 – Time Server.
- Option 6 – Domain Server.
- Option 15 – Domain Name.
- Option 51 – Address Time.
- Option 53 – DHCP Message Type.
- Option 138 – CAPWAP Access Controller Address
Honestly, the list of options that are available, and what they do, is pretty astounding when you start to dig into it. It makes me think about some of the different things I could configure on my DHCP servers to try and help my clients negotiate the network easier and faster, improving customer experience. At least that’s the “pie in the sky” thought. The honest answer is the ever classic and pervasive It Depends™. Hence the conversation on Slack about Apple devices and how they “negotiate” their DHCP lease from the DHCP server.
There are a myriad of things that happen within the DHCP process, starting with the typical 4 way exchange that most are aware of:
- DHCP Discover (A client transmitting an initial BootP packet)
- DHCP Offer (The intitial response from the DHCP server)
- DHCP Request (The client requesting the IP offered in Step 2)
- DHCP Ack (the DHCP server confirming the IP as being assigned to that client)
What surprised me was the number of options and things that happen within those 4 “simple” packets and how they differed between vendors. Now as anyone who has dealt with client devices can attest, different vendors can wreak havoc within an infrastructure, but did you realize how much it can do just to a DHCP scope?
With just a little bit of experience, and some time staring at packet captures, this is a pretty easy exchange to watch, and then take for granted. For the sake of this conversation, I performed a bunch of captures on my test DHCP server (pfSense – I should probably do a post about that) and focused on Option 51. This is the part where an Apple device will “negotiate” its lease time. On the right is a look at my new iPad as it starts the DHCP process with a DHCP Discover message (Option 53). Notice the time in Option 51? This is in the initial packet of the process and this Apple device is requesting 90 DAYS (!) for its lease. I tested this with multiple Apple products and found this to be the same across the board. Every time an Apple product that I tested sent a discover packet to the DHCP server, it asked for 90 days.
Conversely, the Android devices I tested asked for what the lease time was as part of the Option 55 section (Parameter Request List), but never asked for a time as a specific option. Windows devices, pictured here, never even inquire about Option 51; either as a standalone option request or part of the Option 55 Parameter Request List. This becomes critical when we get to the “DHCP Request” packet or #3 in the process.
Now while a lot of the parameters requested in this DHCP Request packet on the right aren’t configured on my test DHCP server, the device is still asking for them. That’s fine, the server will only respond with what it knows. The cool thing about a pfSense DHCP server is that it knows about TWO different timers for the DHCP Lease Time. A “default time” and a “maximum time.” These are configurable through the the GUI and until recently, I never knew why this was such an important thing.
Apple Devices and Option 51
Enter the Apple discussion on Slack that I referred to earlier. The conversation centered around the fact that Apple devices didn’t like having short lease times for its DHCP. I don’t have the conversation to post, but I need to give credit to Kristian Roberts for originally bringing light to this subject. I couldn’t find him on Twitter but he is on Slack (more at the end.)
What Kristian discovered, and I confirmed, is that Apple products will always request for 90 days. What gets weird is when it request 90 days. As part of the standard 4 packet exchange, the discover and request (1 & 3) in the exchange come from the clients, the offer and acknowledge (2 & 4) come from the server. An Apple device will only request the 90 days in the discover packet of an initial DHCP process. In the request packet, it doesn’t include Option 51 for the initial request. Where this changes is the renewal that happens at the half life of the lease time as defined in the last packet of the exchange, the acknowledge packet.
In my scenario with 2 different lease times defined, this is what an Apple Mac Book Pro looks like from a DHCP scenario:
I did the math for you, and 7,776,000 really is 90 days. 3,600 seconds is an hour that the server responds with, which in pfSense is defined as the maximum lease time. Notice that the request packet (#3) has no value? Apple devices don’t request a time in their initial DHCP request packet so the server responds with 1,800 seconds, or the default time of 30 minutes.
900 seconds after the first DHCP ACK from the server, the client sends a DHCP Request packet when it starts the renewal process; half life of the 30 minute lease that both the server and client respect from the first ACK packet. If the client didn’t respect that value, the renewal wouldn’t have come in until later. What I want to call your attention to is the IP Address Lease Time for packet numbers 5 through 10. In the initial request packet (line 3) the Mac Book Pro didn’t ask for a time but in every subsequent request it asks for 90 days. The server, programmed with a maximum lease time of 60 minutes, keeps offering it. The other oddity happens at line 7. The request comes 900 seconds after an ACK with a 3,600 second (1 hour) lease. The source and destination IP address revert to a broadcast like the initial request, but it’s a renewal. This time, however, the client “accepts” the 3,600 second lease because the renewal at line 9 happens 1,800 seconds (30 minutes) later.
The one thing that I can state is that Apple definitely has some “negotiation” happening within their DHCP process. What I saw above I saw on multiple Apple products so it’s not just a one off. When the same type of test is compared to a Windows laptop, it’s easy to see the similarities, and the differences.
Windows and Option 51
After examining the Apple products in depth, I wanted to contrast that to other devices. In environments that support a mix of devices and can’t just focus on a single vendor, this might come in handy in the future. What I learned, and have alluded to earlier, is that Windows and Android devices just don’t care about Option 51. This is why, in my opinion, that Windows DHCP servers don’t offer a second lease time in the normal configuration. We are still playing around with the Windows server to see if we can add a max lease time, but for now I can’t find it.
What we have here is the same test as before, but this time with a Dell laptop running Windows 10. The only things that changed was the client device and the time of day. While the general look is the same; starting with the standard 4 packets and then going into the renewal process. The first thing that jumps out is there isn’t 90 days anywhere in the IP Address Lease Time (Option 51) so even though the server will allow a lease of 3,600 seconds, it only ever offers the 1,800 second lease. At no point in this test does the laptop EVER request a time. The only time values come from the server. The test I did with an Android device looks identical to the summary above. The only way to tell the difference is to look into the request packet and see that the Android device included Option 51 in the Option 55 Parameter Request List.
That’s it. Windows and Android devices just don’t care to use Option 51 the way that Apple does.
The last thing that I can draw from comparing the 2 summaries above is why the DHCP Request during the renewal process every once in a while comes as a broadcast from 0.0.0.0 to 255.255.255.255. For both Apple, Windows, and Android, even though the packet is a DHCP “renewal”, the device still remembers the initial lease time. When you see the request from 0.0.0.0 without a preceding offer packet, it means that it is the end of the initial lease period. I’m pretty sure that if I went and read an RFC it would explain that, but I learn so much better doing it this way!
So what does all of this mean? That’s an easy answer!
I spent a bunch of time digging into what was really happening with Apple devices on my network, and made an adjustment to allow those devices to eventually gain a longer lease. I don’t have empirical evidence it made a difference, but I feel like it did. I still have some more work to do, but one thing I can tell you is I have a much better idea of what happens during this process than I did two weeks ago. All it took was some free software, some time, and a bunch of different wireless devices to play around with.
In an environment with almost all Apple, I could see some benefit to having 2 different lease times, and adjust those to find a sweet spot based on how long the clients stay in the environment. If you only have Windows or Android, this won’t help you. What I hope it does is to prod you to do your own testing and see what options you can use for your environment. It’s not difficult, just takes some dedication and some curiosity.
If you aren’t on the Wi-Fi Pros Slack and want to be, contact the infamous Sam Clements and he can hook you up. The conversations are more detailed and thorough, and you can even meet Kristian Roberts!
My site won’t let me upload the actual packet captures I collected during my research, but if you want them send me a message and I will work to get them to you.
Thanks for reading!
Thanks for the mention! Our default lease was set to 15 mins and the max 30 – what we noticed was that after 30 minutes the Mac went offline for a period of time. The WLC showed it in PEM state of ‘DHCP_Req’ – that’s because it expected a DHCP conversation but never got one – it also coincided with the authentication time out.
We moved the auth time out to 45 minutes and the problem went away.
It’s worth noting a different behaviour with iOS and also now Mojave. Before, the client would sit without IP connectivity until the next renewal, but iOS and Mojave recognises that there is no IP connectivity and attempts an ARP – if the ARP fails it will initiate DHCP again.
Although I’m not on Twitter – I do have my own blog which I neglect – https://beaconsandwich.com
Great blog, it is a puzzling scenario!
I never saw the PEM state on the WLC but based on some of your comments I suspected that was happening. The one thing I have learned in working with networks that allow anyone to bring whatever they want is I need to anticipate what the coders of the client devices think is going to happen and then try to meet that requirement. If I don’t, the guest network doesn’t work for them and they complain. Then they go and change the code and I have to start all over again. I guess we can call that job security.
One of the big, eye opening lessons learned here is the idea that even though the device renews at the half life of the lease, it still remembers the original lease time and will honor that. The fact that the Apple devices won’t pick up the longer lease time, even when offered, until the end of the original lease time blew me away.
Thanks for reading, and the great follow up. Hopefully we can meet up one day!
Apple are a nightmare for taking shortcuts too. If an Apple device has been on your network before I have seen them try reuse the same IP address if still within its lease time rather than go via DHCP – which some networks don’t like – they recover after a couple of seconds.
I agree, we have customer wireless in our branches all across the UK – it is a challenge to get right!
I got my website wrong, quite embarrassing – https://beaconsandwich.co.uk!
I actually saw that same behavior on an Android phone. Even though it had been off that network for days when I connected the device sent a Request, not a Discover.
I was wondering about the link I clicked on, but didn’t want to second guess you!
LikeLiked by 1 person
Great article. How were you able to give Apple devices a longer lease period? Thanks.
LikeLiked by 1 person
It Depends on the flavor of operating system that’s running the DHCP. Windows won’t give you the option for different times because Windows devices don’t ask or want it. Not sure about Linux but I was testing on pfSense. Look for additional timer options on the configuration screen.
Ok, thanks! I’ll take a look at that.
Using option 43 for Lightweight Access Point Protocol (LWAPP) APs requires vendor option 43 if you are using Cisco Prime Network Registrar as the DHCP server. This example is specific to the Cisco Aironet 1130 series. You can modify the example to configure option 43 for other vendor options, such as Cisco Aironet 1200 series and Cisco Aironet 1240 series.
This is great stuff. Thank you so much for sharing man.
Hey Jim, great article. Believe the 0.0.0.0 broadcast has to do with the DHCP timings, there’s a T1 and T2 percentage timings. If I’m remembering right, the T2 is sent as a Broadcast to ensure that the lease remains valid, used as a fallback in the case the DHCP server that initially provided the lease is no longer reachable, to ensure it doesn’t lose it’s IP. I didn’t realize Apple products used these options in their leases. One of the sessions at the WTF2020 linked to this article in their notes.
we find that very often the mac DHCP client won’t follow up on an Offer, and prefers to sit there like a turkey with no network access rather than accept the lease offered (even if it’s a reservation). Very frustrating. Apple definitely has some things going wrong in DHCP, but it’s like the rest of their networking – they can’t make a working IMAP client, or even allow you to disable logins. I’ve come to the conclusion that Apple is clueless when it comes to networking, and the lunatics are in charge of the asylum.
It all really comes down to if the device is expecting an offer or not. A reservation is only on the server-side of the equation so the client doesn’t know, or care, that there is a reservation. To me, it all boils down to two distinct things:
1 – Who requests 90 days on a DHCP reservation?
2 – Why not accept the time offered by the server?
If the device decides it doesn’t care and is going to pretend it has a valid 90-day lease then any packet from the server that it doesn’t specifically request is just noise to it and it won’t respond. I am curious if the private MAC address feature that was introduced in iOS 14 is going to have any impact on this but so far, nothing has changed.
We see in the DHCP server logs that the client does a discover, gets an offer then doesn’t follow up, so it should have been expecting an offer, but for some reason didn’t like it. We don’t see any other discover messages either, and sometimes the client will just start “illegally” using the lease for a short time (e.g. 1 minute then turn off again), or just sit there like a stuffed turkey. The offer just sits there showing in our DHCP server.
We don’t have any problems with any other DHCP clients (e.g. MS, Android). We have a lot of network problems with Apple devices. Can’t even properly print – only sends half jobs. Amazing they don’t fix this stuff after years of people complaining about it either.
In terms of restervations, the client does know. There’s a special value for option 51 used to indicate a reservation which basically sets the address time to the largest value 0xFFFFFFFF.
I’ll have to look into that, but if the client device doesn’t read or support that does it really happen? What fascinated me was the number of DHCP options there are, and how some are used, some aren’t, but nothing seems to be standard other than IP, mask, and DFGW.
Whoever told you these were frames instead of packets was incorrect. DHCP messages are carried by IP (a layer 3 protocol) and have IP headers, therefore they are packets, not mere frames.
But since they have a frame header applied to the IP header, is it incorrect to call it a frame? There is still the IP header, but it’s wrapped in a frame.