Tag Archives: squid

Caching iOS updates on a Squid proxy server

24 Comments

Update (22 December 2014): The following instructions have been updated and tested with iOS 8.

Right now, my challenge is upgrading almost 200 iPads to iOS 7 with minimal pain (read: zero device handling). Factor in less-than-ideal Internet bandwidth and Apple’s disinterest in allowing proxies to cache iOS updates, and it’s been a bit of a headache.

First, a word of advice: ask your users not to upgrade when prompted. Do this before Apple release a major update, to buy yourself some time to test it on your network and to check that the update is being cached properly.

Hopefully your iPad fleet is already using your Squid proxy. Ours is configured (via Apple’s Profile Manager) to use a PAC file when it’s on our WiFi network. The PAC file directs all but onsite requests to Squid.

Unfortunately, iOS doesn’t use the proxy for everything; system update authorizations, in particular, don’t get out unless permitted on your firewall. Here’s the relevant rule on our iptables firewall (no_proxy_ok is one of our custom chains, as is tcp_allowed):

-A no_proxy_ok -p tcp -m comment -m tcp -m multiport -d 17.0.0.0/8 -j tcp_allowed --dports 80,443,5223,2195,2196 --comment "allow Apple services (e.g. APNs, updates)"

Mercifully, the update itself is requested via the proxy, but getting it to cache is non-trivial. Obviously max_object_size needs to be big enough to accommodate a 1GB+ file. I went with 2GB:

maximum_object_size 2048000000 bytes

But this wasn’t enough to get the update to cache. A bit of sleuthing led to the first problem: Apple adds HTTP headers like these to its updates, so Squid discards them:

Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache

The workaround is to break HTTP a little by adding this line above any other refresh_pattern entries in your squid.conf:

refresh_pattern -i appldnld\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate

refresh_pattern -i phobos\.apple\.com 129600 100% 129600 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate

This forces Squid to treat objects from *.appldnld.apple.com and *.phobos.apple.com as “fresh” (i.e. cacheable) for 90 days (129600 minutes), no matter what appldnld.apple.com and phobos.apple.com say.

Finally, I made sure appldnld.apple.com requests were excluded from Squid’s delay pools and filtering ACLs; you may need to make similar tweaks. I also found that maximum_object_size wasn’t being applied correctly to cache_dir, so I defined it explicitly, i.e.:

cache_dir aufs /var/spool/squid3 256000 128 256 max-size=2048000000

iOS 7 is rolling out smoothly as I type.

Squid authentication via OS X Profile Manager and Active Directory

Updated on 6-Nov-13 for OS X Server 3.0 on Mavericks

My last post was about getting access to OS X Server’s Profile Manager database; this post is about doing something useful with it.

Hypothesis: given live access to data from Profile Manager and Active Directory, it should be easy to write a Squid external_acl_type helper that maps incoming IP addresses to usernames. An optional check for group membership? Trivial. Amirite?!

I was half-right. The lookups weren’t hard, but getting the helper to terminate when Squid wanted it to, and to NOT terminate prematurely, required a little trial-and-error. Turns out Squid keeps its helpers alive by sending them empty lines, so terminating on empty input isn’t such a good idea.

Anyway, here’s the code that has our iPad fleet “authenticating” with our Squid proxy server transparently. It’s been tested on Linux (Ubuntu 12.04 LTS) and OS X. Yes, Python would have been better than PHP, but I’m more fluent in PHP, and the PHP CLI interpreter is efficient enough for this purpose.

Update 23-Dec-2014: this script is now hosted on GitHub.

To use it in squid.conf (assuming you’ve pulled it down to /opt/git/extensions/squid/external_auth.php):

external_acl_type external_auth ttl=300 negative_ttl=5 children-startup=10 children-max=40 children-idle=10 ipv4 %SRC %MYPORT /opt/git/extensions/squid/external_auth.php

acl Apple_Devices external external_auth
acl Staff_Apple_Devices external external_auth staff
acl No_Filter_Devices external external_auth no_filter
acl No_Access_Devices external external_auth no_access

The “staff”, “no_filter” and “no_access” values map to $SQUID_LDAP_GROUP_DN in the configuration file – customise as needed (many groups may be defined).

Finally, use your new acls in some access rules, e.g.:

http_access allow localnet Staff_Only_Websites Staff_Apple_Devices
http_access deny localnet Staff_Only_Websites Apple_Devices

Questions? Errata? Do comment.