Recent Posts

The future never gets old.

Latest Topics

Where our attentions lie today.

Recent Authors

Eager beavers.

Archives

The past, in blog form.

HOWTO: Create a transparent local Apple software update server

posted by Nick Bonadies
Nick Bonadies
Nick Bonadies

Senior Systems Administrator : Boston

topics: Games, Internet Culture, Software, Barbarian, and Rock and Roll

on May 29, 2009 at 09:02 AM
filed under: Software, Developers, Barbarian, and Internet Culture
http://tbg.cm/hFWNUr
If the benefit isn’t clearly obvious from the title, let me explain why you would want to set up something like this in your office network. Bandwidth is expensive. Especially if you need a bunch of it and require a solid SLA. Apple updates can be HUGE; the most recent Mac OS 10.5.7 combo update clocked in at a whopping 729MB. Now imagine you have an office of 20 people all using the same internet connection to download the same package from the same place. Madness right?
They’ve just saturated your inbound link and business is grinding to a halt! If you’re in this position you’re already too late, and you’ll have to make up for it using some fancy QoS rules or blocking traffic from the Apple software update servers all together. Don’t live in fear of the next Mac OS update because now you have time to set up your very own Apple software update server!
This guide assumes you have an Apple Leopard Server; you’ll need Leopard Server to configure the Software Update Service (SUS). You’ll also need a decent router, firewall, or internal DNS server.
The major goal here is to make it so our SUS is completely transparent to the client machines on our network; we want them to think they are talking straight up with Apple. This is key because we don’t want to change our client configurations at all. Why? well because if we changed the way our client machines update their computers, the system would likely breakdown or become too resource expensive to operate (when clients were out of the office they would be looking into the office for updates). We want our client machines to update no matter where they are, but if they happen to be in an office we want them to download locally from us because it is relatively cheap in regard to network resources.
We need to first figure out how we’re going to point our clients transparently at our SUS for updates. I can think of two ways to do this, with some internal DNS changes or using DNAT.
I’ll quickly cover both.
DNS is probably the simplest route. In your internal DNS you would set swquery.apple.com, swcdn.apple.com, and swscan.apple.com as CNAMEs for the server running SUS. If you want go to the NAT route you would simply DNAT those hosts from clients in your internal network to your server running SUS. If you are running a DMZ you want to make sure the servers in your DMZ are NOT looking at your SUS for updates.
Now we’ve got all our update traffic pointed square at our SUS lets configure it! In server admin, enable Software Update and Web from the settings tab. After both are enabled click on Software Update in the list of available services. Within the service, click the settings tab. You should see a place to set a custom port, I left it as the default 8088. You’ll also notice some other configuration options, but I won’t go into that here RTFM for that.
Now that our SUS is configured and we know what port it will be running on click the Web service in the list of available services. In the Sites tab click on your default site. The window pane below will then activate and you’ll have even more tabs to choose from, click the Aliases tab. What we need to do here is tell our web server to redirect all traffic looking for Apple updates to the correct place. Remember how we wanted to avoid client configuration and keep all this transparent? Click the plus sign under the box for URL Aliases and Redirects. The type you’re creating is a straight up redirect. For pattern input: /content/catalogs/others/index-leopard.merged-1.sucatalog (this assumes all your clients are running Leopard) and the path you want to input would look something like: http://your.server.com:8088/content/catalogs/index.sucatalog.apple where your.server.com is the DNS name for your server.
You may be thinking that this redirect doesn’t look awesome, and you’d be right. Its not awesome and it could break with a future Mac OS update. But luckily it doesn’t matter if it breaks! Clients will silently fail to check for updates until we fix it.
Finally, fire up the Web and Software Update services we just configured and run Software Update on a client machine. If it doesn’t complain you’ve probably succeeded! Good job, go grab a beer or something. If you just can’t believe it was that easy, fire up Safari and try this url if you’ve done a good job it should get immediately redirected to your SUS and you should see an awesome PLIST file full of updates! Delicious.
Thanks to Ian
Ian Westcott
Ian Westcott

Director of IT : New York

topics: Mobile, Measurement and Analytics, Art, Photography, and Internet Culture

!
UPDATE:
We ran into a little issue with this, if you check your apache logs you’ll probably find a lot of errors similar to this: “File does not exist: /Library/WebServer/Documents/content.” To fix this problem we’ll add a little symlink from our content store in SUS to the Documents folder in our webserver, so do “sudo ln -s /usr/share/swupd/html/content /Library/WebServer/Documents/content”
You’ll probably also see some errors like: “File does not exist: /Library/WebServer/Documents/WebObjects/SoftwareUpdatesStats.” Lets fix this similarly by making a simlink to the correct resource on our SUS like: “sudo ln -s /usr/share/swupd/cgi-bin/SoftwareUpdateServerStats /Library/WebServer/Documents/WebObjects/SoftwareUpdatesStats”
And thats it!
photo by joyosity
adapted from this post

21 comments

On June 09, 2009 at 07:38 AM, Mac Fanboy wrote:
This article is STOLEN from http://www.macosxhints.com/article.php?story=20071009082248452
On June 09, 2009 at 08:33 AM, nick wrote:
yeah i donno if i would call it stolen, adapted sure. they don't talk about natting or leopard specifics in that article. but hey, you're right so i changed the post.
On July 05, 2009 at 11:55 AM, Jason wrote:
Hey nick
The web redirect configuration here assumes all clients as running leopard, correct? What if we had some Tiger clients on our network, like some emacs. Any ways of doing an 'additional' redirect just for those tiger clients and what would that configuration be?

*Even if a new mac update breaks this web redirect, I am hoping we would still be able to use the same DNS route for SUS?

And thanks for the very helpful post!
Update for Snow Leopard -- add a Redirect from /content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog to http://yourserver.local:8088/index.sucatalog

This at least shuts up Snow Leopard updates... however, I don't know if it is downloading SL updates (I'll let you know after 10.6..1). The reason I thing Leopard server isn't downloading SL updates is I didn't get an update for the 10.6 server admin tools... had to download them manually. (Needed the new ones on SL to make a NetInstall of the SL DVD, even the server is 10.5 Server)

Without the redirect, it doesn't fail silently... it says it could not find "/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog" which is extremely handy from an admin perspective, but rather cryptic to users who think this stuff just works.
I followed these steps verbatim, and I'm still getting the mothership... my dns resolves back on itself just fine. I've tried just about every kind of combo and still nothing. I'm the dumb one in the back and would like a pointer in the right direction if anyone could! Thanks much.
On December 16, 2009 at 12:02 PM, William wrote:
Question- We have a mostly PC network running Active Directory with a smattering of Macs running Snow Leopard or older OSs. Any suggestions on how to set this up without buying a Mac Server?
Cheers
William
On December 23, 2009 at 10:33 AM, Micah002 wrote:
I'm with John here. Everything looks good, but it still only talks to the mother ship. Any pointers for the other guy in the back scratching his head after two hours of poking at it with a Google stick.

The issue appears to be that the redirect isn't working, but I'm not sure what else I might need to do to get it up and running properly.

Any ideas would be much appreciated.

Thanks!
You don't really need all the redirects in the web service. The standard caveat applies to be sure your Mac OS X Server is not using itself for DNS; otherwise, you'll never get any new updates in the Software Update service. All your clients, though, must be using your Mac OS X Server for DNS; otherwise, they'll always hit Apple's update server instead of yours.

1. Set up Mac OS X Server, and enable the Software Update, Web, and DNS services.
2. In the DNS service, add a master zone named "swscan.apple.com." (with the trailing dot!)
3. Create an A (machine) record that maps "swscan.apple.com." (with the trailing dot!) to the IP of your Mac OS X Server.
4. Create two CNAME (alias) records that point "swcdn.apple.com." and "swquery.apple.com." to "swscan.apple.com." (with the trailing dots!)
5. In a terminal window, run the following script to create all the necessary directories and symlinks:
# Create required directory for 10.5.x/10.6.x catalogs
sudo mkdir /usr/share/swupd/html/content/catalogs/others/ # Add symlink for 10.6.x updates
sudo ln -s /usr/share/swupd/html/content/catalogs/index.sucatalog /usr/share/swupd/html/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog # Add symlink for 10.5.x updates
sudo ln -s /usr/share/swupd/html/content/catalogs/index.sucatalog /usr/share/swupd/html/content/catalogs/others/index-leopard.merged-1.sucatalog # Add symlink for 10.4.x updates
sudo ln -s /usr/share/swupd/html/content/catalogs/index.sucatalog /usr/share/swupd/html/content/catalogs/index-1.sucatalog # Add swupd symlink to default webserver directory
sudo ln -s /usr/share/swupd/html/content /Library/WebServer/Documents/content # Add symlink for stats engine
sudo ln -s /usr/share/swupd/cgi-bin/SoftwareUpdateServerStats /Library/WebServer/Documents/WebObjects/SoftwareUpdatesStats
On September 21, 2010 at 10:06 PM, Jesse Endahl wrote:
Unfortunately there are a lot of bits and pieces that will make all of the above information break if you are trying this on 10.6 Snow Leopard. Assuming you only have one Mac OS X Server and that it is also acting as your DNS server, the instructions below are the only ones that will work. I've compiled the information from this blog, this blog's comments, and comments on Mac OS X hints (one of them by JLG).

In the DNS service, add a master zone named "swscan.apple.com." (with the trailing dot!). For the nameserver, just click in the area at the bottom and go with the default. Hit save.
Create an A (machine) record that maps "swscan.apple.com." (with the trailing dot!) to the IP of your Mac OS X Server.
Repeat the above steps for swcdn.apple.com and swquery.apple.com

Now, so that Mac OS X Server can still see the real server IPs and pull down the updates, add the following to /etc/hosts

17.250.248.95 swscan.apple.com
17.250.248.93 swquery.apple.com
74.203.241.19 swcdn.apple.com
74.203.241.25 swcdn.apple.com

Then flush the dns cache and check your work:

dscacheutil -flushcache
dscacheutil -q host -a name swcdn.apple.com
dscacheutil -q host -a name swquery.apple.com
dscacheutil -q host -a name swscan.apple.com

Back up your existing /etc/swupd/swupd.conf file:

sudo mv /etc/swupd/swupd.conf /etc/swupd/swupd.conf.orig

Back up /System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources/swupd.conf with these commands:

cd /System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources

sudo cp swupd.conf swupd.conf.orig

Append these lines to /System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources/swupd.conf:

LoadModule rewrite_module libexec/apache2/mod_rewrite.so
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Darwin/10
RewriteRule ^/index\.sucatalog$ /index-leopard-snowleopard.merged-1.sucatalog
RewriteCond %{HTTP_USER_AGENT} Darwin/9
RewriteRule ^/index\.sucatalog$ /index-leopard.merged-1.sucatalog

Add these two aliases so that there are symlinks in the Webserver Documents folder to the real SUS file locations. Note that these file locations are new in Snow Leopard.

ln -s /var/db/swupd/html/content/ /Library/WebServer/Documents/content
ln -s /usr/libexec/swupd/cgi-bin/SoftwareUpdateServerStats /Library/WebServer/Documents/WebObjects/SoftwareUpdatesStats

Use Server Admin or serveradmin(8) to start the Software Update service.

Note that System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources/swupd.conf could be overwritten by a software update, so you might want to back it up:

sudo cp System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources/swupd.conf System/Library/PrivateFrameworks/SUServer.framework/Versions/A/Resources/swupd.conf.localsus

Enjoy!
hey jesse-

Thanks so much for this!

nick
This article is bad candy.

Rather than being worried about the off-site implications of the defaults write method, you should be concerned about people updating off Apple's website (with its all-updates-enabled policy) when they're off-site, rendering your ASUS server's update enabling/disabling, not to mention your entire pre-deployment update testing regimen, completely useless.

Basically, you'd prefer that when they're off-site, everything you've not approved and everything you've deliberately disabled, floods onto the machine. Here comes the Java update that breaks your CRM system; here comes the Safari update that's incompatible with your student teaching platform; here comes the QuickTime update that's incompatible with your pro audio/video editing system. That's really quite poor. Why do you even bother to test updates at all? (And if you say you don't, then you need to find another line of work.)

If you absolutely must switch ASUS servers, implement crankd instead. Change over ASUS servers when a network transition occurs. In any event, client software update checking does not "hang". If it can't contact a server, it just times out. This also occurs if you're completely off-line.

As an administrator, you should have a mechanism in place already to, at a moment's notice, pump out a revised setting. Login script, MCX, pkg with script payload, whatever. Reconfiguring client computers should be a non-issue.

Like I say, bad candy.
@TGB

excellent point. When we were much smaller this was an effective mechanism. The one you describe is far better from an administrative standpoint.
On December 20, 2010 at 02:25 AM, Network Inventory Software wrote:
Seems like an awful lot of hoops to jump through to do something that you could easily workaround by having your staff schedule their Apple updates for the weekends or overnight. If your office network can’t handle 20 employees making heavy downloads, quite frankly, your problem is worse than just keeping your apple software up to date.
A little off topic, but worthy of comment on this post. I note that a commenter pointed out that this article was copied, or stolen from another individuals work. I rest my case the importance of copyright, patenting products and reinforce the need to identify when we use the written word of someone else. The behaviors of traditional writers must be introduced as a matter of urgency within the real of internet content and editorial writing.
Thanks for the message albert, however, i can't disagree more.

The internet is a living thing, taking an outdated tech document (or how-to in this case) and updating it for current software/hardware while still attributing the source is important to keeping the information relevant and fresh.

That said, this document has become outdated and incorrect since I posted it. How quickly we evolve.
On February 07, 2011 at 11:42 PM, Painted Turtle wrote:
Thanks for all the wonderful information.

There are a few things that I'm still confused about. Hopefully someone can help.

The question:

What part do the following two apple servers actually play in the software update process?
http://swquery.apple.com
 http://swdownload.apple.com


Background Information:
I wanted to setup a Transparent Software Update Server, that is, I wanted to have Mac clients use my software update server, when they were on site, and to be able to use Apples, if they were off site. There are a couple of good articles on how to do this. But I wanted to see if I could find a simpler approach. And that meant that I needed to understand it pretty well. But, I realized that there are some things happing here that I just don't understand.

So, following is what I think I understand with some questions, if anyone already knows more about how Apple's software update process. (Before adding a SUS to the mix.)

To help focus the questions, following is a bit of background.

http://support.apple.com/kb/ht3923 (Article HT3923) says the following need to be accessible via port 80.
• http://swscan.apple.com
 • http://swquery.apple.com
 • http://swdownload.apple.com
 • http://swcdn.apple.com

Doing a forward and reverse lookup shows the following DNS information for these hosts:
swscan.apple.com. 3434 IN A 17.250.248.95
swquery.apple.com. 3261 IN A 17.250.248.93
swdownload.apple.com. 3440 IN A 17.250.248.91
swcdn.apple.com. 3600 IN CNAME swcdn.apple.com.akadns.net.

95.248.250.17.in-addr.arpa. 3600 IN PTR swscan.apple.com.
93.248.250.17.in-addr.arpa. 3600 IN PTR swquery.apple.com.
91.248.250.17.in-addr.arpa. 3600 IN PTR swdownload.apple.com.

Note that swcdn.apple.com is an "alias" for a number of of alkamai servers.

I installed Little Snitch, ran software update (10.6.6 machine) and watched what it tried to connect to. I only ever saw it use the following two servers (please let me know if you got different results.):
http://swscan.apple.com
 http://swcdn.apple.com

I never, ever saw it use the following two servers:
http://swquery.apple.com
 http://swdownload.apple.com

This raises the obvious question: Is access to these two servers necessary? And if so why?

Since these are all on port 80, one should be able to point a web browser, such as Safari at them. All you need to know, is what the full URL is that you want to load.

Combining the information on the following two web sites:
http://www.centenary.org.au/users/robertm/apple-swupd/info/
http://support.apple.com/kb/HT4069

Apple Software Update client queries the following URLs for update catalogs by default:
• 10.6 client: http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
• 10.5 client: http://swscan.apple.com/content/catalogs/others/index-leopard.merged-1.sucatalog
• 10.4 client: http://swscan.apple.com/content/catalogs/index-1.sucatalog
• Windows client: http://swcatalog.apple.com/content/catalogs/others/index-windows-1.sucatalog
• 10.3 client: http://swscan.apple.com/scanningpoints/scanningpointX.xml

So, being curious, I replaced "http://swscan.apple.com" with the four apple URLs mentioned in Article HT3923. I found what appear to be identical contents for the following three URLs:
• http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
 • http://swdownload.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
 • http://swcdn.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog

They appear to be XML format lists of various available software downloads. In both cases, the downloads themselves, appear to always be on "swcdn.apple.com" I found no cases where the downloads were listed on any other site. However, there is a single reference to swquery.apple.com, corresponding to the key "ApplePostURL"

Finding an arbitrary iLife download, I test it against the four apple URLs mentioned in Article HT3923. The following three all provide the requested download.

• http://swscan.apple.com/content/downloads/56/33/041-0080/QdvgKMQ3kch6JHc9DfWTzHtJggVyLNck3X/iLifeMediaBrowser.pkg
 • http://swdownload.apple.com/content/downloads/56/33/041-0080/QdvgKMQ3kch6JHc9DfWTzHtJggVyLNck3X/iLifeMediaBrowser.pkg
 • http://swcdn.apple.com/content/downloads/56/33/041-0080/QdvgKMQ3kch6JHc9DfWTzHtJggVyLNck3X/iLifeMediaBrowser.pkg

This seems to suggest that swscan, swdownload, and swcdn all have both the XML files and the actual downloads on them. Except for the fact that the XML files all point to swcdn, they would appear to be interchangeable. But swquery has neither. And oddly, it is listed in the XML files corresponding to the key "ApplePostURL".

It doesn't appear that swdownload.apple.com is used. However, it does appear to have the necessary files on it so that it could be used. swquery.apple.com doesn't even appear to have the necessary files on it. So what role do these server play in the update process?
******** ******** ********
Not part of the question:

If you've read this far and wonder where I'm trying to go, here's the answer. I was looking for an alternate way to allow the Software Update Server to get it's updates, while still having clients connect to it. For example, the software update process doesn't appear to use swdownload.apple.com, yet it does have both the catalogs and the downloads on it. So maybe one could get the Software Update Server to get it's own updates from swdownload.apple.com, while letting the clients get them from swscan.apple.com, which is really your-server.com, etc. This might reduce the DNS tricks necessary. (As a side note, SUS seems to automatically convert all the swcdn.apple.com download references to your-server.com references. So it may be that swcdn.apple.com can be left alone too.)



On February 07, 2011 at 11:49 PM, Painted Turtle wrote:
Link Issue in above posting. Sorry, this is my first posting to this site. And the automatic conversion of URLs into links seems to have screwed up some of them where there is a "•" bullet symbol. I tried copying and pasting a couple of them into Safari, and that seemed to work.
@painted turtle

Ok so basically all you really have to do is to get your clients to use your server to pick up those XML catalogs instead of the apple servers. I'm not sure why they use so many different subdomains, or what their exact purpose is, perhaps they are future or failure proofing their system, or they are for legacy support. Either way all you need to do is get your clients (when they're on your network) to pull the .sucatalog files from your SUS server.

And you're right, when you set up SUS it formats the file such that links to download updates are directed at the hostname of the local server, which is handy IMO.
I can understand how an apple-centric office manager might be concerned about clogging the network with software update downloads but wouldn't the simplest solution be to have all your macs to download updates overnight? It can't be any harder than making sure all you Apples are running the exact same OS versions (something I assume would be a necessity for this local update system to work properly).
I've been able to implement this (mainly with the help from @Jesse Endahl and thanks to you!) and it worked day of. Checking the link for the Apple Update Server from my server points to Apple, from my client machines it points to our server. BUT since doing the this, On March 23rd. The Update Server doesn't show a newer date as last checked. It's still reporting the 23rd on the log and apples logs show April 1st as the last time they updated the catalog. Any advice on what might be causing my Update Server to not check for updates?
On April 11, 2011 at 10:59 AM, Nick wrote:
@Jim we had a similar problem with one of our sus servers. We solved it by reinstalling the OS unfortunately. Admittedly didn't do much troubleshooting to fix though, as we had other issues with the box that required a reinstall anyway.