Mac Daddy World

You can separate the OS X feature engineering work done by Apple into two categories. There’s the features that are exciting to the typical Mac user, and then there’s the features that are only exciting to software developers. When Apple says that they’re “hitting the pause button on new features”, they’re talking about that first category of features. From a developer’s perspective, Snow Leopard is overflowing with exciting system enhancements, tantalizing new goodies to allow for faster development and huge performance improvements.

The tradeoff triangle. Image credit: Microsoft

Pausing to focus on stability and performance is every large software company’s fantasy. The typical process of software development is a cycle: New feature requirements are handed down from on high, developers work furiously on the new features, cut corners to get them just to the point where they’re (mostly) working, and then rush them out the door on a slipped deadline. Fixes to critical bugs come iteratively as point release updates, but the majority of the engineering team has already moved on to the next batch of new features. There’s never any time to go back and improve on things.

It’s unprecedented for a company to have this luxury: To be able to pause the infinite loop and concentrate entirely on making the current feature set work better. In the large part, working better means running faster. It also means adding new frameworks and paradigms to allow future features to be built on a firm foundation. They’re improving the underpinnings of the OS. You can think of it like Verizon’s FIOS project. Customers might not see a huge benefit today, but by replacing their copper infrastructure with fiber-optic, they’re enabling future technologies. Similarly, Apple is building a powerful new infrastructure for the future of the platform. Competitors are going to be left with crackly old wiring.

A winning strategy

Clearly, I’m very excited by this strategy. By taking a whole engineering cycle to focus on enhancements and create performance-improving technologies, Apple is going to move so far ahead of competition that the competition might never be able to recover. They’re going to skunk the competition.

Concentrating on performance improvements isn’t something new for Apple. I haven’t verified this scientifically, but from personal experience, upgrading to Leopard actually results in a speed boost for things like boot time and application launch time. This comes as a surprise to many people. We just assume that a newer OS will require a faster machine and say things like: “I’m not sure my old laptop can handle Leopard.” In fact, Leopard is a faster operating system than its predecessors. Upgrading will actually result in a performance increase.

The things I saw this week at WWDC made my eyes bug out. They’re determined to squeeze every last cycle out of the CPU (and the GPU for that matter), constantly asking the question “How can we make this faster?”. I imagine a big banner, a la Office Space, hanging above the cubicle farm in Cupertino. Another important focus is enabling 3rd party developers to easily (or even automatically) benefit from these new technologies.

With each normal development cycle, performance might move up by some small amount. (In the competition’s case, it might actually go down.) In the Snow Leopard release, it’s going to make a huge jump.

Putting all your cores in one basket

While every shipping Mac is now a multi-core machine, programming languages and application architectures have not kept up in a way that allows developers to easily take full advantage. Until now, efficiently and properly taking advantage of all available cores has been a tricky and error-prone process even for the most brilliant of engineers. Snow Leopard will solve this problem in many ways, with new language features, and a new operation paradigm which shifts the burden of threaded programming away from the developer and into the capable hands of the OS.

Who needs so many bits?

For developers, moving to 64-bit is a cheerless and time consuming process, made even worse by the fact that consumers will not even notice an immediate difference. Apple is going to do it in Snow Leopard because they know it will make things better over the long run. It’s like getting your wisdom teeth removed when they’re not yet causing any discomfort. It’s going to need to happen at some point, and better to do it now when you can schedule a convenient time for the procedure. You won’t feel better right afterwards. In fact, for a short period of time afterwards you’ll feel worse. But once you’re recovered, you’ll be glad you did it. Apple’s investing in a healthy future. Competitors are going to end up wearing braces.

As hip as a software engineer

I love that Apple doesn’t try to use the current buzzwords and trendy technologies. They could be trying to generate investor excitement with things like XML and social networking, but they’re not. Instead they’re putting lay-people to sleep with things like OpenCL and gigaflops. It’s as if they’ve put the developers in charge and everyone else is out to lunch.

Way to go Apple. Power to the Programmers!

We rolled out a cool new version of DockStar on Christmas day. The feature everyone’s talking about is the clickable indicators in the menu bar.

Before I was even finished coding this feature, I already knew I couldn’t live without it. Glen used more explicit terms; something leading to him and the new feature having babies.

In short, you can see unread counts for any mailbox or folder up in the status area, and a simple click on the indicator pops open Mail.app and brings up the right mailbox. This was a requested feature from DockStar fans. (We get a lot of our best ideas from customers.) It’s not limited to unread counts either: You can set each indicator to count flagged messages, total message counts, and even monitor Smart Mailboxes.

This upgrade brings many new features for Leopard users. If you use To Do items in Mail.app or Calendar.app, you can use DockStar to monitor the number of incomplete To Dos. You can also count Notes or even keep track of unread RSS feed items.

The DockStar Dashboard Widget also got some improvements. You can now click on the widget to jump right into the relevant mailbox.

If you’re a serious emailer, and you’ve never tried DockStar, you should try the free trial, but be warned: Afterwards, you won’t be able to live without it.

I spent a lot of time working this issue, so I’ll go into some detail here in the hopes that it will help others.

If you take a look at my previous post, you’ll see a list of new restrictions which Leopard puts on Input Manager plugins. Number 3 is: Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager. While I do consider this overkill (given #2), it seemed a very straightforward requirement and sensible if you’re trying to be as safe as possible.

A handful of users weren’t getting our plugins to load, so we created a test application to run down all of the requirements one by one and see which one was failing. To our surprise, we found the test application (and presumably all of their apps) was running geteuid() == 0! Yes, all running as super user, root, the big 0. A quick check in Activity Monitor revealed that sure enough, most every process on the machines in question was running as root. Now remember, they’re not logged in as root. They’re logged in as their normal user; let’s call him Steve. Whenever Steve opens a program from the Dock, it runs with root privileges instead of Steve privileges. This is nasty. Steve doesn’t even know it, but I’m sure his system is acting rather strange. Most people with this problem reported “other stability issues” and eventually did a clean install of Leopard (which fixed the problem.)

I asked around on the Apple-cdsa listserv to see if anyone had seen this problem. I quickly heard from another developer who had already seen the same issue, but nobody else on the list had seen it and nobody had a quick solution.

I went back on a few weeks later asking for any help with troubleshooting the problem. This was fruitful. Although it wasn’t entirely on topic, heads were scratched, hallway discussions were had, and I heard from an Apple engineer who quickly guessed at the actual cause of this.

Apparently, some major changes have been made to how Leopard launches applications.

If you’ve spent the last several years seeing Tiger crash reports, you’ve probably become quite familiar with seeing:

Parent: WindowServer [123]

On Tiger, the very first process (PID 1) is launchd, which launches all daemons, including WindowServer, which is in turn the parent of all of your “apps”. launchd runs as root, WindowServer runs as windowserver, and your apps should all be running as, well, as you.

On Leopard, you might have noticed that the parent process in crash reports has now become launchd. When you login on Leopard, the PID 1 launchd forks a second launchd, which is the parent of all your apps. This second launchd runs as your normal user.

In our messed up case, this second launchd was running as root.

On these affected machines /sbin/launchd had its setuid bit set. The setuid attribute is a permissions bit flag, which causes a program to run as root even if a non-privileged user executes it. Sound like bad idea jeans to you too? Set this bit on launchd, and that’s exactly what happens: It always runs as root no matter who starts it.

launchd is normally setuid on Tiger, but it’s not supposed to be on Leopard. I’m assuming this is some kind of 10.4 -> 10.5 migration issue.

It’s not hard to turn off the setuid bit and fix this problem, but the user will have lingering ownership issues. Every file they’ve created since upgrading to Leopard is going to be owned by root and unwritable by their normal user. This isn’t hard to fix but who knows what other weird issues they’ll see. Yuck. Clean install highly recommended.

There are obvious reasons why having all your apps running with root privileges is bad, especially if your machine is being used by non-admin users.

It’s interesting that a single bit flag can give you super user privileges on your own machine. If at least half dozen of my customers had this problem, then there must be thousands of users out there running as root. At least one person at Apple is now aware of the issue, so hopefully there will eventually be a fix for this.

What exactly caused this bit to be set for these unfortunate users? This is the one question we still don’t have an answer to.

To quote from the immortal words of They Might Be Giants:

“Where’s my hovercraft?
Where’s my jet pack?
Where’s the font of acquired wisdom that eludes me now?”

Sometimes it seems like we really are stuck in a World Before Later On, but then, when you least expect it, you realize that it’s the future. The future, suddenly, is now.

Secure Future

Take this comment from deep in the Security framework for example:

/*
... You need to aquire [sic] this right to be able to perform a AuthorizationExecuteWithPrivileges() operation. In addtion [sic] to this right you should obtain whatever rights the tool you are executing with privileges need to perform it's [sic] operation on your behalf. Currently no options are supported but you should pass in the full path of the tool you wish to execute in the value and valueLength fields. In the future we will limit the right to only execute the requested path, and we will display this information to the user.
*/


I was in the process of debugging something that used to work but had stopped working on a new version of OS X. Suddenly I realized that the future was here, now. It sent shivers down my Spine.

Corrupt Future

Remember floppy disks? Remember colorful 800kb floppy disks? I clearly remember them, and will admit to having boxes full of hundreds of them. They’re filled with everything Glen and I did on a computer for 10 years: From 1988 when we got our first Mac, up until when floppy disks went the way of ADB, built-in modems, trackballs, Netscape, and the Palm Pilot. In fact, as early as 1998, the very first iMac boldly omitted a floppy disk drive. So it’s been nearly 10 years since Apple began phasing out the little suckers.

To get back to my point: I also clearly remember conversations to the effect of, “Someday in the future we won’t be able to use these disks. We won’t have a computer that reads them or even programs that will open the files. Also, aren’t these things only supposed to last about 15 years before going bad?”

Well, I do still have a computer that can read the disks… but all of the word processing files are in Microsoft Works 2.0 format. Guess what? Nothing reads that format anymore. Most of the drawings I did when I was 10 are in SuperPaint 1.0 or CricketPaint. Not sure how I’ll look at those again. You might be interested to know that Preview.app will open MacPaint files. Maybe a running inside joke at Apple?

Oh yeah, guess what else. Most of the disks that date from before 1992 have bad blocks… Yeah… they’re dropping like flies.

The future is now.

Unexpected Futures

With OS X 10.5 due out next month, and likely a team in Cupertino already hard at work on OS X 10.6, somebody in Apple marketing is tossing and turning in their sleep: “How long can we keep raising the number? How many big cat varieties are there anyways? What if we run out of cool sounding species? What happens when we get past 10.9? 10.10? I just don’t think that sounds right. I still can’t even believe they went with 10.4.10. Oh god I think I’m going to be fired.”

Certain Future

If there’s one thing we can be sure of, it’s that the future will arrive.

And I’ll close with some more They Might Be Giants:
You’re older than you’ve ever been.
And now you’re even older.
And now you’re older still.