Mac Daddy World

I spent a lot of time working this issue, so I’ll go into some detail here in the hopes that it will help others.

If you take a look at my previous post, you’ll see a list of new restrictions which Leopard puts on Input Manager plugins. Number 3 is: Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager. While I do consider this overkill (given #2), it seemed a very straightforward requirement and sensible if you’re trying to be as safe as possible.

A handful of users weren’t getting our plugins to load, so we created a test application to run down all of the requirements one by one and see which one was failing. To our surprise, we found the test application (and presumably all of their apps) was running geteuid() == 0! Yes, all running as super user, root, the big 0. A quick check in Activity Monitor revealed that sure enough, most every process on the machines in question was running as root. Now remember, they’re not logged in as root. They’re logged in as their normal user; let’s call him Steve. Whenever Steve opens a program from the Dock, it runs with root privileges instead of Steve privileges. This is nasty. Steve doesn’t even know it, but I’m sure his system is acting rather strange. Most people with this problem reported “other stability issues” and eventually did a clean install of Leopard (which fixed the problem.)

I asked around on the Apple-cdsa listserv to see if anyone had seen this problem. I quickly heard from another developer who had already seen the same issue, but nobody else on the list had seen it and nobody had a quick solution.

I went back on a few weeks later asking for any help with troubleshooting the problem. This was fruitful. Although it wasn’t entirely on topic, heads were scratched, hallway discussions were had, and I heard from an Apple engineer who quickly guessed at the actual cause of this.

Apparently, some major changes have been made to how Leopard launches applications.

If you’ve spent the last several years seeing Tiger crash reports, you’ve probably become quite familiar with seeing:

Parent: WindowServer [123]

On Tiger, the very first process (PID 1) is launchd, which launches all daemons, including WindowServer, which is in turn the parent of all of your “apps”. launchd runs as root, WindowServer runs as windowserver, and your apps should all be running as, well, as you.

On Leopard, you might have noticed that the parent process in crash reports has now become launchd. When you login on Leopard, the PID 1 launchd forks a second launchd, which is the parent of all your apps. This second launchd runs as your normal user.

In our messed up case, this second launchd was running as root.

On these affected machines /sbin/launchd had its setuid bit set. The setuid attribute is a permissions bit flag, which causes a program to run as root even if a non-privileged user executes it. Sound like bad idea jeans to you too? Set this bit on launchd, and that’s exactly what happens: It always runs as root no matter who starts it.

launchd is normally setuid on Tiger, but it’s not supposed to be on Leopard. I’m assuming this is some kind of 10.4 -> 10.5 migration issue.

It’s not hard to turn off the setuid bit and fix this problem, but the user will have lingering ownership issues. Every file they’ve created since upgrading to Leopard is going to be owned by root and unwritable by their normal user. This isn’t hard to fix but who knows what other weird issues they’ll see. Yuck. Clean install highly recommended.

There are obvious reasons why having all your apps running with root privileges is bad, especially if your machine is being used by non-admin users.

It’s interesting that a single bit flag can give you super user privileges on your own machine. If at least half dozen of my customers had this problem, then there must be thousands of users out there running as root. At least one person at Apple is now aware of the issue, so hopefully there will eventually be a fix for this.

What exactly caused this bit to be set for these unfortunate users? This is the one question we still don’t have an answer to.

In brief, an Input Manager is a plugin architecture in Cocoa, originally intended as a way to provide alternative text input methods to NSTextViews. Whenever a Cocoa application launches, it loads all the Input Managers that it finds in the InputManagers folders. Apple provides sample code with their dev tools for making an Input Manager called HexInputServer.

Input Manager plugins are now commonly used as a quick and easy way to make plugins for any Cocoa app. At Ecamm, we use a single simple Input Manager, which loads any of our plugins that you have installed such as DockStar, iGlasses, or Call Recorder into their appropriate applications.

The Mark Twain Effect

We received a lot of emails in the months leading up to Leopard with people asking, “Hey, I heard Input Managers are not going to work in Leopard. What are you going to do?” I don’t know where they heard this, because at no point in the many Leopard pre-releases did Apple completely remove Input Manager support. In short, rumors of their death were greatly exaggerated.

We worked our butts off and on October 26th, when the final release of OS X 10.5 rolled off the factory floor, all of our Input Manager-based plugins were ready for Leopard and working quite well.

The Truth About Input Managers

So what’s the deal, what are we doing about it you might ask?

In response to security concerns about InputManagers, Apple decided to place a whole slew of new restrictions on whether or not to allow them to load into apps. These restrictions are very briefly summarized in the 10.5 AppKit Release Notes, and I’ll excerpt that here:

1. The valid installation is now restricted to the /Library/InputManagers folder only. Bundles in other locations are silently ignored.
2. All the files in the bundle and /Library/InputManagers folder itself must be owned by the root user and admin group. No files inside the bundle can have group or other write permissions.
3. Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager.
4. Processes running with the wheel group privilege cannot load any bundle input manager.
5. The process must be in the active workspace session at the time of loading the bundles.
6. The process must not be tainted by changing user or group id (checked by issetugid()).
7. No 64-bit processes can load any bundle input managers.

So what does it all mean?

Let’s go over each restriction in detail:

1. The valid installation is now restricted to the /Library/InputManagers folder only. Bundles in other locations are silently ignored.

On 10.4 and earlier, Input Managers can be installed in the root Library (/Library/InputManagers) or your user Library (/Users/[your user]/Library/InputManagers). If there is an Input Manager with the same name in both places, it completely ignores the one in the root Library and uses the home Library instead.

On 10.5, only Input Managers in the root Library are allowed to load. However, if there’s an Input Manager with the same name in the home Library, it will ignore the one in the root Library and load neither.

Ecamm’s Input Manager has always lived in the root Library so this change did not affect us. Additionally, because people sometimes manually move stuff around, our installer has always zapped any Ecamm Input Manager it finds in the home Library.

This change requires all Input Managers be global (affecting all users). There’s no longer any way for developers to provide the option to install only for the current user. This will be a real pain for unprivileged users, as they will no longer be able to install Input Managers without an admin password. We did have a customer who is an instructor, and uses Call Recorder in a locked-down computer lab environment. We had helped him install Call Recorder in the user Library. This will no longer be possible, at least not possible using Input Managers.

2. All the files in the bundle and /Library/InputManagers folder itself must be owned by the root user and admin group. No files inside the bundle can have group or other write permissions.

In 10.5, every file in an Input Manager plugin’s directory is recursively checked to make sure they meet these ownership and permissions requirements. Additionally, the InputManagers directory itself must meet these same requirements.

In a pinch, the following commands can be used to fix ownership and permissions of the InputManagers folder:

sudo chown -R root:admin /Library/InputManagers
sudo chmod -R go-w /Library/InputManagers

As of their latest versions, the Ecamm installers automatically fix permissions and ownership.

This requirement was the main hurtle to getting Input Managers working on Leopard, but it makes sense to have these requirements if the goal is make this all safer.

3. Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager.

This one is pretty self explanatory as they’ve given the code they’re using to make the check. If getuid() == 0 , it means you’re logged into your Mac as root. I’m not sure why someone would do this or if it’s even possible. geteuid() == 0 would indicate that the process is running effectively as root (e is for effective). A process is running effectively as root if you run it using sudo, if it is started programmatically using AuthorizationExecuteWithPrivileges, or if it’s running with the setuid bit.

This requirement seems like overkill. The Input Manager has to be owned by root per #2, so why can’t root run its own plugin? Oh well, no big deal. This keeps Input Managers out of sketchy places like loginwindow.

UPDATE: We’ve had a handful of users with all their apps running with geteuid()==0. We don’t yet know why it happens.

4. Processes running with the wheel group privilege cannot load any bundle input manager.

In 10.5, the process’s primary group cannot be wheel (wheel is the name of the root group, gid == 0). Both getgid() and getegid() are checked for 0.
Additionally, the process owner’s supplementary groups are checked for wheel.

This last bit is the one that caught us off guard. We eventually determined that a handful of users who reported problems getting our plugins to load were actually failing this requirement. Their main admin user was a member of the wheel group! This situation is most likely the result of migrating a user account forward from OS X 10.1 (Puma) or earlier, where all admin users were added to the wheel group by default.

If our installer finds that the current user has wheel in its supplemental group list, it simply removes it. Because of the way Cocoa is checking the supplemental groups, a full reboot is required before the change takes effect and Input Managers will load.

5. The process must be in the active workspace session at the time of loading the bundles.

Having an inactive workspace session means that your user is currently switched out via “Fast User Switching”. Fast User Switching allows user B to login without making user A, who was already logged in, have to quit all his programs. It’s then possible to quickly switch back and forth between the two users.

Since Input Managers load when an application starts, I’m not completely sure how an application can start while a user is switched out or what the implications of this are. Anyone have an idea?

6. The process must not be tainted by changing user or group id (checked by issetugid()).

issetugid() makes sure that neither setuid nor setgid have been called. In short, this function makes sure that the process is currently running with the same uid and gid that it was given at birth (execve).

7. No 64-bit processes can load any bundle input managers.

This is an interesting requirement. I can’t think of any good reason for this.

It turns out that checks 2 through 6 are not made when running in 64-bit. Instead, the validation routine simply returns NO.

If they were to allow 64-bit Input Managers, and we actually wanted to have them load into a 64-bit app, we’d simply have to rebuild them with 64-bit architectures.

If you thought things got confusing back when binaries were PowerPC, Intel, or both (Universal), then you’re in for a treat. There’s now up to 4 possible architectures that can be in one mach-o file: PowerPC, Intel, PowerPC 64-bit, and Intel 64-bit. The dynamic loader can’t mix and match architecture types inside one process. So if you want a plugin to load into a 64-bit application, the plugin will have to be built with 64-bit as one of its architectures. To get people around problems like this, Apple has provided a “Open as 32 Bit” checkbox next to the “Open using Rosetta” checkbox. If, for example, Photoshop is made 64-bit capable, then you’ll need to use this new checkbox if you want it to be able to load any of your “old” 32-bit-only plugins.

Another problem plugin developers are going to have on Leopard involves the new Garbage Collection (GC) in Objective-C 2. A process running with GC turned on cannot load non-GC plugins. It’s a little bit more complicated than this, but I won’t go into details.

A strange turn of events

As a result of check #3, Input Managers no longer load into certain processes where you’d probably agree you don’t want Input Managers.

For example, on 10.4 and earlier, Input Managers load into loginwindow. This is no longer the case on 10.5.

However, in a very interesting turn of events, Input Managers now load into the Finder. Go figure.

To quote from the immortal words of They Might Be Giants:

“Where’s my hovercraft?
Where’s my jet pack?
Where’s the font of acquired wisdom that eludes me now?”

Sometimes it seems like we really are stuck in a World Before Later On, but then, when you least expect it, you realize that it’s the future. The future, suddenly, is now.

Secure Future

Take this comment from deep in the Security framework for example:

/*
... You need to aquire [sic] this right to be able to perform a AuthorizationExecuteWithPrivileges() operation. In addtion [sic] to this right you should obtain whatever rights the tool you are executing with privileges need to perform it’s [sic] operation on your behalf. Currently no options are supported but you should pass in the full path of the tool you wish to execute in the value and valueLength fields. In the future we will limit the right to only execute the requested path, and we will display this information to the user.
*/


I was in the process of debugging something that used to work but had stopped working on a new version of OS X. Suddenly I realized that the future was here, now. It sent shivers down my Spine.

Corrupt Future

Remember floppy disks? Remember colorful 800kb floppy disks? I clearly remember them, and will admit to having boxes full of hundreds of them. They’re filled with everything Glen and I did on a computer for 10 years: From 1988 when we got our first Mac, up until when floppy disks went the way of ADB, built-in modems, trackballs, Netscape, and the Palm Pilot. In fact, as early as 1998, the very first iMac boldly omitted a floppy disk drive. So it’s been nearly 10 years since Apple began phasing out the little suckers.

To get back to my point: I also clearly remember conversations to the effect of, “Someday in the future we won’t be able to use these disks. We won’t have a computer that reads them or even programs that will open the files. Also, aren’t these things only supposed to last about 15 years before going bad?”

Well, I do still have a computer that can read the disks… but all of the word processing files are in Microsoft Works 2.0 format. Guess what? Nothing reads that format anymore. Most of the drawings I did when I was 10 are in SuperPaint 1.0 or CricketPaint. Not sure how I’ll look at those again. You might be interested to know that Preview.app will open MacPaint files. Maybe a running inside joke at Apple?

Oh yeah, guess what else. Most of the disks that date from before 1992 have bad blocks… Yeah… they’re dropping like flies.

The future is now.

Unexpected Futures

With OS X 10.5 due out next month, and likely a team in Cupertino already hard at work on OS X 10.6, somebody in Apple marketing is tossing and turning in their sleep: “How long can we keep raising the number? How many big cat varieties are there anyways? What if we run out of cool sounding species? What happens when we get past 10.9? 10.10? I just don’t think that sounds right. I still can’t even believe they went with 10.4.10. Oh god I think I’m going to be fired.”

Certain Future

If there’s one thing we can be sure of, it’s that the future will arrive.

And I’ll close with some more They Might Be Giants:
You’re older than you’ve ever been.
And now you’re even older.
And now you’re older still.

Antone Gonsalves posted a great article to Information Week on Friday titled Renegade Developers Zero In On Apple iPhone OS. It includes some hilarious and informative quotes from yours truly. Here’s my favorite quote:

But so-called “native” applications are not for mainstream consumers. “There’s no easy way to get applications on the iPhone,” Aspeslagh said. “You have to make changes to your iPhone, and that really freaks people out. It’s a procedure that could turn your iPhone into a paperweight, if it’s not done correctly.”

Fun stuff. Let’s just hope it all has a happy ending.

Very cool. I’m quoted in today’s Boston Globe. It’s a decent article about the iPhone being closed to 3rd party developers. It’s humbling to be quoted before Bob Borchers, “Apple’s senior director of iPhone worldwide product marketing”.

It’s odd to think of Apple as potentially stifling innovation. Sometimes they seem like the only innovative company around. My point here is that each 3rd party app that could be written for the iPhone would immediately increase the usefulness of the phone. Bray (the reporter from the Globe) asked me “Why do we need 3rd party apps? What kind of apps might be useful on the iPhone?” The answer to this question is that the possibilities are only limited by the imaginations of the developers. That’s part of what makes software development so exciting: The real killer apps are never predictable. You could be vague and say: business apps, creative tools, GAMES! But the real innovative ones will catch us all by surprise; I promise. And you can quote me on that.