Mac Daddy World

We rolled out a cool new version of DockStar on Christmas day. The feature everyone’s talking about is the clickable indicators in the menu bar.

Before I was even finished coding this feature, I already knew I couldn’t live without it. Glen used more explicit terms; something leading to him and the new feature having babies.

In short, you can see unread counts for any mailbox or folder up in the status area, and a simple click on the indicator pops open Mail.app and brings up the right mailbox. This was a requested feature from DockStar fans. (We get a lot of our best ideas from customers.) It’s not limited to unread counts either: You can set each indicator to count flagged messages, total message counts, and even monitor Smart Mailboxes.

This upgrade brings many new features for Leopard users. If you use To Do items in Mail.app or Calendar.app, you can use DockStar to monitor the number of incomplete To Dos. You can also count Notes or even keep track of unread RSS feed items.

The DockStar Dashboard Widget also got some improvements. You can now click on the widget to jump right into the relevant mailbox.

If you’re a serious emailer, and you’ve never tried DockStar, you should try the free trial, but be warned: Afterwards, you won’t be able to live without it.

I spent a lot of time working this issue, so I’ll go into some detail here in the hopes that it will help others.

If you take a look at my previous post, you’ll see a list of new restrictions which Leopard puts on Input Manager plugins. Number 3 is: Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager. While I do consider this overkill (given #2), it seemed a very straightforward requirement and sensible if you’re trying to be as safe as possible.

A handful of users weren’t getting our plugins to load, so we created a test application to run down all of the requirements one by one and see which one was failing. To our surprise, we found the test application (and presumably all of their apps) was running geteuid() == 0! Yes, all running as super user, root, the big 0. A quick check in Activity Monitor revealed that sure enough, most every process on the machines in question was running as root. Now remember, they’re not logged in as root. They’re logged in as their normal user; let’s call him Steve. Whenever Steve opens a program from the Dock, it runs with root privileges instead of Steve privileges. This is nasty. Steve doesn’t even know it, but I’m sure his system is acting rather strange. Most people with this problem reported “other stability issues” and eventually did a clean install of Leopard (which fixed the problem.)

I asked around on the Apple-cdsa listserv to see if anyone had seen this problem. I quickly heard from another developer who had already seen the same issue, but nobody else on the list had seen it and nobody had a quick solution.

I went back on a few weeks later asking for any help with troubleshooting the problem. This was fruitful. Although it wasn’t entirely on topic, heads were scratched, hallway discussions were had, and I heard from an Apple engineer who quickly guessed at the actual cause of this.

Apparently, some major changes have been made to how Leopard launches applications.

If you’ve spent the last several years seeing Tiger crash reports, you’ve probably become quite familiar with seeing:

Parent: WindowServer [123]

On Tiger, the very first process (PID 1) is launchd, which launches all daemons, including WindowServer, which is in turn the parent of all of your “apps”. launchd runs as root, WindowServer runs as windowserver, and your apps should all be running as, well, as you.

On Leopard, you might have noticed that the parent process in crash reports has now become launchd. When you login on Leopard, the PID 1 launchd forks a second launchd, which is the parent of all your apps. This second launchd runs as your normal user.

In our messed up case, this second launchd was running as root.

On these affected machines /sbin/launchd had its setuid bit set. The setuid attribute is a permissions bit flag, which causes a program to run as root even if a non-privileged user executes it. Sound like bad idea jeans to you too? Set this bit on launchd, and that’s exactly what happens: It always runs as root no matter who starts it.

launchd is normally setuid on Tiger, but it’s not supposed to be on Leopard. I’m assuming this is some kind of 10.4 -> 10.5 migration issue.

It’s not hard to turn off the setuid bit and fix this problem, but the user will have lingering ownership issues. Every file they’ve created since upgrading to Leopard is going to be owned by root and unwritable by their normal user. This isn’t hard to fix but who knows what other weird issues they’ll see. Yuck. Clean install highly recommended.

There are obvious reasons why having all your apps running with root privileges is bad, especially if your machine is being used by non-admin users.

It’s interesting that a single bit flag can give you super user privileges on your own machine. If at least half dozen of my customers had this problem, then there must be thousands of users out there running as root. At least one person at Apple is now aware of the issue, so hopefully there will eventually be a fix for this.

What exactly caused this bit to be set for these unfortunate users? This is the one question we still don’t have an answer to.

Following a continuing trend of adding new things without telling anyone, the recent QuickTime 7.2 update includes more than just bug fixes.
We’ve confirmed that the QTKit Capture functionality previously billed as a feature of Leopard is included with the update. (See the section on QuickTime Improvements.)

The fun new functionality is all there after updating to 7.2, but there are no new header files (no public API yet). Also, the functionality is now used by QuickTime Player Pro. However, you probably won’t notice much of a change except a little progress indicator in the Recording section of the prefs when it’s scanning for cameras, and hopefully a performance improvement.

Since it’s in the wild, I can now talk freely about this cool new feature of QuickTime. All of this I learned simply by hunting around and looking at sample traces of a running video stream in the new version of QuickTime Player.

QTKit Capture looks to be a total ground-up replacement of the Sequence Grabber. The Sequence Grabber is a very ancient part of QuickTime, designed long ago as the way to capture streaming audio/video. It doesn’t appear to be going away, but QTKit Capture doesn’t use it internally except for reverse compatibility* with old vdig drivers. Where the Sequence Grabber uses QuickDraw, QTKit Capture makes heavy use of Core Video and OpenGL. Also, the top layer is Objective-C so it can be integrated easily into a Cocoa app.

I’ve done some looking around and found some interesting things:

It makes use of some new private frameworks. New to the club are:
CoreMediaAuthoringPrivate.framework
CoreMediaIOServicesPrivate.framework
CoreMediaPrivate.framework

CoreMediaIOServicesPrivate’s framework Resources folder contains a number of plugin modules. There’s one for AVC (DV cams), IIDC (external iSight), and VDC (the built-in iSight). In a nutshell, CoreMedia appears to implement a Device Abstraction Layer (A way to allow access to all video devices without having to know the details of the device.)

Other notes:

• * QTKit Capture still works with existing QuickTime component vdig modules (macam still works). In this case, it is actually running the Sequence Grabber below CoreMedia. For the built-in iSight however, the Sequence Grabber is not being used. Hopefully, Apple will provide a new way to write plugins for third party devices so the SequenceGrabber doesn’t have to be used. However, they do support 3 major video camera standards (IIDC, AVC, and UVC), so device developers going forward would be wise to implement one of these standards, eliminating the need for writing their own driver.
• You can build and link against the new functionality if you have the proper header files.

Often times, you want to change your program’s behavior based on available CPU Mhz (or GHz as is more common nowadays). For example, Call Recorder won’t default to using AAC encoding for audio recording if your Mac is under 800 Mhz.

So let’s say you want to find out the current CPU speed. It’s common to use the ancient but useful sysctl call.

int args[] = {CTL_HW, HW_CPU_FREQ};
int freq = 0;
size_t len = sizeof(freq);
if (sysctl(args, 2, &freq, &len, NULL, 0) == 0)
     printf("It's %d Hz\n", freq);


A Gold Star if you can see what’s wrong with that code.

Wait for it…

Since sysctl returns the clock speed in tiny little old Hz (1 billionth of a GHz), we’re quickly running out of space in our 32-bit int. We’d overflow an unsigned int when we hit 4,294,967,296 Hz and we’ll overflow the signed int we used in the code above at only 2,147,483,648 Hz.

But that’s only 2.14 Ghz.

So on my 2.33 Ghz MacBook Pro, the code above will return:
It’s -1,964,967,296 Hz.
Oops…

Obviously, using an unsigned int instead of a normal int would have been wiser. However, even that will overflow when clock speeds double (in 2 years if you trust Moore’s law.)

Indeed, the solution is to use a long long or int64 (whichever camp you’re from.)

Here’s better code, using sysctlbyname instead, which is the recommended way to do it nowadays:

uint64_t freq = 0;
size_t size = sizeof( freq );
if (sysctlbyname( "hw.cpufrequency_max", &freq, &size, NULL, 0 ) == 0)
     printf("It's %lld Hz\n", freq);


This introduces a new overflow point when CPU speeds exceed 18,446,744,073 Ghz.

Interestingly, if you believe that Moore’s law will continue to hold, we should reach these speeds around 2071. That is, of course, assuming that the human race survives the Year 2038 Problem which will cause a similar overflow issue with the way OS X keeps track of the date seconds in an unsigned int.

Switching the date seconds to an int64 will cause a new problem in about 290 billion years.
However, due to predicted changes to the Solar System, it is not widely regarded as a pressing issue.

Also, don’t forget about good old OS 9’s Gestalt routine. You can’t use gestaltProcClkSpeed anymore for the same reason. (It returns a long.) But Gestalt also has a selector gestaltProcClkSpeedMHz, which drops the six useless zeros.

The iApps like iChat and Photo Booth only request a 640×480 or smaller image from the iSight. Therefore it can be troublesome to find a way to test the full capabilities of the new 1.3 MP iSight. That’s why we wrote our own little test program.

Today we cleaned it up and are posting it here so new-MacBook-Pro-having visitors can try it out see the new and improved image for themselves.

The app requests and displays a video stream from the camera at 1280×1024. If you’re on an older iSight it will still stretch the VGA image out to be this size but it won’t look very good. The app will report the frame actual size from the ImageDescription at the top.

There’s also a snapshot button so you can take your own picture and make your friends jealous of your fancy new laptop. Sorry: no Photo Booth effects yet!

If you get to try it, leave a comment and let me know how it works for you.
The Test App (68 k)

Tech Talk Follows:

The app does Hi-Lite™ a potential reason why Apple might not be yelling from the rooftops about the new camera: You can see right away how slow it is to stream the full sized video. Why? It’s more than four times more data than VGA. The Sequence Grabber was designed decades ago and wasn’t designed to handle more than NTSC/PAL sizes. Until the Sequence Grabber is revamped, it’s a little too slow to run such a large stream.

1,280 x 1,024 = 1,310,720 pixels
It’s 4:2:2 compressed YUV data, so that’s 2 bytes per pixel or 2,621,440, or 2.5 megabytes per raw frame.
So for full 30 fps video, this translates to about 75 megabytes per second or 600 Mbps.
Now keep in mind the camera controller allows for a compressed MJPEG mode which cuts the required USB bandwidth signifcantly. (You’d never fit that over USB 2.0 without compression.)

But even if it is being compressed on-camera, it’s then being decompressed in the driver and passed to QuickTime in YUV. That’s a lot of data to push through the poor Sequence Grabber pipes. Technically, you should be able to request MJPEG data from the sequence grabber, but the default seems to be YUV. I haven’t tried requesting other compression types as I don’t actually have one of these new machines in front of me.*

More details about the new camera:

The original iSight didn’t advertise itself as a UVC device (It was Vendor Specific).
The new one does advertise as UVC, so you can easily use USB Prober to decode the device descriptor:
According to the descriptor the camera supports the following frame formats:

MJPEG 640×480 @ up to 60fps
MJPEG 720×480 @ up to 60fps
MJPEG 800×600 @ up to 30fps
MJPEG 1024×576 @ up to 30fps
MJPEG 1024×768 @ up to 30fps
MJPEG 1280×960 @ up to 30fps
MJPEG 1280×1024 @ up to 30fps
Uncompressed 640×480 @ up to 30 fps
Uncompressed 352×288 @ up to 30 fps
Uncompressed 160×120 @ up to 30 fps
Uncompressed 704×576 @ up to 25 fps
Uncompressed 720×480 @ up to 25 fps

Maybe another SequenceGrabber savvy engineer out there with the new MBP could experiment and see what you get back from VDGetCompressionTypes. Can we request MJPEG data through the pipes?

*Another thanks to Dave for running some tests for me on his MBP. Dave is a web designer and has a website that I’ll plug.