In the spirit of the contest, we procrastinated until late Thursday afternoon and hatched a plan in a Starbucks-fueled brainstorm. (The employees at Starbucks are thoroughly convinced that we have no job.)

We had some Core Location-related ideas, but they would have involved lots of running around Chicago with iPhones, and since Wolf had given explicit permission to deviate from the theme “if the hack was cool enough”, we decided not to go the Core Location route. Once we came up with the stupidest idea on the planet, we headed out to buy Radio Shack’s last four remaining infrared LEDs and some tiny calculator batteries.

Six hours later, we had our hack ready: Using the WiiRemote.framework and some Apple sample code, we created a Mac app which could track infrared light sources using a Bluetooth link to a Wii Remote to draw colorful lines on the Mac screen. We also added an HTTP server using Cultured Code’s Simple HTTP Server Cocoa project. We then created an iPhone application to send HTTP requests to the Mac via Wifi. These requests controlled what kind of “artwork” would be generated by the Mac application. (Line color, thickness, etc.) As a finishing touch, we taped the infrared LEDs and batteries to the front of the iPhone. This allowed for a “digital whiteboard” effect. By pressing button on the iPhone’s cool UI and waving it around in a drawing motion, we were able to paint designs in the Mac program.

Friday was something of a blur and no work happened on the hack. I think I signed up for Twitter at some point. Tweet.

Saturday afternoon, we decided that we needed a “clincher” feature. A new mode was added to the application (using a UITabBarController of course) which allowed for the positioning and stamping of image files. We also added iPhone X-axis accelerometer data to the mix to allow for rotating of the stamped images by simply twisting the iPhone.

The next 24 hours was spent trying to decide what images we would use. We finally decided on Ken and Glen heads, tinfoil hats, nose glasses and fake moustaches, inspired by Josh and Dave’s Mr. Shuffle app, and the “paranoia” theme.

After spending Sunday morning in Photoshop lassoing fake moustaches, the live demo was a complete disaster. We positioned the Wii remote too far away from the Mac for the Bluetooth link to function, and our iPhones randomly jumped from out of our ad-hoc network to the conference network midway through the demo. Much awkwardness was twittered.

In the end, we somehow managed to take first place. Thanks for your votes, and thanks to all of the companies who donated software and t-shirts!

Here’s the source (875k) for our hack including Mac and iPhone projects. You’ll need Xcode 3.1, iPhone SDK and the WiiRemote.framework if you would like to build it. You’ll also need infrared LEDs… and tape…

Known issues: HTTP did not prove to be the best way to send rapid-fire requests from the iPhone to the Mac. The iPhone’s HTTP request system seems to freeze up if we send requests too quickly. Also, the IP address of the Mac server is hardcoded into the iPhone app for extra credit laziness.

One more thing: This code was mashed together in a few hours, and just barely. It contains some nasty bits and should not be used as an example of proper or efficient Cocoa programming!

So will anyone actually try this themselves? Let us know if you do!

Actual Program Output

Before I was even finished coding this feature, I already knew I couldn’t live without it. Glen used more explicit terms; something leading to him and the new feature having babies.

In short, you can see unread counts for any mailbox or folder up in the status area, and a simple click on the indicator pops open Mail.app and brings up the right mailbox. This was a requested feature from DockStar fans. (We get a lot of our best ideas from customers.) It’s not limited to unread counts either: You can set each indicator to count flagged messages, total message counts, and even monitor Smart Mailboxes.

This upgrade brings many new features for Leopard users. If you use To Do items in Mail.app or Calendar.app, you can use DockStar to monitor the number of incomplete To Dos. You can also count Notes or even keep track of unread RSS feed items.

The DockStar Dashboard Widget also got some improvements. You can now click on the widget to jump right into the relevant mailbox.

If you’re a serious emailer, and you’ve never tried DockStar, you should try the free trial, but be warned: Afterwards, you won’t be able to live without it.

If you take a look at my previous post, you’ll see a list of new restrictions which Leopard puts on Input Manager plugins. Number 3 is: Processes running with the root privilege (getuid() == 0 or geteuid() == 0) cannot load any bundle input manager. While I do consider this overkill (given #2), it seemed a very straightforward requirement and sensible if you’re trying to be as safe as possible.

A handful of users weren’t getting our plugins to load, so we created a test application to run down all of the requirements one by one and see which one was failing. To our surprise, we found the test application (and presumably all of their apps) was running geteuid() == 0! Yes, all running as super user, root, the big 0. A quick check in Activity Monitor revealed that sure enough, most every process on the machines in question was running as root. Now remember, they’re not logged in as root. They’re logged in as their normal user; let’s call him Steve. Whenever Steve opens a program from the Dock, it runs with root privileges instead of Steve privileges. This is nasty. Steve doesn’t even know it, but I’m sure his system is acting rather strange. Most people with this problem reported “other stability issues” and eventually did a clean install of Leopard (which fixed the problem.)

I asked around on the Apple-cdsa listserv to see if anyone had seen this problem. I quickly heard from another developer who had already seen the same issue, but nobody else on the list had seen it and nobody had a quick solution.

I went back on a few weeks later asking for any help with troubleshooting the problem. This was fruitful. Although it wasn’t entirely on topic, heads were scratched, hallway discussions were had, and I heard from an Apple engineer who quickly guessed at the actual cause of this.

Apparently, some major changes have been made to how Leopard launches applications.

If you’ve spent the last several years seeing Tiger crash reports, you’ve probably become quite familiar with seeing:

Parent: WindowServer [123]

On Tiger, the very first process (PID 1) is launchd, which launches all daemons, including WindowServer, which is in turn the parent of all of your “apps”. launchd runs as root, WindowServer runs as windowserver, and your apps should all be running as, well, as you.

On Leopard, you might have noticed that the parent process in crash reports has now become launchd. When you login on Leopard, the PID 1 launchd forks a second launchd, which is the parent of all your apps. This second launchd runs as your normal user.

In our messed up case, this second launchd was running as root.

On these affected machines /sbin/launchd had its setuid bit set. The setuid attribute is a permissions bit flag, which causes a program to run as root even if a non-privileged user executes it. Sound like bad idea jeans to you too? Set this bit on launchd, and that’s exactly what happens: It always runs as root no matter who starts it.

launchd is normally setuid on Tiger, but it’s not supposed to be on Leopard. I’m assuming this is some kind of 10.4 -> 10.5 migration issue.

It’s not hard to turn off the setuid bit and fix this problem, but the user will have lingering ownership issues. Every file they’ve created since upgrading to Leopard is going to be owned by root and unwritable by their normal user. This isn’t hard to fix but who knows what other weird issues they’ll see. Yuck. Clean install highly recommended.

There are obvious reasons why having all your apps running with root privileges is bad, especially if your machine is being used by non-admin users.

It’s interesting that a single bit flag can give you super user privileges on your own machine. If at least half dozen of my customers had this problem, then there must be thousands of users out there running as root. At least one person at Apple is now aware of the issue, so hopefully there will eventually be a fix for this.

What exactly caused this bit to be set for these unfortunate users? This is the one question we still don’t have an answer to.

The fun new functionality is all there after updating to 7.2, but there are no new header files (no public API yet). Also, the functionality is now used by QuickTime Player Pro. However, you probably won’t notice much of a change except a little progress indicator in the Recording section of the prefs when it’s scanning for cameras, and hopefully a performance improvement.

Since it’s in the wild, I can now talk freely about this cool new feature of QuickTime. All of this I learned simply by hunting around and looking at sample traces of a running video stream in the new version of QuickTime Player.

QTKit Capture looks to be a total ground-up replacement of the Sequence Grabber. The Sequence Grabber is a very ancient part of QuickTime, designed long ago as the way to capture streaming audio/video. It doesn’t appear to be going away, but QTKit Capture doesn’t use it internally except for reverse compatibility* with old vdig drivers. Where the Sequence Grabber uses QuickDraw, QTKit Capture makes heavy use of Core Video and OpenGL. Also, the top layer is Objective-C so it can be integrated easily into a Cocoa app.

I’ve done some looking around and found some interesting things:

It makes use of some new private frameworks. New to the club are:
CoreMediaAuthoringPrivate.framework
CoreMediaIOServicesPrivate.framework
CoreMediaPrivate.framework

CoreMediaIOServicesPrivate’s framework Resources folder contains a number of plugin modules. There’s one for AVC (DV cams), IIDC (external iSight), and VDC (the built-in iSight). In a nutshell, CoreMedia appears to implement a Device Abstraction Layer (A way to allow access to all video devices without having to know the details of the device.)

Other notes:

• * QTKit Capture still works with existing QuickTime component vdig modules (macam still works). In this case, it is actually running the Sequence Grabber below CoreMedia. For the built-in iSight however, the Sequence Grabber is not being used. Hopefully, Apple will provide a new way to write plugins for third party devices so the SequenceGrabber doesn’t have to be used. However, they do support 3 major video camera standards (IIDC, AVC, and UVC), so device developers going forward would be wise to implement one of these standards, eliminating the need for writing their own driver.
• You can build and link against the new functionality if you have the proper header files.

So let’s say you want to find out the current CPU speed. It’s common to use the ancient but useful sysctl call.

int args[] = {CTL_HW, HW_CPU_FREQ};
int freq = 0;
size_t len = sizeof(freq);
if (sysctl(args, 2, &freq, &len, NULL, 0) == 0)
     printf("It's %d Hz\n", freq);


A Gold Star if you can see what’s wrong with that code.

Wait for it…

Since sysctl returns the clock speed in tiny little old Hz (1 billionth of a GHz), we’re quickly running out of space in our 32-bit int. We’d overflow an unsigned int when we hit 4,294,967,296 Hz and we’ll overflow the signed int we used in the code above at only 2,147,483,648 Hz.

But that’s only 2.14 Ghz.

So on my 2.33 Ghz MacBook Pro, the code above will return:
It’s -1,964,967,296 Hz.
Oops…

Obviously, using an unsigned int instead of a normal int would have been wiser. However, even that will overflow when clock speeds double (in 2 years if you trust Moore’s law.)

Indeed, the solution is to use a long long or int64 (whichever camp you’re from.)

Here’s better code, using sysctlbyname instead, which is the recommended way to do it nowadays:

uint64_t freq = 0;
size_t size = sizeof( freq );
if (sysctlbyname( "hw.cpufrequency_max", &freq, &size, NULL, 0 ) == 0)
     printf("It's %lld Hz\n", freq);


This introduces a new overflow point when CPU speeds exceed 18,446,744,073 Ghz.

Interestingly, if you believe that Moore’s law will continue to hold, we should reach these speeds around 2071. That is, of course, assuming that the human race survives the Year 2038 Problem which will cause a similar overflow issue with the way OS X keeps track of the date seconds in an unsigned int.

Switching the date seconds to an int64 will cause a new problem in about 290 billion years.
However, due to predicted changes to the Solar System, it is not widely regarded as a pressing issue.

Also, don’t forget about good old OS 9’s Gestalt routine. You can’t use gestaltProcClkSpeed anymore for the same reason. (It returns a long.) But Gestalt also has a selector gestaltProcClkSpeedMHz, which drops the six useless zeros.

Today we cleaned it up and are posting it here so new-MacBook-Pro-having visitors can try it out see the new and improved image for themselves.

The app requests and displays a video stream from the camera at 1280×1024. If you’re on an older iSight it will still stretch the VGA image out to be this size but it won’t look very good. The app will report the frame actual size from the ImageDescription at the top.

There’s also a snapshot button so you can take your own picture and make your friends jealous of your fancy new laptop. Sorry: no Photo Booth effects yet!

If you get to try it, leave a comment and let me know how it works for you.
The Test App (68 k)

Tech Talk Follows:

The app does Hi-Lite™ a potential reason why Apple might not be yelling from the rooftops about the new camera: You can see right away how slow it is to stream the full sized video. Why? It’s more than four times more data than VGA. The Sequence Grabber was designed decades ago and wasn’t designed to handle more than NTSC/PAL sizes. Until the Sequence Grabber is revamped, it’s a little too slow to run such a large stream.

1,280 x 1,024 = 1,310,720 pixels
It’s 4:2:2 compressed YUV data, so that’s 2 bytes per pixel or 2,621,440, or 2.5 megabytes per raw frame.
So for full 30 fps video, this translates to about 75 megabytes per second or 600 Mbps.
Now keep in mind the camera controller allows for a compressed MJPEG mode which cuts the required USB bandwidth signifcantly. (You’d never fit that over USB 2.0 without compression.)

But even if it is being compressed on-camera, it’s then being decompressed in the driver and passed to QuickTime in YUV. That’s a lot of data to push through the poor Sequence Grabber pipes. Technically, you should be able to request MJPEG data from the sequence grabber, but the default seems to be YUV. I haven’t tried requesting other compression types as I don’t actually have one of these new machines in front of me.*

More details about the new camera:

The original iSight didn’t advertise itself as a UVC device (It was Vendor Specific).
The new one does advertise as UVC, so you can easily use USB Prober to decode the device descriptor:
According to the descriptor the camera supports the following frame formats:

MJPEG 640×480 @ up to 60fps
MJPEG 720×480 @ up to 60fps
MJPEG 800×600 @ up to 30fps
MJPEG 1024×576 @ up to 30fps
MJPEG 1024×768 @ up to 30fps
MJPEG 1280×960 @ up to 30fps
MJPEG 1280×1024 @ up to 30fps
Uncompressed 640×480 @ up to 30 fps
Uncompressed 352×288 @ up to 30 fps
Uncompressed 160×120 @ up to 30 fps
Uncompressed 704×576 @ up to 25 fps
Uncompressed 720×480 @ up to 25 fps

Maybe another SequenceGrabber savvy engineer out there with the new MBP could experiment and see what you get back from VDGetCompressionTypes. Can we request MJPEG data through the pipes?

*Another thanks to Dave for running some tests for me on his MBP. Dave is a web designer and has a website that I’ll plug.

or… Built-in iSight The Next Generation 

At WWDC, Glen had the unenviable task of hitting the compatibility lab and testing the upcoming release of iGlasses on as many computers as he could.  When checking out the new 2.2 and 2.4 GHz MacBook Pros, he noticed something interesting about the built-in iSight:

It’s a whole new camera.  Really!

First, looking at System Profiler:

It has a new USB PID.  What I’ll henceforth have to refer to as the original built-in iSight, had the USB Product ID: 0×8501 and reports its manufacturer as Micron. The new built-in iSight has the Product ID of 0×8502 and reports manufacturer of Apple Inc.   

Now the cool part:  It’s a 1.3 megapixel sensor!  I wrote a simple test application to capture a frame of video using an SGDataProc and save it to the Desktop.  On my older 2.33 GHz MB Pro with the original built-in iSight, what I get is a VGA (640×480) image.  This is the maximum size of the original camera, which I know for sure uses a VGA sensor.

Then, braving the 95 degree the heat, we got on our bikes and cruised down to the Cambridge Apple Store.  After fighting our way through the crowds of people asking if they can pre-order an iPhone, we commandeered some MB Pros.  The same test program produced a nice, big 1.3 MP (1280×1024) image. 

UPDATE: Test app available here.

Other differences:

The new camera supports all of the VDSetBrightness, VDSetContrast, etc. calls. The old camera only supported Saturation and Sharpness.  This causes an odd bug in iGlasses which we’ll have to fix very soon.

Whereas the original camera was uyvy (’2vuy’ in QuickTime), the new camera’s data is yuyv compressed. (that’s YUY2 for you Windows users or ‘yuvs’ in QuickTime parlance.) It’s the same as the previous 4:2:2 format except the 16-bit words are byte swapped. This doesn’t cause a problem for iGlasses which knows how to deal with both data types as this is the YUV format used by our iMage USB Webcam.

Interesting:

- I haven’t verified this yet, but I’m going to guess that 10.4.10 (which includes a new 2.0 version of the UVC driver) is required to get the larger sized image.  This would explain why nobody noticed this last month when they were first released.

- I’m guessing they’re still using a Micron sensor.  A few sources have indicated that the iPhone will also use a Micron sensor for its camera.  The phone is supposed to have a 2 MP camera. 

- iChat and other iApps don’t take advantage of the larger image size.  If we stretched our iChat preview window up to around 1280 wide (the aspect ratio is not 4:3 in iChat 3), it was clear that we were looking at a VGA image scaled up.  iChat and other such apps are probably calling SGSetChannelBounds with 320×240 or 640×480 because that’s the size they use for conferences, whereas I’m specifying a really large size (1600×1200).

- Thanks to Dave for letting me mess around with his new laptop this afternoon.