Bad week

It's been a bad week. My main hard drive crashed last Monday or so, while today my scooter is gone, presumed stolen. On the bright side, the final component of my new PC arrived and I was able to get Windows 7 up and running.

The new machine has an i7 920, so things like %NUMBER_OF_PROCESSORS% returns 8 - it's a quad core with 2x hyperthreading. Having so many logical cores has some downsides: maxed-out single-threaded processes only take up 12.5% CPU, so it can look like the machine is not particularly busy even though a process is working as hard as it can. Other specs include 12GB RAM, AMD 5870 graphics, 80GB Intel SSD and 1TB Samsung F3.

Maximizing the value of the small boot drive has been interesting. Windows 7 all by itself takes up a good 18G of space, what with the side-by-side (SxS) feature keeping lots of versions of the same APIs, and a big hiberfil.sys in the root - I moved the page file to the big drive. Because of the acute space constraints, I've been strongly motivated to keep almost all the junk that apps usually dump into the file system off the boot drive. By and large, I achieve that by creating directories on the 1TB drive and pointing to them with junctions on the boot drive. That way, apps think they are installing onto the C drive but they don't steal too much space. My prior drive - the one that died - was a 10K WD Raptor, so I've had some experience with this, and I wrote a script to automate the process:

#!/bin/bash

. _die

function usage
{
    echo "usage: $(basename $0) <directory>..."
    echo "Replace directories with junctions pointing to /bulk, after moving contents."
    exit 1
}

test -n "$1" || usage

for f in "$@"; do
    
    test -d "$f" || die "'$f' isn't a directory"
    
    dest="$(abs-path "$f")"
    dest=/bulk/mathom/"${dest//\//_}"
    
    test -d "$dest" && die "'$dest' already exists"
    
    # Try to copy stuff over; if an error, delete target and give up.
    cp -r "$f" "$dest" || {
        rm -rf "$dest"
        die "couldn't copy '$f' to '$dest'"
    }
    
    # Rename source, in case something can't be deleted.
    aside="$(move-aside "$f")" || die "failed in move-aside"
    
    # Create junction.
    junction "$dest" "$f" || die "failed to create junction"
    
    # Remove the aside.
    rm -rf "$aside" || {
        echo "warning: couldn't delete '$aside'"
    }
done

This script uses a number of little wrapper utilities I wrote - at this point I really ought to be thinking of putting them in an online repository. Here's a summary so you can follow along:

• abs-path: Canonicalizes and makes an absolute path by using cygpath twice, once to Windows format and once again back to Unix format, and prepending $PWD if necessary.
• _die: Defines a die function which prints out appropriate error and exits.
• move-aside: Renames a file or directory so that a failed attempt at creating a junction can be rolled back. Also helps with locked files - generally these can be copied and renamed, but not deleted. The new name is printed out on standard output.
• junction: This wraps SysInternals junction to create a junction in the TARGET LINK_NAME argument order familiar from ln, and permits using Cygwin paths.

With this script, and the big drive mounted at /bulk with a directory called mathom in its root, any given directory tree can be moved across readily enough. However, things in Windows 7 x64 are slightly more complicated.

Some of the directories I moved over to /bulk/mathom included Program Files, Program Files (x86), and ProgramData. Windows 7 itself has some junctions set up (the rats nest here is a proper horrorshow, I should document it at some point just for my own use, if nothing else). For example, "ProgramData\Application Data" points at ProgramData itself, and a simple cmd /c dir will fall into the infinite recursion and print out errors when the file path gets too long. So, in moving these directories across, I had to examine them for existing junctions and replace these as necessary. The Cygwin tools of Unix heritage are more robust against cycles in the file system hierarchy - for example, find keeps track of inodes to avoid infinite recursion.

All this checking of existing junctions wasn't sufficient to get everything working nicely, though. The permissions generally also need tweaking, particularly for folders that get modified by installers. These generally perform operations using either the NT SERVICE\TrustedInstaller or NT AUTHORITY\SYSTEM accounts - and sometimes both, in a dance of setup.exe and msiexec.exe. So, by process of checking, trial and error, using SysInternals Process Monitor to track installation failures, I put together a new script to ensure Program Files, ProgramData and folders of their ilk have the right set of permissions:

#!/bin/bash

function usage
{
    echo "usage: $(basename $0) <file/directory>..."
    echo "Resets files and directories to Windows 7 general install permissions."
    echo "Essentialy this is: SYSTEM/TrustedInstaller/Admin: full; AuthUsers: Modify; Users: Read"
    exit 1
}

test -e "$1" || usage

for arg; do
    
    # First, take ownership so we can definitely set all ACLs.
    # /F <file>; /R - recurse; /D <Y/N> - take ownership if no directory listing
    takeown /F "$(cygpath -w "$arg")" /R /D Y > /dev/null
    
    # Reset ACLs recursively to inherit from parent
    icacls "$(cygpath -w "$arg")" /reset /t /c /q > /dev/null
    
    # Set appropriate ACL on root, to progate recursively
    icacls "$(cygpath -w "$arg")" \
        /grant "Administrators:(OI)(CI)F" \
        /grant "SYSTEM:(OI)(CI)F" \
        /grant 'NT SERVICE\TrustedInstaller:(OI)(CI)F' \
        /grant 'Authenticated Users:(OI)(CI)M' \
        /grant 'Users:(OI)(CI)RX' \
        /inheritance:r /c /q > /dev/null
    
done

The logic is applied in three phases:

1. Take ownership, with takeown
2. Reset permissions recursively so that they inherit from parent
3. Set ACL on the root, so children will inherit from it, but make sure the root itself doesn't inherit permissions.

I tried for a while to get it all into one icacls invocation, but I couldn't get the two effects working together: enabling inheritance in the children, but disabling it in the root. This is something you can do in one "Apply" step using the Security tab in Explorer, though it's quite tedious picking out all the users and setting the appropriate checkboxes.

As an aside, the Security tab is quite confused by junctions; when permissions have been inherited from a directory on a different drive, because the current file / directory is in a subtree pointed to by a junction, the Security tab will search in vain through the parents looking for what provided the given inherited permission. Sometimes it gets lucky, and finds a random parent that coincidentally has the same permission; otherwise, it bails out with "Parent Object" as the indicated "Inherited From" column. But my proper list of Windows 7 issues will have to wait until I really feel the need to rant, which will no doubt happen sooner or later.

I should add that all the drastic file system surgery I write about here was performed with UAC effectively out of the picture, and all Cygwin tools running with the administrative privilege bits set. It's been my experience that command-line apps don't pop up the required UAC prompt; instead, they fail with an access denied message, which is rather vexing when you're trying to automate things.