Here's a brief command-by-command guide to building Android 4 (ICS -Ice Cream Sandwich (with extra sugar on top)) from scratch, and deploying the new images on the Samsung Galaxy Nexus, all from Fedora 16. This is heavily based on the Free your Android article, and of course the instructions at android.com.

First, install the supporting packages. (Instead of the OpenJDK version, you might have to download the Oracle one, if you get version conflict errors at the make step below.)

yum groupinstall "Development Tools"
yum install java-1.6.0-openjdk kernel-devel git gnupg flex bison gperf zip curl zlib-devel glibc-devel glibc-devel.i686 ncurses-devel.i686 glib-devel.i686 libstdc++.i686 zlib-devel.i686 ncurses-devel.i686 libX11-devel.i686 libXrender.i686 libXrandr.i686 mesa-libGL-devel.i686 readline-devel.i686 arm-gp2x-linux-gcc-c++ python-markdown xmlto libxslt

Download the source. The final sync command will take about an hour.

curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
chmod a+x ~/bin/repo

mkdir android
cd android

repo init -u https://android.googlesource.com/platform/manifest
repo sync

Download proprietary binaries and drivers. They are available here:

http://code.google.com/android/nexus/drivers.html

Assuming the Galaxy Nexus - GSM/HSPA+ ("maguro"), there's two drivers. (Please check the link above for new versions).

wget https://dl.google.com/dl/android/aosp/imgtec-maguro-iml74k-a796ffae.tgz
wget https://dl.google.com/dl/android/aosp/samsung-maguro-iml74k-de1cc439.tgz
tar zvxf imgtec-maguro-iml74k-a796ffae.tgz
tar zvxf samsung-maguro-iml74k-de1cc439.tgz
extract-imgtec-maguro.sh
extract-samsung-maguro.sh

Build, still assuming the same phone as above. On my somewhat dated dual core 2.6 GHz CPU, it took almost four hours to compile.

source build/envsetup.sh
lunch full_maguro-eng
make -j4

Then, transfer the image files to the phone. Make sure the phone is connected over USB, is unlocked, and has USB debugging enabled. After the images are transferred, and the userdata and cache partitions are erased, the phone will reboot. It will show the Android logo, reboot one ore two times more, and then wait maybe a minute or two before the UI is available. And there it is, your home-built Android OS.

out/host/linux-x86/bin/adb reboot bootloader
sudo out/host/linux-x86/bin/fastboot -w -p maguro flashall

If something, or everything, failed and you are left with a useless phone, here are the factory images from Google. Download and unpack the archive corresponding to your phone, and run the script

flash-all.sh

For more details on "unbricking" your phone, see Derek Ross' comment.