Kent Spillner and I were trying to set up an Ubuntu Live CD with a full rails stack running our current project. We were seeing weird issues with nginx, such as connections being unexpectedly terminated and file uploads not working.
We asked around and Chris Read pointed out that Live CDs use the UnionFS filesystem which does not support sendfile.
From the Versatility and Unix Semantics in Namespace Unification paper:
”…the sendfile system call requires a matching file structure and address space structure. As Unionfs presently has no address space structure, we cannot properly implement sendfile, which is required for loop device mounts and improves performance for Web and NFS servers.”
Once we turned off sendfile (http://wiki.codemongers.com/NginxHttpCoreModule#sendfile), everything worked perfectly.