Debian Interix technical issues

manually unpacking .deb archives
building .deb packages for Debian/Interix
the DEP issue
long double incompatibility
Hardlinks, Symlinks and Junctions on NTFS
Case sensitive file names on NTFS

Manually unpacking .deb archives

The binary packages provided here have standard Debian deb format. For those who want to manually unpack such a .deb package (NOT necessary when installing Debian for Interix, of course): .deb files are normal ar archives, to unpack do the following:

$ ar x any.deb data.tar.gz control.tar.gz
$ tar xzf data.tar.gz
$ tar xzf control.tar.gz

The data.tar.gz contains the files to install while the control.tar.gz contains all the meta information.

How to build .deb packages for Debian/Interix

To build your own packages for Debian/Interix follow these steps:

make sure you have the following lines in your /etc/apt/sources.list: (of course you should use an appropiate debian mirror for the first line) and after that update the package cache with
Install the following build tools:
Do some modifications: In /usr/share/cdbs/1/rules/buildcore.mk comment out the test for root, and to /usr/bin/dpatch apply this patch until it gets accepted in standard Debian to allow spaces in $HOME.
Set the modified arch:all packages on hold:
You'll probably need cp, ln, install from the not-yet-packaged coreutils you can currently only download separately, for the GNU cmd line options: http://www.debian-interix.net/debian-interix/bootstrap-tools/coreutils/bin.tgz
Now create a new scratch directory, say /srv/packages. For each package make a subdir. The easiest package to (re-)build seems to be athena-jot, so for the beginning:
Download the source package $ apt-get --download-only source athena-jot Be sure to use the source package name, not the binary package name, they may differ. You can find the source package name via http://packages.debian.org/. You now should receive 3 files:
```
athena-jot_9.0-5.diff.gz
athena-jot_9.0-5.dsc
athena-jot_9.0.orig.tar.gz
```
Now extract the source package
As athena-jot is a very simple package, you can build it for Interix without modifications: You should get a *.deb file now in .., which you can install via
If modifications are necessary, remove (or rename) the extracted source tree again, re-extract it, modify it, build a new source package, then build the extracted and modified source package. If the build goes wrong, you can re-extract the new source package, modify it further, repack it to save the changes and try again. So: add a new changelog entry, increasing the version number, to the front of the file, say:
Do other modifications and repack source: You get a new *.diff.gz and a new *.dsc which you can use later to re-extract your changes.
Build the package again as in 9. You don't need to make a new changelog entry every time you do further mods, only 1 time to have a new source package different from standard Debian.
If the package is ok now, then you could create the interdiff like this: and, if you want to contribute to Debian/Interix, send me the *.interdiff.gz as well as the build.log.

There are also Debian native packages which only have 2 files as source package: *.dsc and *.tar.gz. Only if you wonder about them. When modifications are necessary to these, say e.g. dpatch-2.0.27, then add "-0+interix.1" to the version string in changelog, and make a symlink *.orig.tar.gz pointing to the *.tar.gz before building the source package. This way the package gets a normal package, and the interix mods are all in the *.diff.gz.

For more details on Debian packages you may refer to the Debian Policy.

For building X packages you may need to change the symlink /usr/include/X11 from pointing to ../X11R6/include to pointing to ../X11R6/include/X11 at least on SFU 3.5 this is necessary for some ruby lib to build correctly.

The Data Execution Prevention (DEP) issue

Newer AMD and Intel CPUs have hardware support for marking parts of the memory as non-executable. The existence of the NX flag (no execute) on AMD CPUs or the XD flag (execute disable) on Intel CPUs indicates that this feature is available. More information about DEP can be found here.

Problem

On Interix there are 2 problems with DEP:

The dynamic loader, /usr/lib/ld.so, doesn't work when DEP is turned on.
GCC may produce code that doesn't work when DEP is active.

The DEP problem has only been seen on SFU 3.5 running on Windows XP or Server 2003 so far and shows as:

Memory fault (core dumped)

While I don't yet know the exact problem with ld.so, GCC compiled programs (original gcc-3.3 as well as Debian/Interix gcc-4.2) may work with DEP turned on if:

you link statically (to avoid the ld.so issue)
and you don't use nested functions (rarely used anyway)

I successfully run a simple hello world program as well as a statically linked bash. A sample program that uses nested functions and therefore crashes: #include <stdio.h> int main() { int values[] = { 3, 2, 4, 6, 7, 1, 8, 0 }; int num_comparisons = 0; int comparison_function(const void* a, const void* b) { int first = *(const int*)a; int second = *(const int*)b; num_comparisons++; /* look, we're accessing a variable from our parent function */ if (first < second) return -1; else return first > second; } qsort(values, 8, sizeof *values, comparison_function); printf( "%d %d %d %d %d %d %d %d, %d comparisons\n", values[0], values[1], values[2], values[3], values[4], values[5], values[6], values[7], num_comparisons ); return 0; }

More about nested functions with GCC can be found here and here.

Solution

If you need shared libraries (for Debian/Interix you do) or need GCC's nested functions, you can disable DEP completely, see also /Tools FAQ.

Alternatively, you can use the contribution from /Tools member franz (Franz Fischer), which disables DEP only for Interix processes, so that you don't lose the protection for Win32 processes. According to franz, the same thing is done on Interix 5.2 and up, and therefore the DEP issue is not found on these systems. I successfully verified franz's contribution on SFU 3.5 on Windows XP.

The `long double` incompatibility

Microsoft has decided to remove support for the 80-bit long double type from their compilers entirely. They now simply alias long double to double for source code compatibility. For MSVC is therefore sizeof(long double)==8. On the other hand, the GCC that comes with Interix has sizeof(long double)==12, the same as on Linux/i386. So GCC can handle 80-bit long double.

Problem

When one compiles a function taking a long double argument with one compiler, and the call to this function with another, this will result in wrong data being computed, and may even crash the application. This is because the say MSVC compiled function expects an 8-byte argument, but the GCC compiled call to the function generates a 12-byte argument. If other arguments follow, they get messed up. This problem especially can occur with the *printf style functions. The Interix libc is build with MSVC, so expects long double to be 8-byte in size. If you call sprintf("%Lg", (long double) 42) in GCC compiled code, it'll fail to print the expected string.

Problem 2

While GCC stores long double numbers as 12-byte in memory, it doesn't use by default the extended precision for calculations, because the FPU gets initialized by the interix libc / crt0.o to only double precision. See the following program: #include <stdio.h> #include <math.h> int main (void) { #ifdef __GNUC__ long long int i; #else _int64 i; #endif long double d; i = 0xdeadbeef; i = (i << 32) | i; printf ("given: i=0x%lx%lx\n", (long) (i >> 32), (unsigned long) (i & 0xffffffff)); printf ("now do i-=1 via long double\n"); d = i; d -= 1.0; i = d; printf ("expected: i=0xdeadbeefdeadbeee\n"); printf ("got: i=0x%lx%lx\n", (long) (i >> 32), (unsigned long) (i & 0xffffffff)); printf ("\nnow setting extended precision explicitely\n"); _control87 (_PC_64, _MCW_PC); i = 0xdeadbeef; i = (i << 32) | i; printf ("given: i=0x%lx%lx\n", (long) (i >> 32), (unsigned long) (i & 0xffffffff)); printf ("now do i-=1 via long double\n"); d = i; d -= 1.0; i = d; printf ("expected: i=0xdeadbeefdeadbeee\n"); printf ("got: i=0x%lx%lx\n", (long) (i >> 32), (unsigned long) (i & 0xffffffff)); } The output is:

given:    i=0xdeadbeefdeadbeef
now do i-=1 via long double
expected: i=0xdeadbeefdeadbeee
got:      i=0xdeadbeefdeadbe00

now setting extended precision explicitely
given:    i=0xdeadbeefdeadbeef
now do i-=1 via long double
expected: i=0xdeadbeefdeadbeee
got:      i=0xdeadbeefdeadbeee

Hardlinks, Symlinks and Junctions on NTFS

Hardlinks

Possible only within one file system
There is no explicit object for the link, i.e. after creating the link, it's not distinguishable anymore, if the link has been created from A to B or from B to A.
A file is deleted, when the last hardlink to the file has been deleted.
There never are dangling hardlinks.
Additional hardlinks can only be created for files, not for directories.

Symbolic links

There is a special link file which contains the target file name of the symlink. The target file name can live on any file system.
If the target file doesn't exist, the symbolic link can still exist, it's a dangling symlink in this case. The link target doesn't know anything about which symlinks point to it, and therefore symlinks cannot be updated or deleted automatically when the target is renamed or deleted.
Symlinks may refer to files or to directories.
The link target may be specified by an absolute path name or by a relative path name.
Symlinks are always resolved on the client side, even if they reside on a network drive. I.e. the same symlink may refer to different files when seen from different network clients. (Think of a symlink pointing to C:\WINDOWS.)
The client must support resolving symlinks, the server need not necessarily.

Junctions

Junctions are similar to symlinks, so (1) and (2) for symlinks are also true for junctions. Besides that:
Junctions may only refer to targets on local file systems, not on network file systems.
Junctions on Windows may only refer to directories, not to files.
Junction targets are specified with an absolute path. I haven't seen junctions pointing to relative targets, so I'm currently unsure, if junctions can have relative targets also.
Junctions are always resolved locally on the machine which has the file system containing the junction. All network clients see the same link target.
The server and client must support junctions, the client must be aware that the free space within a subdirectory may be different from the free space in the parent directory.

Windows	supported on	Unix equivalent	note	create with
hardlink	every NTFS version	hardlink	exact equivalent	CreateHardlink(), BackupWrite()
Interix symlink	every Windows FS	symlink	exact equivalent, but only available in the Interix subsystem or via NFS	symlink(2), wcs_symlink(2)
junction	Windows 2000 NTFS and up	mount --bind	not available in Interix or via NFS	Sysinternal's junction.exe
Windows symlink	Windows Vista NTFS and up	symlink	On Windows there are 2 different kinds of symlinks, one pointing to files, and another pointing to directories. Windows symlinks are not available in Interix or via NFS.	cmd.exe's mklink

Case sensitive file names on NTFS

NTFS is per se a case sensitive file system, i.e. the file names FILE, File and file can all exist in one directory and refer to different files. The Win32 subsystem is case preserving for file names, but case insensitive when opening existing files or directories. The Windows kernel which handles the requests from the Win32 subsystem, is case insensitive by default, too. It can be made case sensitive, however, if support for Interix or NFS server is required. To enable case sensitive file names in the kernel, while still having case insensitive file names in the Win32 subsystem, you need to set HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive = 0 in the registry. When you now have created some files with names differing only in case via Interix or NFS clients, you may want to copy or backup these from within the Win32 subsystem, which is always case insensitive. Fortunately there are a few Win32 API functions, which can operate in a case sensitive manner if the kernel has been switched to case sensitiveness and which can be used to enumerate, open and delete files case sensitive. These functions are:

FindFirstFileEx() with flag FIND_FIRST_EX_CASE_SENSITIVE
CreateFile() with flag FILE_FLAG_POSIX_SEMANTICS

To delete files or create directories or create hard links with case sensitive names, you need to use CreateFile() with FILE_SHARE_DELETE for deletion and BackupWrite() to create directories or hard links. Moving within the same file system is done via creating a hard link and deleting the original afterwards.

Debian Interix technical issues

Manually unpacking .deb archives

How to build .deb packages for Debian/Interix

The Data Execution Prevention (DEP) issue

Problem

Solution

The long double incompatibility

Problem

Problem 2

Hardlinks, Symlinks and Junctions on NTFS

Hardlinks

Symbolic links

Junctions

Case sensitive file names on NTFS

The `long double` incompatibility