NTFS Find and NTFS Backup
When using a Windows system with Interix or as NFS server,
it can be difficult to backup all the files on a NTFS volume.
This is because NTFS has many features not widely known in the
Windows community and with little to no support in the Windows GUI.
These features are used, however, e.g. by the Interix subsystem or the
NFS server. In particular the interesting NTFS features are:
- Unicode filenames with chars not convertable to ANSI
- NTFS hardlinks
- case sensitive filenames
- reparse points
- symbolic links
- alternate data streams
- pathes longer than 260 characters
- compressed files
- sparse files
- and possibly others
Most backup programs unfortunately can't handle several of these NTFS
features, so I decided to start my own copy program written in Perl,
which is meant to be used as a backup solution for NTFS volumes. The
idea is to simply copy all files to another NTFS volume (e.g. external
HDD drive) while preserving all of the NTFS features. To save space, it
should be possible to clone an existing backup on the external HDD by
creating hard links for each unchanged file
to the existing backup. Several backups
can be stored in a space efficient manner, and restoring doesn't require
a special software.
The backup program comes as a Perl module called Win32::NtfsBackup.
As a side effect I wrote a Perl module called Win32::Find to
traverse a NTFS directory tree, similar to Perl's File::Find,
but with support for NTFS features.
A similar idea for backing up files has been followed by
but unfortunately rsync on Cygwin can't handle case sensitive files,
alternate data streams and ACLs.
There is also
Robocopy but that unfortunately can't handle
hard links. Also it can't be case sensitive.
My NTFS backup script is written in Perl. You need ActivePerl and the
Win32::API package to use it. With ActivePerl 5.10.1 Win32::API is
- 2008-08-20: alpha release 0.03, support for creating dirs case sensitive
- 2008-06-21: alpha release 0.02, support for copying hardlinks
- 2008-06-09: initial pre-alpha release 0.01, which supports
Unicode long file names, case sensitive file names and streams.
- Think about API to get it more general purpose useable
- Implement other NTFS features
technical solved problems
- Pathes longer than 260 characters: use \\?\ prefix
and Unicode filenames.
- Case sensitive file listings: use FindFirstFileEx()
with the FIND_FIRST_EX_CASE_SENSITIVE flag.
- Case sensitive file creation or opening or deletion: use
CreateFile() with the FILE_FLAG_POSIX_SEMANTICS
- Creating hard links case sensitive. CreateHardLink()
has no flags for this and cannot be used. But opening the existing
file in case sensitive mode and then BackupWrite() with a BACKUP_LINK
structure works. This was the only official way on NT4 to create hardlinks.
- Creating new directories case sensitive. CreateDirectory()
has no flags to switch. CreateFile() with
FILE_FLAG_POSIX_SEMANTICS | FILE_FLAG_BACKUP_SEMANTICS | FILE_ATTRIBUTE_DIRECTORY
can be used, however.
- Unlinking a file with the Read-Only attribute set isn't easily possible,
Windows Explorer removes the attribute before unlinking. Unfortunately,
the remaining hardlinks lose the attribute without notice.
Unlinking works like this: opening a file, then look for the Read-Only attribute
unset it if set with NtSetInformationFile(),
then set FILE_FLAG_DELETE_ON_CLOSE via NtSetInformationFile(),
and finally reset FILE_ATTRIBUTE_READONLY before closing the file.
(not yet implemented)
Last update of this document: Friday, 23-Apr-2010 23:46:15 CEST
Copyright 2008-2010 Martin Köppe <mkoeppe 'at' gmx . de>