I am embarrassingly far along in a bash backup script and have missed something major. I know that ext3 filesystems don't track creation date, but copying files within ext3 updates the changed time as seen via stat (which would work). However, when files are copied from one Windows server to another (whose mounts I have via smb/cifs), from my testing, changed time is not updated. I'm using find to do the searching on the cifs shares. Is there really no way to detect when a new file is created over a cifs share from Linux using 'find'?
The RSYNC job FROM the QNAP has been setup and tested to use both root/wheel and the domain admin accounts for access to the CIFs, with the same result - The FreeNAS server has had similar where I've changed permissions/ownership to be the same, with the same result.
Also, I am very familiar with rsync, and in this circumstance rsync's limitations rule it out as an option. I was thinking I could use rsync for the searching and try to pipe the results to the action (gzip), but I think the subshells would be ridiculous. Could be wrong, of course. Any suggestions would be much appreciated. Can provide more detail, but from my research I don't think it's possible.
The Linux kernel has had an xstat(2)
system call in the works for several years, now. David Howells of RedHat did much of the work. xstat(2)
allows one to retrieve the creation time (sometimes rechristened the 'born time' or 'birth time' in the Linux and BSD worlds for no really good reason) of files from the several filesystems that support it, including EXT4 (which has a creation timestamp on disc) and CIFS (which with its DOS/OS/2/Windows heritage has supported a creation timestamp as a first class citizen for decades). M. Howells worked on the CIFS patches that go along with the system call.
OpenSolaris and the BSDs actually do have a st_birthtim
datum in their stat(2)
system call right now, and the feature is accessible to script writers via application programs such as find
and ls
. On the OpenSolaris ls
man page you'll find crtime
alongside atime
, mtime
, and ctime
in several places. Similarly, the FreeBSD find
command has -Bmin
, -Bnewer
, and -Btime
primaries. And the Mac OS ls
has a -U
option.
If you were writing your script for OpenSolaris, the BSDs, or Mac OS 10, you could just get on with whatever you want to do with creation times right now. Indeed if you were writing for Windows you could do the same. Cygwin has supported st_birthtim
since 2007, making Win32's CreationTime
timestamp, which it has had since the first version of Windows NT and which Windows NT maintains on both NTFS and FAT volumes, available to Cygwin tools.
However, the same isn't true in the GNU Linux world. Creation time capability has yet to make it to GNU coreutils' ls
or to GNU findutils' find
. Indeed, it isn't even part of the mainstream Linux kernel yet. Part of the problem is that the xstat(2)
system call got bogged down in a diversion where people wanted to keep three timestamps, instead of have four as in the Windows NT kernel API, and dump ctime
to replace it with crtime
.
Linus Torvalds' response in 2010 was that 'it's all totally useless and people can't evenagree on a name' and 'Let's wait five years'.
In fact, as I suspect most people reading this will know, the world has been widely agreeing the name 'creation time' since the 1980s and we've been waiting at least 25 years already. It's the name used in OS/2 1.0; it's the name used in VMS ODS-1; it's the name used in Windows NT 3.5; it's the name used in SMB; and it's the name used in the question. ☺
I cannot really answer this question, but at least I can give you a hint: the book Implementing CIFS: The Common Internet File System by Christopher R. Hertel has these two very interesting pages:
page 1 and page 2
They seem to indicate that it is possible to retrieve the info you are looking for.
MariusMatutiaeMariusMatutiaeI recently installed a Synology DiskStation on my network. I mounted it from an Ubuntu 12.04.1 computer with the Browse Network button in Nautilus 3.4.2. It shows up as afp://[email protected]/photo/ in Nautilus.
So far, so good. I then uploaded a lot of photos to it, all with modifications times covering several months. When I looked at the directory of photos on the DiskStation, they all had modification times for the moment they were copied, not the modification times on the source computer. So much for sorting them by date on the DiskStation.
Is there a way to re-copy the files but have their modification date be preserved? Perhaps I mounted the DiskStation the wrong way. Perhaps Nautilus was the wrong tool to use. Any suggestions?
BTW, I have moved gigabytes of photos to a different NAS (Plextor PX-EH) over SMB/CIFS from Ubuntu 10.04, 10.10, 11.04, and 11.10 with modification times fully preserved. The problem must be with the Synology or some Ubuntu 12.04 software.
Jorge CastroI believe I have solved the problem. In Ubuntu 12.04, in Nautilus there are two ways to connect to the remote DiskStation NAS. One preserves modification times, one does not.
In the menu on the left-hand side of a Nautilus window, the Browse Network... button eventually leads to an AFP (Apple Filing Protocol) connection to the DiskStation, through which neither Nautilus nor cp -p
copies preserve modification time. I tried disabling Apple support in the DiskStation, but in that mode the DiskStation wasn't even visible in Browse Network.
In Nautilus's File menu there is a Connect to Server... option that offers a host of protocols. I chose Windows, entered my credentials, and connected without trouble. In this mode, modification times are preserved, so I was able to re-copy my photos and have their dates preserved.
Thank you Sergey and david6 for your suggestions. Hopefully people will find this information valuable.
Randall CookRandall CookStandard cp
command has --preserve
flag which preserved certain attributes (by default - mode,ownership,timestamps) when copying.
So something like this:
should do the trick in the 'normal' case. However, the afp://
thing in the URL confuses me - is it Apple Filing Protocol? All bets are off in this case.
One think I'd like to add - relying on file modification dates for cataloging your photos is very fragile. This is what image metadata (EXIF etc.) is for. Or, at least, just put them in directories according to their shooting date: photos/2012/12/05 etc.
SergeySergeyThis is the classic push/pull problem, for remote copy.
The recipient host is not honouring the date-stamp of the received files. Nautilus has this same fault, from 10.04 LTS through 12.10 ..
This is solved (for Nautilus), when copying between two Ubuntu hosts, by always copying from the remote-host (source) to the local-host (recipient). (AKA 'PULL')
Your problem is with the NAS box, and not with Ubuntu.
You need it to honour the date-stamp of received files (by default).
Are you using NFS (Linux) or CIFS (Windows) for file sharing?
david6david6Turns out that preserving timestamps for files and directories is still a problem in 2019! I was copying files from an Ubuntu 16 machine to an Ubuntu 18 one over SFTP, using Nautilus on the Ubuntu 18, and all files had the current timestamp, but directories had the original timestamps. Other tools failed as well:
What did work was to mount the remote filesystem using sshfs:
Copying from the mounted path also enabled Midnight Commander to preserve the timestamps (but didn't help BeyondCompare).
Dan DascalescuDan Dascalescu