Can any API or linux tool follow any changes in any folder below eg. / SharedRoot or I need to customize eg. inotify for each folder?
I have a folder with ~ 10,000 subfolders.
Can any API or linux tool track any changes in any folder below eg. / SharedRoot or do I need to set up inotify for each folder? (i.e. I will lose if I want to do this for 10k + folders). I think so, since I have already seen examples of this ineffective method like http://twistedmatrix.com/trac/browser/trunk/twisted/internet/inotify.py?rev=28866#L345
My problem:
I need to keep folders sorted by time with the most recently active "project" at the top.
When a file is modified, each folder above that file must update the last modified timestamp to match the file. Delays are fine. By opening the file (usually MS Excel) and closing it again, the date of the file may slip and then down again. For this reason, I need to wait until the file is closed, and then stop the folder of this file to check, and only after a while I will go and look for the newest file in my folder, since the filedate from the startup file may already be reverted to the original temporary mark using Excel or similar programs. Also, in the case of using / creating multiple files from the same folder, it makes sense to block the temporary binding of the parents of these folders in order to at least get a bunch of updates collapsed into one deferred update.
I am looking for a Linux solution. I have some code that can be run on a windows server, most of the quering functions can be found here: http://github.com/sesam/FolderdateFollowsFiles/blob/master/FolderdateFollowsFiles/Follower.vb
Available APIs: s The
relative inotify on windows, ReadDirectoryChangesW, can watch a folder and its entire subtree; see bWatchSubtree at http://msdn.microsoft.com/en-us/library/aa365465(VS.85).aspx
Samba?
The original samba source is a possibility, but maybe there are hooks already? Other features like client side (different Windows versions) and file tracking to update folders recursively?
a source to share
Yes, you do need to use inotify, however you do not need to consume hours on every node right away.
The process (similar to how it's done) looks pretty simple:
- Set the clock to your root node.
- Make the width first (not the depth> ) starting at the root node
- Set the watch in the catalogs, in the order of the search
- Make sure the directory creates events, keep adding them. Re-sort your list if this happens.
Breadth-first search is essential, otherwise you might be missing some things due to race start and what the root node clients do.
See this question which also mentions this RFQ . I had the exact same problem you faced.
Essentially, one thread continues to watch for directory creation events, adding new clocks to new directories almost at the same time they are created. Something else is sorting the list on demand or after the inotify stream releases its lock.
I tried to block the versions above, but with .. dubious .. success :)
a source to share
I have seen you use these trees under shared Samba. Perhaps you can use the ClamAV virus scan VFS engine for inspiration to see how they run a "scan on close".
Samba Howto: Stackable VFS Modules
It should be quite simple to check the time of the closed file and change the directory path leading to it, without any performance / memory overhead associated with inotify et al.
Just a thought.
a source to share