Best language for managing large files in scripts
The Natural Sounds Service National Park Service collects several terabytes of data every day to measure soundscapes. In your opinion, what's the best scripting language available for managing a huge number of files and file types? We would like to easily create and run efficient user-friendly scripts to find and get / create copies of files that can be located in different directories under the same static hierarchy. The OS will most likely be Windows. Thank you!
a source to share
Use Python. Easy to learn. Everyone can convert easily.
File size doesn't really matter when looking up directories or looking for metadata outside of files. However, you rarely need to read the entire sample file to remove metadata.
Also, if you do this frequently, you may want to consider
-
Extract all metadata to the relational database.
-
Use a relational database as a complex "index" for sound sample files.
Every addition or change to a file will be done through the application, which synchronizes the file with database updates to ensure that the database index is indeed consistent with the file system.
The bulk of your queries can become SQL queries.
a source to share
I really don't know what you're going to look for in a scripting language, but Eric is right that you should use what all of your developers are familiar with. However, if you don't have developers (yet) and are developing a project (and team) from scratch, C ++ or .Net (C # or VB).
While C ++ offers more powerful programming and performance, C # and VB.Net offer faster production. Regardless of the performance benefits of .Net, I would think that for a huge number of files and file types, you would have the best overall C ++ satisfaction. In my opinion, the best user-friendly design requires very little user input other than pressing buttons or selecting options from a list.
a source to share