# Looking for a program to catalog files on external/offline storage?



## Bobi B. (Nov 24, 2018)

Greetings, All!

I'm looking for a program, a command-line will be nice, but not mandatory, to be able to scan external drives and to allow me to see what was on each drive during the last time filesystem were scanned, even after drive is detached and returned to the shelf. An ability to search by pattern would be nice (i.e. "on which drive/filesystem I put that other file?") Ability to extract and to being able to search against file metadata would be even better ("on which disk I put my pr0n^H^H^H^H Classical records on?") On Mac I use NeoFinder and I believe there are Windows Apps for that, as well.

Can anyone suggest such a program for FreeBSD or any Linux?

Thank you for your time!


----------



## tingo (Nov 24, 2018)

Hmm, it doesn't look like there are many "collection manager" type of software in ports, just misc/gcstar and multimedia/gcfilms. None of them appear to do what you ask, but you might be able to configure them to do it.

I would also like to point out that if you can have all of your files online (on a NAS for example), the need for a collection manager is less (you can just search your files).


----------



## ronaldlees (Nov 24, 2018)

Is the `tree` command too limited for your purposes?

tree man page


----------



## KBK (Nov 25, 2018)

Like disk cataloger I use recoll.
Personal full text search package, based on Xapian. Command line and GUI, FreeBSD Linux and Windows.

https://www.freshports.org/deskutils/recoll
https://www.lesbonscomptes.com/recoll/


----------



## Bobi B. (Nov 25, 2018)

tingo said:


> Hmm, it doesn't look like there are many "collection manager" type of software in ports, just misc/gcstar and multimedia/gcfilms. None of them appear to do what you ask, but you might be able to configure them to do it.
> 
> I would also like to point out that if you can have all of your files online (on a NAS for example), the need for a collection manager is less (you can just search your files).


Those seems to help you make a list of your physical music CDs and movie DVDs, but thanks, nevertheless.



ronaldlees said:


> Is the `tree` command too limited for your purposes?
> tree man page


(Just for the record, tree(1) is in the port sysutils/tree.) I was thinking of abusing updatedb(8) and locate(1), possibly wrapped in a shell script. Perhaps file(1) and mtree(8) might help, as well.



KBK said:


> Like disk cataloger I use recoll.
> Personal full text search package, based on Xapian. Command line and GUI, FreeBSD Linux and Windows.
> 
> https://www.freshports.org/deskutils/recoll
> https://www.lesbonscomptes.com/recoll/


Thank you, will consider evaluating this one, if it doesn't have too many dependencies.


----------



## Phishfry (Nov 25, 2018)

That does look nice. I have files spread over 3 file servers for now and I too need some sort of file database.
I keep one box as cold storage and I hate firing it up just for a file search.
Unsure if this program keeps tabs of files in offline storage though.
I see no database in the dependencies.


----------



## Bobi B. (Nov 25, 2018)

Phishfry said:


> That does look nice. I have files spread over 3 file servers for now and I too need some sort of file database.
> I keep one box as cold storage and I hate firing it up just for a file search.
> Unsure if this program keeps tabs of files in offline storage though.
> I see no database in the dependencies.


If you're referring to updatedb(8) and locate(1), the former can be tricked to support multiple separate databases with help of several configuration files (one config file & database per cold filesystem), whereas the latter can be wrapped in a shell script to run real `locate` against each database, and to label each match with the correct cold filesystem name. Maybe it will worth a try? =)


----------



## Phishfry (Nov 25, 2018)

I was thinking something like this:
https://radupopescu.net/2018/05/deneb/


> The metadata of all files and subdirectories in a Deneb directory is recorded in a catalog file. Catalog entries corresponding to files also contain a list of hashes of the chunks making up the file.


Or this:
http://www.portents.com/disktracker/


----------



## Phishfry (Nov 25, 2018)

I will study your recommendation though. I was unfamilar with the locate/locate.updatedb facility.


----------



## Bobi B. (Nov 25, 2018)

Phishfry said:


> I was thinking something like this:
> https://radupopescu.net/2018/05/deneb/
> 
> Or this:
> http://www.portents.com/disktracker/


This Deneb looks interesting, but way too complex and I don't think will fit my case.

DiskTracker, OTOH, seems like what I need, except it is for Mac.


----------



## Phishfry (Nov 25, 2018)

I agree. The catalog idea sounded good.
Interesting concept though using FuseFS like that.
Disktracker does look similar to Neofinder in features. 32bit and near death on Mac.


----------



## Bobi B. (Nov 25, 2018)

Bobi B. said:


> If you're referring to updatedb(8) and locate(1), the former can be tricked to support multiple separate databases with help of several configuration files (one config file & database per cold filesystem), whereas the latter can be wrapped in a shell script to run real `locate` against each database, and to label each match with the correct cold filesystem name. Maybe it will worth a try? =)


Ta-daa! I put together something in a hurry, cfscat, which should stands for cold file-system catalog. Feedback is welcome.


----------



## Phishfry (Nov 26, 2018)

That is some nice work there, Building on locate(1) and updatedb(8).
Can the db handle dates or does that extra field make it too complex?


----------



## Bobi B. (Nov 26, 2018)

Phishfry said:


> That is some nice work there, Building on locate(1) and updatedb(8).
> Can the db handle dates or does that extra field make it too complex?


I believe only (compressed) file names are in locate database, so, no, it will not be possible in its current state.


----------

