# Checking the existance of files



## balanga (Aug 28, 2018)

Whilst trying to do a big housekeep of files across my various systems it emerges that I cannot trust what a directory listing displays.

`ls` displays, say abc but I try `file -s abc` and see 





> No such file or directory



Is there any way to check if the directory listings are correct? They are usually on remote filesystems.


----------



## leebrown66 (Aug 29, 2018)

You are asking `file` to traverse the directory abc.  You probably want `find -s . -name abc`, ie find starting from current directory, print all files matching the name abc.


----------



## Nicola Mingotti (Aug 29, 2018)

To me `file` never failed.

I belive it is higly probable your problem comes from quoting.

Silly example of what can be the cause.

```
$> echo "foo" > 'hello world.txt'
$> ls -1rt | tail -n 5
...
pinList-H9.csv
hello world.txt
$>  file pinList-H9.csv      ==> ok 
$> file hello world.txt     ==> Error
```


----------



## balanga (Aug 29, 2018)

leebrown66 said:


> You are asking `file` to traverse the directory abc.  You probably want `find -s . -name abc`, ie find starting from current directory, print all files matching the name abc.



The file exists in the directory listing, ie find() finds it but I can't do anything with it. I'm told there is no such file or directory. It may have something to do with invalid characters or embedded spaces.


----------



## leebrown66 (Aug 29, 2018)

ls(1)
Have you tried:
-B      Force printing of non-printable characters (as defined by
             ctype(3) and current locale settings) in file names as \xxx,
             where xxx is the numeric value of the character in octal.  This
             option is not defined in IEEE Std 1003.1-2001 (``POSIX.1'').


----------



## balanga (Aug 30, 2018)

Not really sure what you mean. ls() will display filenames but the files themselves are not accessible and do not appear to exist. I've moved directories in which they appear but it makes no difference.


----------



## aragats (Aug 30, 2018)

leebrown66 meant that ls(1) may not display certain characters in the current locale, but with *-B* option it will show them in octal representation.
In other words, your _"abc"_ may be listed as e.g._ "abc\567"_.


----------



## balanga (Aug 30, 2018)

aragats said:


> leebrown66 meant that ls(1) may not display certain characters in the current locale, but with *-B* option it will show them in octal representation.
> In other words, your _"abc"_ may be listed as e.g._ "abc\567"_.




Ah, I see now. The '?' characters get shown as '\221' or somesuch....

A further development is that I can access the files from a Windows machine. The files are actually on a FreeNAS box so it isn't an NTFS issue.

I was hoping to identify which files I could access and came up with 
	
	



```
find . -exec file {} \; > filelist
```

which shows both files that are accessible and those which aren't. Wondered if there was some way to programmatically split them up. I see file() returns 0 on success,    and non-zero on    error.. Presumably it returns non-zero when it can't access the file, but I can't figure out how to catch that in my example.


----------



## SirDice (Aug 30, 2018)

balanga said:


> I see file() returns 0 on success, and non-zero on error.. Presumably it returns non-zero when it can't access the file, but I can't figure out how to catch that in my example.


http://www.grymoire.com/Unix/Sh.html#uh-46


----------



## ralphbsz (Aug 31, 2018)

As I pointed out in another thread: Problems like this can arise if the file system is running in a strange locale and/or the file names contain strange characters.  This can lead to effects like multiple files with the same name in a directory , or files that are absolutely inaccessible.  Supposedly it often helps to put all user processes involved into a utf8 locale, because then fewer character set conversions need to be done.

The idea of running "file" inside "find" to check whether a file is accessible is insane, because it might fail due to a myriad of other reasons.  It is also unnecessary.  You can easily do the equivalent thing in shell: take the output of "find", pipe it into a shell loop (to prevent names with embedded blanks from being broken apart by the shell, use "-print0" to print the file name in find), then in the shell loop check whether files exist or accessible (readable, executable) using the standardized test in if statements (do a "man test" to read about those).  Since the file names (in the output of find) are null-terminated, it is possible to read them with trickery into shell variables (look at IFS= and the -d option of read) without any trimming of leading or trailing blanks, with blanks in the middle, and with international characters.

And if you find files that even these techniques don't work well on, then the right answer is to call the readdir() system call from a programming language, and process the output yourself, for example looking for special characters.  This can be done quite easily in perl or python.


----------



## balanga (Aug 31, 2018)

I'd forgotten about test()....

I tried to pipe find() into test() using

```
find . -exec test -e {} || echo {} >>badfiles \;
```

but I seem to be way off with the syntax...


----------



## gpw928 (Sep 3, 2018)

Hi, I don't have a FreeBSD system to test it on right not, but my best guess is:

```
find . -exec test ! -e {} \; -print >>badfiles
```


----------



## balanga (Sep 3, 2018)

gpw928 said:


> Hi, I don't have a FreeBSD system to test it on right not, but my best guess is:
> 
> ```
> find . -exec test ! -e {} \; -print >>badfiles
> ```




I tried that but couldn't make it work.


----------

