# Using rsync gives illegal byte sequence (86)



## bangmyhead (Jun 14, 2018)

Using `rsync` for incremental backup its giving me this error in larger files:


```
rsync: recv_generator: failed to stat "/media/da0p1/text1/text2/text3/LARA\#245IGA Lara Ascension.jpg" failed: Illegal byte sequence (86)
```

Is there anyway to fix it? Thanks again.


----------



## jb_fvwm2 (Jun 14, 2018)

rename the file?


----------



## bangmyhead (Jun 15, 2018)

jb_fvwm2 said:


> rename the file?


Impossible, we are talking like about 500 or more files like this and are not mine.


----------



## bangmyhead (Jun 15, 2018)

Can be something related to this maybe? 

https://github.com/aws-samples/aws-workshop-for-kubernetes/issues/196
and this:
https://stackoverflow.com/questions/11287564/getting-sed-error-illegal-byte-sequence-in-bash

I wanted to test but `export` does not work and `set` seems doing nothing. When I do `locale` I get this:


```
# locale
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=
```

Could `rsync` error attached to that? And how I add the "C" to those variables?


----------



## bangmyhead (Jun 15, 2018)

I am not sure but I think it has something to do with that, and more now that I tried to `tar` a folder that content those files and I received this error 

```
Can't translate pathname 'media/da1s1/14/2bs as Tone María.18.doc' to UTF-8
```

Searching more I found this: https://forums.freebsd.org/threads/tar-cant-translate-pathname.32262/

So I tried to use `setenv` and now


```
# setenv LC_MESSAGES en_US.UTF-16
# setenv LC_MONETARY en_US.UTF-16
# setenv LC_NUMERIC en_US.UTF-16
# setenv LC_TIME en_US.UTF-16
# setenv LC_COLLATE en_US.UTF-16
# setenv LC_CTYPE en_US.UTF-16
# setenv LC_ALL en_US.UTF-16
# setenv LANG en_US.UTF-16
```

and again I run `locale`


```
# locale
LANG=en_US.UTF-16
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=en_US.UTF-16
```

So only 2 takes the variable, the rest keep in "C", after this I tried again to `tar` but I still keep getting the same error up there.  I hope someone can give me a hand please. Thanks


----------



## tankist02 (Jun 15, 2018)

Are your files located on an USB drive formatted with FAT(32)? In that case you may need to mount it using -L flag and UTF-8 locale.


----------



## bangmyhead (Jun 16, 2018)

tankist02 said:


> Are your files located on an USB drive formatted with FAT(32)? In that case you may need to mount it using -L flag and UTF-8 locale.


I tried USB with Fat32 and NTFS, even inside a folder on my system but it keeps giving the same error.. By the way how you mount with UTF 8 option?


----------



## bangmyhead (Jun 19, 2018)

I finally solved, after a hard headache ja. It needs to be en_US.UTF-8 so it is like this:


```
#setenv LANG en_US.UTF-8
#setenv LC_ALL en_US.UTF-8
# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8
```

Now my question to close this once for all is how I add it permanent to my system?


----------



## tankist02 (Jun 21, 2018)

Here is the mount command I used:


```
sudo mount_msdosfs -o large -L en_US.UTF-8 /dev/da0s1 ~/mnt/usb
```


----------



## bangmyhead (Jun 21, 2018)

tankist02 said:


> Here is the mount command I used:
> 
> 
> ```
> ...



I will try it and let you know, thanks.


----------



## bangmyhead (Jun 27, 2018)

Well back here again while trying to solve another problem, I found that editing /etc/login.conf I can add it permanent but it does not work, so my guess is that I am avoiding some steep maybe:
As it said here:
https://www.freebsd.org/doc/handbook/using-localization.html

I edited as I said /etc/login.conf and added this:

```
#Espagnol
spanish|Spanish Users Accounts:\
        :charset=es_ES.ISO8859-1:\
        :lang=es_ES.ISO8859-1:\
        :tc=default:
```

then I run `# [B]cap_mkdb /etc/login.conf[/B]` and after that I check by running `locale` and nothing change, I still see 


```
# locale
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=
```

Any help is welcome...


----------



## bangmyhead (Jun 27, 2018)

bangmyhead said:


> Well back here again while trying to solve another problem, I found that editing /etc/login.conf I can add it permanent but it does not work, so my guess is that I am avoiding some steep maybe:
> As it said here:
> https://www.freebsd.org/doc/handbook/using-localization.html
> 
> ...



Ok, I solved by adding to the end of ~/.cshrc this:


```
setenv LANG es_ES.ISO8859-1
setenv LC_ALL es_ES.ISO8859-1
```

So each time computer starts it change it to those variables, thanks to all.


----------



## SirDice (Jun 27, 2018)

Don't set LC_ALL, it overrules any and all other LC_* variables.


> LC_ALL
> This variable determines the values for all locale categories. *The value of the LC_ALL environment variable has precedence over any of the other environment variables starting with LC_ (LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, LC_TIME) and the LANG environment variable.*


http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html



bangmyhead said:


> So each time computer starts it change it to those variables


The ~/.cshrc file is read after you are logged in, not when the computer is started. It is a subtle but very distinct difference.


----------



## bangmyhead (Jun 27, 2018)

SirDice said:


> Don't set LC_ALL, it overrules any and all other LC_* variables.
> 
> http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
> 
> ...



Oww, do you know how I can set it permanent or after each restart?


----------



## SirDice (Jun 28, 2018)

Setting it in ~/.cshrc is permanent. You just need to be aware it's set for _your_ account only and is only executed after you login. It's also only for csh(1) or tcsh(1), other shells use different files (~/.profile) for example. 

If you want to set it for _all_ users you could use /etc/csh.cshrc (for C shells) and /etc/profile (for Bourne shells).


----------



## bangmyhead (Jun 28, 2018)

SirDice said:


> Setting it in ~/.cshrc is permanent. You just need to be aware it's set for _your_ account only and is only executed after you login. It's also only for csh(1) or tcsh(1), other shells use different files (~/.profile) for example.
> 
> If you want to set it for _all_ users you could use /etc/csh.cshrc (for C shells) and /etc/profile (for Bourne shells).


Ok, but this is an "stand alone" server so I wont login on it. I am now inside because I am setting some task and fixing this configurations problems but once it is done I will log in maybe 1 time in a month.
So is there anyway to set those variables? I mean to let it permanent (by default) with out login in the server.


----------



## phoenix (Jun 28, 2018)

If you will be running rsync from a script, then set those variables in the script that calls rsync.


----------



## bangmyhead (Jun 29, 2018)

phoenix said:


> If you will be running rsync from a script, then set those variables in the script that calls rsync.


`rsync` is inside cron.


----------



## densan (Jun 29, 2018)

CHARSET = utf-8


----------



## phoenix (Jun 29, 2018)

bangmyhead said:


> `rsync` is inside cron.



Don't run it directly in the crontab.

Write a script that handles everything; setting the LC variables, setting the PATH, running rsync with all the right options, etc.  Then call that script in crontab.

Trust me.  It will save you weeks of headaches doing it that way.  Especially if you need to change things in the future.


----------



## bangmyhead (Jul 2, 2018)

phoenix said:


> Don't run it directly in the crontab.
> 
> Write a script that handles everything; setting the LC variables, setting the PATH, running rsync with all the right options, etc.  Then call that script in crontab.
> 
> Trust me.  It will save you weeks of headaches doing it that way.  Especially if you need to change things in the future.



Ok, I will follow your advice, but since I do not use scripts in FreeBSD can you give me an example of how to do it please?


----------



## phoenix (Jul 3, 2018)

Save this into a file named something like /root/scripts/rsync-backup.sh

```
#!/bin/sh

# set locale
LANG=es_ES.ISO8859-1
LC_CTYPE=es_ES.ISO8859-1
LC_COLLATE=es_ES.ISO8859-1
LC_TIME=es_ES.ISO8859-1
LC_NUMERIC=es_ES.ISO8859-1
LC_MONETARY=es_ES.ISO8859-1
LC_MESSAGES=es_ES.ISO8859-1

/usr/local/bin/rsync --verbose --archive --other-options --rsh=ssh /path/to/backup/ remoteusername@remoteservername:/path/to/write/to/
```

Make it executable:  `chmod u+x /root/scripts/rsync-backup.sh`

Then enable it in crontab (this would run it at 11:00 pm every night):

```
00 23 * * * /root/scripts/rsync-backup.sh
```


----------



## bangmyhead (Jul 4, 2018)

phoenix said:


> Save this into a file named something like /root/scripts/rsync-backup.sh
> 
> ```
> #!/bin/sh
> ...



Thanks a lot, I will try out now.


----------

