# zfs receive fails, "cannot receive: failed to read from stream"



## wiscodisco (Aug 20, 2019)

Hello,

I backup zfs filesystem from one server (11.1R) to another with the command `zfs send -p $pool/$filesystem | ssh $remotehost "cat $filesystem.zfs"`.  When attempting to receive the zfs filesystem I'm getting the error message (1) `cannot receive: failed to read from stream`.

Is there a way to force the receive, or diagnose why the zfs receive fails?  I get the same error on both an 11.1R and 11.3R instance (amd64).


----------



## wiscodisco (Aug 20, 2019)

Small typo in my command... my backup script has a redirect ">".  i.e.,  `ssh $remotehost "cat > $filesystem.zfs"`


----------



## SirDice (Aug 20, 2019)

wiscodisco said:


> Is there a way to force the receive, or diagnose why the zfs receive fails?


File isn't complete yet? The commands you show don't have a `zfs receive` so I'm assuming you're trying to read that file "on-the-fly", which isn't going to work. Or the send and transfer never completed and thus the resulting $filesystem.zfs is incomplete.


----------



## wiscodisco (Aug 20, 2019)

Thanks for the response...  I'm trying to read from a file created from zfs send.  So the original command, more accurately, was `zfs send -pv $pool/$filesystem@$snap | ssh $user@$remotehost "cat > $filesystem.zfs"`.  
The receive operation is attempted sometime later and is `cat $filesystem.zfs | zfs recv -vud $rpool/`


----------



## SirDice (Aug 20, 2019)

Are you getting the error immediately or after a while? If you get it immediately the cat(1) may simply fail. If it takes a while it's probably because the transfer hasn't completed yet.


----------



## wiscodisco (Aug 20, 2019)

A further observation... I keep four backups of each ZFS filesystem.  The newest three all error out in a similar manner.  I was able to successfully receive the oldest of the four files.  So it would seem something went wrong, silently, with the original zfs send.


----------



## SirDice (Aug 20, 2019)

wiscodisco said:


> So it would seem something went wrong, silently, with the original zfs send.


The connection may have been dropped before it was finished.


----------



## wiscodisco (Aug 20, 2019)

It takes quite some time before it fails, like it's reading most of the file then quits.  The backup I care most about was made months ago.


----------



## SirDice (Aug 20, 2019)

wiscodisco said:


> It takes quite some time before it fails, like it's reading most of the file then quits.


Then it's likely something (most probably the ssh(1) connection) failed before it finished. That would result in an incomplete file at the receiving end.


----------



## wiscodisco (Aug 20, 2019)

I suppose there's no option or sysctl that would force a receive to go as far as it can, rather than abandoning the whole operation?


----------



## SirDice (Aug 20, 2019)

That would result in a partial, and therefor corrupt, filesystem.


----------



## wiscodisco (Aug 20, 2019)

That leaves me with few options.  What's especially insidious is I seen nothing in the output logs of my backup script that there was a failure, either with `ssh` or `zfs send`.  Are you aware of any technique that would detect or alleviate issues like this?  Right now doing a `zfs recv` on the remote host isn't an option.  I appreciate your help.


----------



## SirDice (Aug 20, 2019)

wiscodisco said:


> What's especially insidious is I seen nothing in the output logs of my backup script that there was a failure


You might not be logging the right things. A wrong redirection or getting the return code from the wrong command could do this. 

For example:

```
cat some/non-existing.file | tar -zcvf - test.tgz
if [ $? -ne 0 ]; then
  echo Error!
else 
  echo success!
fi
```
This will always result in a $? of 0, i.e. successful. This is because the last command, the tar(1), is always successful, even if the cat(1) failed.


----------



## wiscodisco (Aug 20, 2019)

true, thanks for the tip


----------



## Eric A. Borisch (Aug 20, 2019)

Consider using bash(1) with the pipefail option, too.


----------

