# tar copy files without metadata



## tOsYZYny (Apr 28, 2020)

How can I copy files and directories without overwriting the existing metadata on the target?  I have configuration files in a repository that configure the system.  The source files are owned by a regular user, but the target has files owned by other users including root.

`tar cf - . | tar xf - -C ../extract`


When running ls -l later, I see that the permissions on a file that was owned by root is now owned by myself.

I tried omitting -p from both creation and extraction as well as --no-same-permissions and --no-xattrs, but it still ends up overwriting the permissions.  I can do a chown and chmod later, but was hoping there was a better way to do that.

I just want to update the contents of the files.  For any files that do not already exist, I will have to chmod/chown them.


----------



## Alain De Vos (Apr 28, 2020)

I would use "rsync --ignore-existing"


----------



## tOsYZYny (Apr 28, 2020)

That doesn't work as the contents at the target are not updated.  I'm not too familiar with rsync, but perhaps there might be a way with it.


----------



## ralphbsz (Apr 29, 2020)

Are you running tar as root, or as yourself? In general, when running as a non-root user, tar will create files (when extracting) with the ownership being the current user; when running as root, it will create files with the ownership as specified in the archive. For details (they are complex!), read the man page: tar(1)


----------



## tOsYZYny (Apr 29, 2020)

I am running tar as root since I need to extract files to /.  Yes, I referenced the man page.  I believe the confusion I have is the gnu tar appears to only update metadata for the leaf nodes whereas the BSD tar appears to update all the nodes (directories) in-between.  For example, I am extracting some configuration files in /etc.  I am noticing that the user I used to create the archive now owns the root "/" directory.

I will try with the gnu tar as well.


----------



## ralphbsz (Apr 29, 2020)

You can list the content of the tar file (if you do "tar -vt ..." you see the full details), that should show you what directories and files are in the archive, and who owns them. If the archive contains the directories, and you are untarring that archive as root, then it stands to reason that tar will change the ownership on the directory to match whatever it's configuration says.

Here's a trick that MIGHT help: Instead of recursively creating a tar archive that has the whole directory tree, only tar the actual files, skipping any directories. That could be done by using find to create a list of files, and then using tar to create an archive, with all the files on the command line, or perhaps using the -T option.

Or switch to using rsync ... it has even more command-line options than tar, and can be taught to do nearly everything, if you have enough patience.


----------



## mark_j (Apr 29, 2020)

tOsYZYny said:


> How can I copy files and directories without overwriting the existing metadata on the target?  I have configuration files in a repository that configure the system.  The source files are owned by a regular user, but the target has files owned by other users including root.
> 
> tar cf - . | tar xf - -C ../extract
> 
> ...


What do you mean by metadata? Like ZFS arc meta data?


----------



## ralphbsz (Apr 29, 2020)

I think he means object ownership. File system language is inconsistent: the same user-visible things are sometimes called file attributes, sometimes file system metadata. They include things that can be read via the stat system call (file size, ownership, protection bits, times), and other things like ACLs and extended attributes. Backup/archive/packaging programs like tar need to read/write them, and depending on which file system system and which archive tool you're using, they can to a better or worse job.

In addition to all that, there is file-system internal metadata, such as disk location tables (inodes, indirect blocks), quota, allocation tables, and lots of other things. Those tend to not be visible in user-facing interfaces.


----------



## ralphbsz (Apr 29, 2020)

Actually, going back to the original question: tOsYZYny asked how he could:

Overwrite existing files in the target from the source, without modifying their ownership (and I presume their permissions).
Create new files in the target from the source, and he is willing to manually fix their ownership and permissions.
All that without modifying the attributes of directories in the target tree.
And I presume create new directories in the target tree where necessary.
Here is a solution that might be much easier, and doesn't involve heavyweight tools like tar or rsync: Write a little script that runs find in the source tree, locating all files (not directories). For each of those files, create a new directory if needed (take dirname of the name of the file, put the result into mkdir -p, or perhaps do an if statement to only create the directory if it doesn't exist). Then copy the file into place with "cat source > target", which will not modify the ownership of the target if it exists. Would be nice to also make sure the source tree doesn't contain anything other than directories or files (like soft links). I think this could be done in a 20-line script which would take someone comfortable with sh about 15 minutes to write. Much less time than it takes to read all the man pages for complex tools, understand them, and then ultimately they still do something one didn't expect.


----------



## mark_j (Apr 29, 2020)

ralphbsz said:


> I think he means object ownership. File system language is inconsistent: the same user-visible things are sometimes called file attributes, sometimes file system metadata. They include things that can be read via the stat system call (file size, ownership, protection bits, times), and other things like ACLs and extended attributes. Backup/archive/packaging programs like tar need to read/write them, and depending on which file system system and which archive tool you're using, they can to a better or worse job.
> 
> In addition to all that, there is file-system internal metadata, such as disk location tables (inodes, indirect blocks), quota, allocation tables, and lots of other things. Those tend to not be visible in user-facing interfaces.


Yes, sorry, I should have read it thoroughly rather than glancing at it.


----------



## tOsYZYny (May 7, 2020)

Thanks for all your replies - I think the solution that is working for me so far is using rsync.  I think am using this, but it is commented from my install script, so I presume it must need further testing:
rsync -lmrt $1/ /

I have files and some symlinks, so that is why I was using tar.


----------



## mark_j (May 7, 2020)

rsync can copy symlinks, see -L

Beware though of symlinks and rsync as you might end up in some loop. (Not knowing your data structure, of course).


----------



## tOsYZYny (May 7, 2020)

Thanks, I will test thoroughly first.  The last thing I want is to nuke my system .


----------



## gpw928 (May 7, 2020)

The default options for rsync are completely disappointing.  If favours speed over accuracy.  I would try this syntax and option set (for fewest surprises in the outcome):
	
	



```
rsync -SHAXax --ignore-existing /rooted/source-directory/ /rooted/destination-directory/
```
It's worth your while to decipher that option bundle.


----------



## mark_j (May 7, 2020)

Yes, that's probably a good idea to avoid ruining your file system. Can you set up a virtual machine?


----------



## tOsYZYny (May 7, 2020)

I do have some options ... Right now, I am using boot environments, and I did mess up my system earlier, but was able to recover thanks to some very helpful hints on how to boot another BE after the boot menu would not come up.  And, what I have now is basically a setup script that restores my install to how it was, packages, configs, users, etc.  But yes, to save the aggravation, I will use a 'chroot' first before writing the changes live and before that, verify the list I'm about to overwrite.


----------



## mark_j (May 7, 2020)

Yes, aren't boot environments grand!


----------

