# rm -rf dirname/ && several times in a row deleted all my files (?)



## Avery Freeman (Aug 27, 2019)

Hi,

I just had something weird happen and I was wondering if someone could explain why

I use FreeBSD to record TV.  I have to clear it out every once in a while because I record a lot of news, so today I tried running a lot of `rm -rf <dirname> &&` commands, but it deleted everything after running for a while.

At first, I thought it must've been a `rm -rf *` accidentally put in there somewhere, but I don't see it in the command chain that I ran.  Here's exactly what I ran:


```
rm -rf New\ Day\ With\ Alisyn\ Camerota\ and\ John\ Berman/ && rm -rf Morning\ Joe/ && rm -rf The\ Late\ Show\ With\ Stephen\ Colbert/ && rm -rf PBS\ NewsHour * && rm -rf The\ Situation\ Room\ With\ Wolf\ Blitzer/ && rm -rf CNN\ Newsroom\ * && The\ Axe\ Files\ With\ David\ Axelrod/ && rm -rf The\ Mueller\ Report\ What\ You\ Need\ to\ Know/ && rm -rf CNN\ Debate\ * && America\ Under\ Assault\ The\ Gun\ Crisis\ Cuomo\ Prime\ Time\ Town\ Hall/ && rm -rf The\ Van\ Jones\ Show/ && rm -rf S.\ E.\ Cupp\ Unfiltered/ && rm -rf Saturday\ Night\ Politics/ && rm -rf Smerconish/ && rm -rf 2020\ Democratic\ Candidates\ Debate/ && rm -rf MSNBC\ Live* && rm -rf Reliable\ Sources\ with\ Brian\ Stelter/ && rm -rf State\ of\ the\ Union\ With\ Jake\ Tapper/ && rm -rf CNN\ Democratic\ Presidential\ Debate/ && rm -rf This\ Week\ With\ George\ Stephanopoulos/ && rm -rf Washington\ Week/ && rm -rf Matter\ of\ Fact\ With\ Soledad\ O\'Brien/ && rm -rf Meet\ the\ Press/ && rm -rf Anthony\ Bourdain\ Parts\ Unknown/ && rm -rf Kasie\ DC/ && rm -rf PoliticsNation/ && rm -rf Cuomo\ Prime\ Time/ && rm -rf PBS\ NewsHour* && rm -rf Face\ the\ Nation/ && rm -rf AM\ Joy/ && rm -rf Up\ with\ David\ Gura/ && rm -rf Weekends\ With\ Alex\ Witt/ && rm -rf The\ Beat\ With\ Ari\ Melber/ && rm -rf The\ Rachel\ Maddow\ Show/ && rm -rf Andrea\ Mitchell\ Reports/ && rm -rf CNN\ Right\ Now\ With\ Brianna\ Keilar/ && rm -rf MTP\ Daily/ && rm -rf The\ Lead\ With\ Jake\ Tapper/ && rm -rf All\ In\ With\ Chris\ Hayes/ && rm -rf Deadline\ White\ House/ && rm -rf Anderson\ Cooper\ 360/ && rm -rf CNN\ Tonight\ With\ Don\ Lemon/ && rm -rf Shepard\ Smith\ Reporting/
```

I monitored the progress in another terminal and it ran fine, deleting single dirs at a time, until somewhere around `rm -rf The\ Beat\ With\ Ari\ Melber/`.  I don't see anything weird about the commands, though.

Does anyone see something in that I don't?  Is there another reason that might explain why everything got deleted?

The version is FreeBSD 12.0-RELEASE-p8, and the zpool is a pool+dataset shared from another fileserver through NFS to ESXi and then presented as a local drive (ESXi NFS dataset).  

Thanks!


----------



## spectrum48 (Aug 27, 2019)

```
rm -rf PBS\ NewsHour *
```

You didn't escape the space before the * so you recursively removed both directory PBS NewsHour and * (i.e. everything else in the current directory).


```
rm -rf CNN\ Debate\ *
```

Here you got it right instead, thanks to the backslash before the last space.


----------



## Deleted member 9563 (Aug 27, 2019)

I avoid using spaces in file names. It's just cleaner and easier.


----------



## Avery Freeman (Aug 27, 2019)

OJ said:


> I avoid using spaces in file names. It's just cleaner and easier.



Yeah, the TV recording software just does that by default.  I'm right there with you when I have control over it.


----------



## Avery Freeman (Aug 27, 2019)

spectrum48 said:


> ```
> rm -rf PBS\ NewsHour *
> ```
> 
> ...



Good lord.  I thought that might have been it.  Thanks for your help.

I didn't bother to take a snapshot before I ran it and now my girlfriend is pissed I deleted all her shows.  Wish me luck!


----------



## spectrum48 (Aug 27, 2019)

Avery Freeman said:


> Yeah, the TV recording software just does that by default.  I'm right there with you when I have control over it.



A small trick:

create a temporary directory called delete (no spaces or other special characters);
move the files there (if you make a mistake you can move them back);
double check that all files you moved there need deletion;
delete the temporary directory (double check the command before hitting enter).
This way the dangerous `rm -rf` command is short and easy to check


----------



## ralphbsz (Aug 27, 2019)

Avery Freeman said:


> Good lord.  I thought that might have been it.  Thanks for your help.
> 
> I didn't bother to take a snapshot before I ran it and now my girlfriend is pissed I deleted all her shows.  Wish me luck!


There used to be a joke about storage: "There are two kinds of people. Those who religiously perform backups, and those who haven't lost data yet."

Your sad fate gives rise to a new version of the joke: "... and those aren't single yet."  Well, good luck.


----------



## userxbw (Aug 28, 2019)

rm -rf /dir/* && rmdir /dir 

that'd remove files within dir then remove the same dir. it's a little safer.


----------



## SirDice (Aug 28, 2019)

userxbw said:


> that'd remove files within dir then remove the same dir. it's a little safer.


This would be safer still: `rm -rf /dir`. No wildcard. No need to empty the directory beforehand too, the `-r` takes care of that.


----------



## spectrum48 (Aug 28, 2019)

userxbw said:


> rm -rf /dir/* && rmdir /dir
> 
> that'd remove files within dir then remove the same dir. it's a little safer.



I don't think it's safer, as SirDice noted.

Also, it will fail if there are hidden files since they are not matched by `/dir/*`:


```
$ mkdir /tmp/dir
$ touch /tmp/dir/foo /tmp/dir/.bar
$ rm -rf /tmp/dir/* && rmdir /tmp/dir
rmdir: /tmp/dir: Directory not empty
$ ls -a /tmp/dir
./    ../   .bar
```


----------



## olli@ (Aug 28, 2019)

If you use zsh as your login shell, I strongly recommend to set the option `RM_STAR_WAIT` (also make sure that `RM_STAR_SILENT` is _not_ set; this is the default, however).

The effect is that each time you use `rm` with a wildcard pattern that consists _only_ of a star, like `rm *` or `rm some_dir/*` (but _not_ `rm part*`), the shell will wait ten seconds and ignore anything typed within that time (except for Ctrl-C), then ask for confirmation. This would have saved your day in the above case.

If you don't want to wait, you can always press the `Tab` key. That will expand a wildcard pattern immediately, so you see exactly which files and directories will be affected, and _then_ you can press `Enter` to actually execute the command (without having to wait).

If you don't use zsh as your login shell yet, you should give it a try. It has a lot of useful features. Also, it supports most of the things bash does (some have to be enabled explicitly), so you don't have to learn a lot of new stuff when switching from bash to zsh. You can start learning zsh's features step by step.


----------



## olli@ (Aug 28, 2019)

Another advice, independent from your shell:
If you have important files and directories that you want to keep indefinitely (like your girlfriend's TV shows), protect them by setting the “immutable” flag:

```
chflags -R uchg Valuable_Directory
```
If you do that, `rm -rf` will _not_ be able to remove it. You'll have to clear the immutable flag first, then you can remove it.

By the way: It's a very bad habit to always use the `-f` flag with `rm`. Don't get used to that, it's not necessary. For the same reason, I think it is *bad* to have an alias `rm="rm -i"` because it makes people always use `-f` to override it, making things worse. `rm` should not be an alias; the default behavior is fine.


----------



## SirDice (Aug 28, 2019)

With regards to the original question, I would recommend using quotes. It's really easy to forget or miss escaping a space. 

So instead of `rm -rf New\ Day\ With\ Alisyn\ Camerota\ and\ John\ Berman/` use `rm -rf "New Day With Alisyn Camerota and John Berman/"`


----------



## userxbw (Aug 28, 2019)

spectrum48 said:


> I don't think it's safer, as SirDice noted.
> 
> Also, it will fail if there are hidden files since they are not matched by `/dir/*`:
> 
> ...


well being that it is his (my) data, one should know what one is dealing with. If one needs a more elaborate algorithm, then so be it, that is just one of them basic examples in how the command(s) can be applied.


rm -rfv /dir/* as apposed to rm -rfv /dir which as pointed out already that if spaces resided within the structure then if no precautions are taken one can run into trouble, as already pointed out prior to my post.

run a script that puts everything within a more controlled file structure (perhaps).

```
baseDIr="$HOME/MyStuff"

#To put stuff into and keep it sepertated.
source1="$baseDir/data1"
source2="$baseDir/data2"
#To remove everything including the baseDir.
#the mv command too could be put to use here.
rm -rf "$baseDir"
```
which should only effect everything within the set baseDir

man rm (FreeBSD)


> Recursively remove*  all files* contained within the _foobar_ directory    hierarchy:
> 
> $ rm    -rf foobar



source:




__





						rm
					






					www.freebsd.org


----------



## zirias@ (Aug 28, 2019)

olli@ said:


> By the way: It's a very bad habit to always use the `-f` flag with  rm. Don't get used to that, it's not necessary. For the same reason, I think it is *bad* to have an alias `rm="rm -i"` because it makes people always use `-f` to override it, making things worse.  rm should not be an alias; the default behavior is fine.


Agreed with the first part, but not so much with the second ... AFAIK, `-i` actually stands for _interactive_, so it's designed to be used when typing commands. Whether this should be forced by an alias is a matter of taste -- the design of `rm` clearly supports this by making `-i` and `-f` override each other.

Whether this is dangerous or not depends a bit on your mindset. You should always see `-f` as a _red flag_, and I personally think the wish to interactively remove a whole tree of files without being bothered about every file deserves the usage of such a flag. Sure, if you put `-i` in your alias just to use `-f` always defeats the purpose 

I personally don't have such safety aliases on my own systems, but I wouldn't say it's generally a bad idea ... I'm often logged into other systems that have these configured system-wide, and if an `rm` command starts bothering me, I hit Ctrl+C, maybe double-check my command, and throw an `-f` in...

As for captain hindsight: Another way to avoid this data loss would have been to make use of the autocompletion feature of your shell for typing the names of the directories


----------



## olli@ (Aug 28, 2019)

Zirias said:


> Agreed with the first part, but not so much with the second ... AFAIK, `-i` actually stands for _interactive_, so it's designed to be used when typing commands. Whether this should be forced by an alias is a matter of taste -- the design of `rm` clearly supports this by making `-i` and `-f` override each other.


The problem is, once you have that alias, there is no option to restore the default behaviour. You have to type something like `/bin/rm` or `command rm` in order to circumvent the alias, but nobody does that because it's too much to type. So everybody uses `-f`, but that's more dangerous.


> Sure, if you put `-i` in your alias just to use `-f` always defeats the purpose


Yeah, many people do exactly that.

By the way, FreeBSD supports an option `-I` (upper-case “i”) that is much more user-friendly, see the rm(1) manual page. If this helps people make less (ab)use of `-f`, I'd suggest using an alias with this option instead.


> As for captain hindsight: Another way to avoid this data loss would have been to make use of the autocompletion feature of your shell for typing the names of the directories


Definitely. Also see my comment about zsh’s `RM_STAR_WAIT` above.


----------



## spectrum48 (Aug 28, 2019)

userxbw said:


> rm -rfv /dir/* as apposed to rm -rfv /dir which as pointed out already that if spaces resided within the structure then if no precautions are taken one can run into trouble, as already pointed out prior to my post.



No, there is absolutely no problem in file with spaces in their name *within the structure*. Problems can arise when explicitly typing these files on the command line since an error in escaping or quoting can lead to disaster.


----------



## usakhncit (Aug 28, 2019)

On my Linux box, I am using following strategy:
1. Put following script (saferm.sh) in my /usr/bin/








						zetro - Pastebin.com
					

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.




					pastebin.com
				



2. Created a softlink in my home-dir with name "trash" pointing towards: ~/.local/share/Trash/files/
3. Created an alias: alias rm='saferm.sh'

Now, whenever I use rm command, instead of permanently deleting my files, it moves them to Trash, which can be emptied later.

I haven't checked it on my FreeBSD, but I believe it will also work there.
Thanks


----------



## tingo (Aug 28, 2019)

Also remember that any "dangerous" command can be replaced with `echo` if you want to see what the arguments do before shooting yourself in the foot...


----------



## _martin (Aug 28, 2019)

You could also use find with the appropriate matching pattern and/or other search conditions. First print then remove.


```
cd /dir/where/recordings/are
find . -type d -name <pattern>
# once OKed then
find . -type d -name <pattern> -exec rm -rf {} \;
```


----------



## Avery Freeman (Aug 28, 2019)

spectrum48 said:


> A small trick:
> 
> create a temporary directory called delete (no spaces or other special characters);
> move the files there (if you make a mistake you can move them back);
> ...


They're huge files, so they take a long time to move.  I can just take a snapshot like I should have in the first place.

But I will probably copy-paste all the dir names from `# du -hd 1 | sort -n`output instead of using any wildcards from now on 

It's a bunch of stuff I record regularly so I'll put it in a script...


----------



## Avery Freeman (Aug 28, 2019)

_martin said:


> You could also use find with the appropriate matching pattern and/or other search conditions. First print then remove.
> 
> 
> ```
> ...



That's a great idea!  Thanks!


----------



## olli@ (Aug 29, 2019)

Avery Freeman said:


> They're huge files, so they take a long time to move.


If the destination directory is within the same filesystem, moving files takes near zero time, no matter how huge they are, because only the directory entry is moved. No actual data is moved.


> I can just take a snapshot like I should have in the first place.


That's a good idea.

If you use ZFS, snapshots are rather “cheap” (compared to UFS snapshots), so you might even consider creating snapshots regularly via cron, like nightly or even hourly (depending on the amount of write activity and free space), and removing them automatically after a while. That way you have a simple short-term backup without having to think about it. Remember that snapshots themselves take zero space initially because they only store data that has changed.

Slightly off-topic: The HAMMER filesystem of DragonFly BSD has even better support for fine-grained point-in-time recovery. This is really cool – Once you get used to it, you miss something like that on FreeBSD. Quote: “All non-temporary HAMMER filesystems in DragonFly by default automatically maintain 60 days worth of 1-day snapshots and 1-day worth of fine-grained (30-second) snapshots. These options can be further tuned to meet one's needs.”

By the way, contrary to popular opinion, I think that spaces in file names are perfectly fine, and also non-ASCII characters are no problem. Thanks to the features of modern shells, handling such files is not a big deal. POSIX semantics for filenames supports UTF-8, the _only_ characters that are disallowed in filenames are the slash `/` and the NUL byte (0x00). The only things that – although allowed – should be avoided are non-printable characters and control codes, like newline, backspace, escape codes, zero-width space and similar.
For example, I have files like this: `WALL·E – Der Letzte räumt die Erde auf (Trailer).mkv` (contains three UTF-8 characters and several spaces). To watch the video, I simply type `mplayer WALL[Tab][Enter]`.


----------

