# why  ncdu and du -sch show different size in FreeBSD and Linux



## mfaridi (Dec 30, 2010)

I have one folder in Ubuntu and move it to FreeBSD Box with scp and run ncdu 
I see this in Linux Box

```
271.1MiB  /portabl
```
and I see this in FreeBSD box 

```
238.2MiB [##########] /portable
```
So I run du -sch in Linux Box and I see this 

```
272M	portable/
272M	total
```
and I run du -sch in FreeBSD and I see this 

```
238M	portable
238M	total
```
why I see different size of one folder in FreeBSD and Linux?


----------



## ckester (Dec 30, 2010)

Any hidden files or directories on the Linux box that might have been missed by the scp?

What filesystem on the Linux box?  What blocksize is it using?


----------



## Pushrod (Dec 31, 2010)

do this on both:

[CMD=""]find . -type f | perl -nle '$s+=-s; END {print $s}'[/CMD]


----------



## mfaridi (Jan 1, 2011)

ckester said:
			
		

> Any hidden files or directories on the Linux box that might have been missed by the scp?
> 
> What filesystem on the Linux box?  What blocksize is it using?



In Linux I use Ext4 and when I run this command df -T , I see this 

```
ext4     7583436   1447540   5750668  21% /
none      devtmpfs      248088       204    247884   1% /dev
none         tmpfs      254064         0    254064   0% /dev/shm
none         tmpfs      254064        32    254032   1% /var/run
none         tmpfs      254064         0    254064   0% /var/lock
none       debugfs     7583436   1447540   5750668  21% /var/lib/ureadahead/debugfs
/dev/sda1     ext2      233191     19949    200801  10% /boot
m
```
and when I run this command mount I see this 

```
/dev/mapper/ubuntu-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
none on /var/lib/ureadahead/debugfs type debugfs (rw,relatime)
/dev/sda1 on /boot type ext2 (rw)
```


----------



## mfaridi (Jan 1, 2011)

Pushrod said:
			
		

> do this on both:
> 
> [CMD=""]find . -type f | perl -nle '$s+=-s; END {print $s}'[/CMD]



when I run your command in linux I see this 

```
229365985
```
and when I run it in FreeBSD I see this

```
214845083
```
they are different


----------



## ckester (Jan 2, 2011)

If you list the files on each machine with `# ls -al` is there any difference in file sizes reported in the results?

You should also try `# ls -as` to see if there is any difference in the number of filesystem blocks used for each file. 

I'm not a filesystem expert (*), and don't know if this is even possible on Unix systems,
but I'm wondering if perhaps what you're seeing is: 
(a) the difference between a somewhat fragmented disk and an unfragmented one; or
(b) a fundamental difference in the efficiency with which the two filesystems use diskspace.

I.e, the difference between

[xxx ][xxx ][xxx ][xxx ] 
and
[xxxx][xxxx][xxxx][    ]

If you count x's they're the same.
If you count [] blocks used, they're different.

(* It sounds like you really need to get an answer, from someone who knows what they're talking about. Sorry, that's not me. You might try asking the freebsd-fs@freebsd.org mailing list instead.)


----------



## wblock@ (Jan 2, 2011)

It sounds like hidden files to me, too.
`% ls -a | wc -l`
is probably different on the two systems.


----------



## mfaridi (Jan 3, 2011)

wblock said:
			
		

> It sounds like hidden files to me, too.
> `% ls -a | wc -l`
> is probably different on the two systems.



I run that command in FreeBSD and I see this 

```
narges ~/jadi/portable [433] ls -a | wc -l                                                                                                 10
```
and I run thay command In Linux and I see this 

```
mostafa@ubuntu:~/portable$ ls -a | wc -l
10
```
FreeBSD and Linux show me one thing


----------



## mfaridi (Jan 3, 2011)

ckester said:
			
		

> If you list the files on each machine with `# ls -al` is there any difference in file sizes reported in the results?
> 
> You should also try `# ls -as` to see if there is any difference in the number of filesystem blocks used for each file.
> 
> ...



when i run ls -la in Linux I see this 

```
total 48
drwxr-xr-x 6 mostafa mostafa  4096 2010-12-30 16:26 .
drwxr-xr-x 7 mostafa mostafa  4096 2010-12-30 16:25 ..
drwxr-xr-x 4 mostafa mostafa  4096 2011-01-02 18:41 db
drwxr-xr-x 3 mostafa mostafa  4096 2010-12-30 16:25 etc
drwxr-xr-x 2 mostafa mostafa 12288 2011-01-02 16:57 pkg
-rwxr-xr-x 1 mostafa mostafa   826 2010-12-30 16:25 powerpill
-rwxr-xr-x 1 mostafa mostafa   129 2010-12-30 16:25 pp
-rwxr-xr-x 1 mostafa mostafa  1461 2010-12-30 16:26 README
-rwxr-xr-x 1 mostafa mostafa   254 2010-12-30 16:25 sync
drwxr-xr-x 4 mostafa mostafa  4096 2010-12-30 16:25 Xyne
```
and when I run ls -as in Linux I see this 

```
total 48
 4 .   4 ..   4 db   4 etc  12 pkg   4 powerpill   4 pp   4 README   4 sync   4 Xyne
```

=======================================================================
when I run ls -la in FreeBSD , I see this

```
total 28
drwxr-xr-x  6 narges  narges   512 Dec 30 12:39 ./
drwxr-xr-x  4 narges  narges  5120 Jan  1 12:21 ../
-rwxr-xr-x  1 narges  narges  1461 Dec 31 15:20 README
drwxr-xr-x  4 narges  narges   512 Dec 30 12:39 Xyne/
drwxr-xr-x  4 narges  narges   512 Dec 30 12:39 db/
drwxr-xr-x  3 narges  narges   512 Dec 30 12:39 etc/
drwxr-xr-x  2 narges  narges  4608 Dec 30 12:46 pkg/
-rwxr-xr-x  1 narges  narges   826 Dec 30 11:36 powerpill
-rwxr-xr-x  1 narges  narges   129 Dec 31 15:20 pp
-rwxr-xr-x  1 narges  narges   254 Dec 31 15:20 sync
```
and when I run ls -as in FreeBSD , I see this

```
total 28
 2 ./          6 ../         2 README      2 Xyne/       2 db/         2 etc/        6 pkg/        2 powerpill   2 pp          2 sync
```
========================================================================
they are different output in FreeBSD and Linux'


----------



## ckester (Jan 3, 2011)

For the time being, disregard the results for the directories in the lists.  Look at the results for the files.  The file sizes are the same, while the block counts are different.  What does that tell you?

Different filesystems, apparently different blocksizes, and probably different levels of fragmentation too.  

What motivates your question in this thread?  Are you worried that you didn't get a complete copy when you did the scp?   

From what I see here, I think you got a complete copy.  But du probably isn't the best way to verify that.  A total checksum of the all the files copied would be better.


----------



## Pushrod (Jan 3, 2011)

If you must know, do a diff.
[CMD=""]diff -rq srcdir destdir[/CMD]

The output from the Perl command should have been the same. I think you did NOT get a complete copy. Use rsync or something to sort this out.


----------



## ckester (Jan 3, 2011)

Pushrod said:
			
		

> The output from the Perl command should have been the same.



For those of us who find Perl opaque, can you briefly explain what that command does?


----------



## ckester (Jan 11, 2011)

Assuming that Pushrod's perl command sums up the file sizes, then I agree, you didn't get a complete copy.

If you haven't already solved the problem, here's one way to track down the missing or incomplete files:

Run this command on each system, in the top directory you used for the scp:

`$ find . -type f -ls | sort -k11 | awk '{print $7 "\t" $11}' > /tmp/list`

Then copy the resulting list from the Linux system to the FreeBSD system and compare them:

`$ diff /tmp/list.linux /tmp/list.freebsd`

The two lists contain the filesize and pathname for each file found. 

If all of the files were copied and are the same size on each system, the two list files will be the same and diff should exit without printing any output.

If a file was incompletely transferred, you'll see it listed twice in the diff output, with different filesize numbers.

If it wasn't transferred at all, you'll see it only once in the diff output, with a *<* as the first character on the line.

If some of the files on your Linux machine are symbolic links, they might have been scp'ed to your FreeBSD box as regular files with size zero.  Each of them will show up only once in the diff output, because the find command I gave you only lists regular files, not symbolic links.    In this case, the first character on the diff output line will be a *>*, indicating that the file was found on the FreeBSD box.

(BTW, that's an eleven, not two L's, in the sort and awk commands above.)


----------



## Pushrod (Jan 11, 2011)

The command `find . -type f | perl -nle '$s+=-s; END {print $s}'` adds the size of each file to "$s". Just before Perl exits, it prints the value of $s.

"-s" in Perl is analogous to "-s" in /bin/test.


----------



## mfaridi (Jan 12, 2011)

ckester said:
			
		

> Assuming that Pushrod's perl command sums up the file sizes, then I agree, you didn't get a complete copy.
> 
> If you haven't already solved the problem, here's one way to track down the missing or incomplete files:
> 
> ...



wow great
I run that perl command in both of system Linux and FreeBSD and move output to one directory and run this command 
	
	



```
diff list.linux freebsd.list
```
 and I see this. I attach txt file, it is big file, how much they are different, I want use rsysnc for this problem


----------

