# FreeBSD BananaPi SDcard u-boot, sector alignment and stripesize - HowTo and DontKnowHowTo +Resume



## Zwie (Jan 14, 2022)

Hello BSDler,

my first post here in the BSD area at all. Please be patient. 

I used FreeBSD before in some virtual environments, compiled simple applications / driver. But not that much.
Instead, I have several years of experience in linux. Just as a sideinfo about me.

I want to use FreeBSD on one or two ARM devices. Because the support for BananaPi M1 A20 SoCs seems good enough (source: https://wiki.freebsd.org/arm/Allwinner#Supported_devices), I start there.

I got it up and running with linux.
Even all documentations and howto's I found were uncomplete about the process how to boot freebsd on arm, because with GENERICSD for armv7 sdcard images found on https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/, it does not boot just plain written on a blank sdcard. The u-boot files where needed too. (A little bit this is a rant on why is it so difficult to find useful information)



This thread shows the progress of installing FreeBSD on an embedded device.

1. Create FreeBSD for embedded devices from Linux
2. TODO: Create FreeBSD for embedded devices from FreeBSD
3. Create a custom Banana Pi image
3a. FreeBSD preparation for SDcard - the sector size
3b. Use FreeBSD to create an image ready to write on sdcard



Create embedded FreeBSD operational system with linux:

Within linux, write the downloaded image (in example https://download.freebsd.org/ftp/re...eeBSD-13.0-RELEASE-arm-armv7-GENERICSD.img.xz) onto a sdcard:

`xzcat -d -c https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/FreeBSD-13.0-RELEASE-arm-armv7-GENERICSD.img.xz | dd of=/dev/mmc0 oflag=direct status=progress`

Then mount the EFI partition (first MS-DOS partition) of the sdcard:

`mount /dev/mmc0p1 /mnt/removable`

Download precompiled u-boot files for FreeBSD 13 as a package from https://freebsd.pkgs.org/13/freebsd-amd64/u-boot-bananapi-2021.07.pkg.html, extract the package and copy u-boot files to the mounted EFI partition:

`tar xf u-boot-bananapi-2021.07.pkg`
`cp {uboot,ubldr} /mnt/removable/`

See the Warning several lines below NOT to copy the boot.scr file.

Now write the u-boot bios file u-boot-sunxi-with-spl.bin, also extracted from the package, to the sdcard's boot sector:

`dd if=u-boot-sunxi-with-spl.bin conv=notrunc,sync of=/dev/mmc0 bs=1k seek=8`

So 8192 Bytes will be skipped from the beginning before writing u-boot-sunxi-with-spl.bin.

As shown on the FreeBSD wiki for ARM (https://wiki.freebsd.org/arm/Allwinner/booting), the u-boot bios file is written after the MS-DOS partition table (MBR). Here the table again, which depends on a sector size of 512 bytes per sector:



sector startsizeusage08192 BytesMS-DOS partition table1632768 BytesInitial SPL loader80524288 BytesU-boot bios1104472 KBreserved2048Free for partions / slices


Covacat commented in this thread:


> the boot space reservation applies only to sunxi boards


That seems the reason the HowTo for the beaglebone - even booting via u-boot - does not need bios data written after the MS-DOS partition table on the boot medium.
Other embedded devices not depending on a bios file needed on the boot medium could start at sector 1024.


Warning: Do _NOT_ copy the boot.scr file onto the boot partition. Yet, if you do, u-boot boots, but stucks on finding the root partition. Here is a UART console output of the failed boot:


```
U-Boot 2020.10 (Jan 24 2021 - 07:06:32 +0000) Allwinner Technology



CPU:   Allwinner A20 (SUN7I)

Model: LeMaker Banana Pi

I2C:   ready

DRAM:  1 GiB

MMC:   mmc@1c0f000: 0

Loading Environment from FAT... *** Warning - bad CRC, using default environment



Setting up a 720x576i composite-pal console (overscan 32x20)

In:    serial

Out:   vga

Err:   vga

Net:   eth0: ethernet@1c50000

starting USB...

Bus usb@1c14000: USB EHCI 1.00

Bus usb@1c14400: USB OHCI 1.0

Bus usb@1c1c000: USB EHCI 1.00

Bus usb@1c1c400: USB OHCI 1.0

scanning bus usb@1c14000 for devices... 1 USB Device(s) found

scanning bus usb@1c14400 for devices... 1 USB Device(s) found

scanning bus usb@1c1c000 for devices... 1 USB Device(s) found

scanning bus usb@1c1c400 for devices... 1 USB Device(s) found

scanning usb for storage devices... 0 Storage Device(s) found

Hit any key to stop autoboot:  0

switch to partitions #0, OK

mmc0 is current device

Scanning mmc 0:1...

Found U-Boot script /boot.scr

199 bytes read in 1 ms (194.3 KiB/s)

## Executing script at 43100000

462412 bytes read in 27 ms (16.3 MiB/s)

## Starting application at 0x42000000 ...

Consoles: U-Boot console

Compatible U-Boot API signature found @0x7af6d138


FreeBSD/armv7 U-Boot loader, Revision 1.3


DRAM: 1024MB

Number of U-Boot devices: 1

U-Boot env: loaderdev not set, will probe all devices.

Found U-Boot device: disk

  Probing all <unknown> devices...

  Checking unit=0 slice=<auto> partition=<auto>...

Found U-Boot device: net

Booting from net0:

net_probe: no network devices found, maybe not enumerated yet..?
netboot: couldn't probe uboot_eth0
net_open: netif_open() failed

ERROR: cannot open /boot/lua/loader.lua: device not configured.
```

This was the only way, the FreeBSD GENERICSD image was able to boot on BananaPi M1.

I've read that the error: *ERROR: cannot open /boot/lua/loader.lua: device not configured.* is known since years and hit several people and user of different system architectures. Yet no fix is available.

So, deleting boot.scr boots the system up and I could login via ssh, because ip address was successfully fetched via dhcp.

TODO: Create embedded FreeBSD operational system with FreeBSD:
TODO
Either way, lets create a custom (or better) image of FreeBSD within FreeBSD itself.


Create a custom Banana Pi image

​FreeBSD preparation for SDcard - the sector size:​​Checking and learning I found that it seems the downloaded and written GENERICSD image is not aligned to sector size boundary? Because of:​​`gpart show /dev/mmcsd0`​
```
=>      63  31116225  mmcsd0  MBR       (15G)[/INDENT]
[INDENT]        63      2016          - free -  (1.0M)[/INDENT]
[INDENT]      2079    102312       1  fat32lba  [active] (50M)[/INDENT]
[INDENT]    104391  31008825       2  freebsd   (15G)[/INDENT]
[INDENT]  31113216      3072          - free -  (1.5M)
```
​It seems FreeBSD does sub-partition the freebsd partition/slice of the MS-DOS partition table, because with the command:​`gpart show -p /dev/mmcsd0`​...it does show the same information with additional the device name.​
```
=>      63  31116225    mmcsd0  MBR       (15G)[/INDENT]
[INDENT]        63      2016            - free -  (1.0M)[/INDENT]
[INDENT]      2079    102312  mmcsd0s1  fat32lba  [active]  (50M)[/INDENT]
[INDENT]    104391  31008825  mmcsd0s2  freebsd   (15G)[/INDENT]
[INDENT]  31113216      3072            - free -  (1.5M)
```
But mmcsd0s2 (freebsd) itself is partitioned:​​`gpart show -p /dev/mmcsd0s2`​
```
=>       0  31008825   mmcsd0s2  BSD          (15G)[/INDENT]
[INDENT]         0        57             - free -     (29K)[/INDENT]
[INDENT]        57  31008768  mmcsd0s2a  freebsd-ufs  (15G)
```
Whereas mmcsd0s1, the boot slice is not sub-partitioned.​`gpart show -p /dev/mmcsd0s1`​
```
gpart: No such geom: /dev/mmcsd0s1.
```
​It is not clearly visible in the simple device overview and at least the command *lsblk* shows another partition mmcsd0s2a within the mmcsd0s2 slice of the msdos partition table:​
```
DEVICE         MAJ:MIN SIZE TYPE                                          LABEL MOUNT[/INDENT]
[INDENT]mmcsd0           0:96   15G MBR                                               - -[/INDENT]
[INDENT]  <FREE>         -:-   1.0M -                                                 - -[/INDENT]
[INDENT]  mmcsd0s1       0:97   50M fat32lba                          msdosfs/MSDOSBOOT /boot/msdos[/INDENT]
[INDENT]  mmcsd0s2       0:98   15G BSD                                               - -[/INDENT]
[INDENT]    <FREE>       -:-    29K -                                                 - -[/INDENT]
[INDENT]    mmcsd0s2a    0:101  15G freebsd-ufs                              ufs/rootfs /[/INDENT]
[INDENT]  <FREE>         -:-   1.5M -                                                 - -
```
​For a beginner like me this is something BSD that does not show of clearly and a new user should be aware of!?​​​_Side-information:_ As SirDice stated in his post (https://forums.freebsd.org/threads/...to-and-dontknowhowto-resume.83723/post-551216), MS-DOS partition are called slices in FreeBSD. Because FreeBSD and UNIX is way older than MS-DOS.​
​This rises a question... (See in the question area).​​Slices start of fat32lba and _especially_ freebsd are not alignet properly to 4096 boundary?​
```
echo "104391/4096" | bc -l[/INDENT]
[INDENT]25.48608398437500000000
```
​As commented here (https://forums.freebsd.org/threads/microsd-card-partition-not-aligned-warning.79559/post-502833) the performance could be bad.​​`diskinfo -c mmcsd0`​
```
mmcsd0[/INDENT]
[INDENT]    512             # sectorsize[/INDENT]
[INDENT]    15931539456     # mediasize in bytes (15G)[/INDENT]
[INDENT]    31116288        # mediasize in sectors[/INDENT]
[INDENT]    4194304         # stripesize[/INDENT]
[INDENT]    0               # stripeoffset[/INDENT]
[INDENT]    SDHC SU16G 8.0 SN 012345678 MFG 08/2012 by 3 SD    # Disk descr.[/INDENT]
[INDENT]    0343B946        # Disk ident.[/INDENT]
[INDENT]                    # Attachment[/INDENT]
[INDENT]    Yes             # TRIM/UNMAP support[/INDENT]
[INDENT]    0               # Rotation rate in RPM[/INDENT]
[INDENT]I/O command overhead:[/INDENT][/INDENT][/INDENT]
[INDENT]    time to read 10MB block      0.481722 sec    =    0.024 msec/sector[/INDENT]
[INDENT]    time to read 20480 sectors   8.433074 sec    =    0.412 msec/sector[/INDENT]
[INDENT]    calculated command overhead                  =    0.388 msec/sector
```
​Half a second for reading 10 MB block does not sound that good, does it? Have to check further...​​So far compared to other distributions freebsd does not seem to provide a "ready for good (am)usement" image, or am I to unexperienced to not see the beauty of the work a user does have to do before using FreeBSD?​​Creating a new sdcard slices/partition layout within FreeBSD 13, lets say via dmarion's HowTo (https://people.freebsd.org/~dmarion/beaglebone/creating_bootable_sd_card/) does seem to make the layout more worse and disaligned? (unproven):​
```
root@freebsd13onarm:~ # gpart show da0[/INDENT]
[INDENT]=>       63  122814401  da0  MBR            (59G)[/INDENT]
[INDENT]         63     131072    1  !14  [active]  (64M)[/INDENT]
[INDENT]     131135   16777216    2  freebsd        (8.0G)[/INDENT]
[INDENT]   16908351  105906113       - free -       (50G)
```

But still, alignment needs to be tested. Therefore, alignment-topic unfinished.


Use FreeBSD to create an image ready to write on sdcard:​​Installing *pkg* within FreeBSD to be able to install packages:​​`pkg ...`​

*Questions:*
Why does the user Fulda in the Thread https://forums.freebsd.org/threads/bananapi-m1-problem-with-ahci-channel-0-freebsd-12-2.81148/ have the *scsi* command, which is not available in my FreeBSD 13r? Is it not available anymore or do I have to install a package? I tried to find scsi command of freebsd in search sites or any command if pkg provides a package name which installs scsi. No luck so far. Because ahci0 is not working for me either, but that should be solved if u-boot for bananapi is configured and compiled. Answered in next post of this thread: https://forums.freebsd.org/threads/...to-and-dontknowhowto-resume.83723/post-551205


Another question came up.
With the command *gpart show -p /dev/mmcsd0* the hidden partition is visible within the freebsd partition of the msdos partition table (as I wrote about in the updated text above), so... does, of the hidden partition mmcsd0s2a as shown by the command *gpart show -p /dev/mmcsd0s2*, the beginning sector add to the unaligned sector of the freebsd mmcsd0s2 of the msdos partition table?
-
In other words:
The freebsd slice mmcsd0s2 starts at sector 104391.
The freebsd-ufs partition mmcsd0s2a starts at sector 57.
Does both sector start add, so the real sector of the partition start is 104448 and fits *also* not so smooth with the 4k boundary, which should be calculated with?


> echo "104448/4096" | bc -l
> 25.50000000000000000000


If so, then everything freebsd writes to is only aligned if both sector beginnigs are added and therefor the comment https://forums.freebsd.org/threads/microsd-card-partition-not-aligned-warning.79559/post-502833 is missing some facts.


Currently FreeBSD is running and usable on bananapi. I will update this thread/post after creating a ready to write image made within FreeBSD or the progress of it.


----------



## SirDice (Jan 14, 2022)

Zwie said:


> Why does the user Fulda in the Thread https://forums.freebsd.org/threads/bananapi-m1-problem-with-ahci-channel-0-freebsd-12-2.81148/ have the *scsi* command, which is not available in my FreeBSD 13r? Is it not available anymore or do I have to install a package? I tried to find scsi command of freebsd in search sites or any command if pkg provides a package name which installs scsi.


That command is part of the U-boot environment, not FreeBSD.

U-boot is a small environment you commonly find on these embedded devices. It takes care of initializing the hardware and booting from an SD card, MMC or USB disk. In a sense it's more or less similar to what a BIOS (or nowadays UEFI) would do on PC (x86) hardware.


----------



## Zwie (Jan 14, 2022)

SirDice said:


> That command is part of the U-boot environment, not FreeBSD.
> 
> U-boot is a small environment you commonly find on these embedded devices. It takes care of initializing the hardware and booting from an SD card, MMC or USB disk. In a sense it's more or less similar to what a BIOS (or nowadays UEFI) would do on PC (x86) hardware.



Thanks for answering both here and in the thread linked.
I thought, I read carefully but it seems I missed the point where Fulda wrote:


> Problem is not on FreeBSD level, but on u-Boot level.
> when I try  scsi info on working one, I get ...



in his post https://forums.freebsd.org/threads/...ahci-channel-0-freebsd-12-2.81148/post-521123

Thanks for clarification.

So what Betzi wrote in his post https://forums.freebsd.org/threads/...ahci-channel-0-freebsd-12-2.81148/post-533486, seems to be the best solution.
I will update my first post.


----------



## Zwie (Jan 14, 2022)

Another question came up. (This text is a copy from the Question section of the first post of this thread)
With the command `gpart show -p /dev/mmcsd0` the hidden partition is visible within the freebsd partition of the msdos partition table (as I wrote about in the updated text above), so... does, of the hidden partition mmcsd0s2a as shown by the command `gpart show -p /dev/mmcsd0s2`, the beginning sector add to the unaligned sector of the freebsd mmcsd0s2 of the msdos partition table?
-
In other words:
The freebsd partition mmcsd0s2 starts at sector 104391.
The freebsd-ufs partition mmcsd0s2a starts at sector 57.
Does both sector start add, so the real sector of the partition start is 104448 and fits *also* not so smooth with the 4k boundary, which should be calculated with?

```
echo "104448/4096" | bc -l
25.50000000000000000000
```
If so, then everything freebsd writes to is only aligned if both sector beginnigs are added and therefor the comment https://forums.freebsd.org/threads/microsd-card-partition-not-aligned-warning.79559/post-502833 is missing some facts.


----------



## SirDice (Jan 14, 2022)

Naming convention is a little tricky for newcomers. With the MBR partitioning scheme, disks are split up into _slices_ (MS-DOS calls those partitions). Disks can have a maximum of 4 slices (in MS-DOS speak; primary partitions). Slices (s1, s2, etc) contain a bsdlabel(8) (a, b, etc), those are called partitions. You are limited to 7 partitions in a bsdlabel(8) (there are 8 but 'c' is special, you can't use it). They're not 'hidden' partitions, MS-DOS/Windows just doesn't know how to deal with it. 



Zwie said:


> The freebsd partition mmcsd0s2 starts at sector 104391.


This is a _slice_, not a partition. You can tell by the s2 name.

And no, we didn't create that confusion. MSDOS came later than BSD. Our naming convention was there first.


----------



## Zwie (Jan 14, 2022)

SirDice said:


> Naming convention is a little tricky for newcomers. With the MBR partitioning scheme, disks are split up into _slices_ (MS-DOS calls those partitions). Disks can have a maximum of 4 slices (in MS-DOS speak; primary partitions). Slices (s1, s2, etc) contain a bsdlabel(8) (a, b, etc), those are called partitions.
> 
> 
> This is a _slice_, not a partition. You can tell by the s2 name.



I will then update my first post to correct the naming given for partitions and slices.
This is something irritating as an linux user, but could be adapted because in linux partitions of MBR or GPT could also be subdivided with the logical volume manager (lvm2) and this is also not clearly visible at first sight.
But, why is gpart not called gslice?  (Don't answer this, really.)


----------



## covacat (Jan 14, 2022)

dont forget to leave about 1M free before the 1st partition for u-boot


----------



## SirDice (Jan 14, 2022)

Zwie said:


> This is something irritating as an linux user


Linux mostly adopted the MS-DOS naming convention. That only added to the utter confusion for newcomers to BSD.

With GPT things have gotten a little more sane. As it was new for all existing operating systems. With GPT disks are simply split up into partitions, you can have a maximum of 128 (from the top of my head). You can easily tell the difference by looking at the name; ada0p1 for example is the first partition on disk ada0. Because of the pX name you instantly know that disk is using the GPT partitioning scheme. A ada0s2a would be the first partition in the second slice of ada0 and you also know this has to be an MBR partitioning scheme.


----------



## Zwie (Jan 14, 2022)

covacat said:


> dont forget to leave about 1M free before the 1st partition for u-boot


Yes.
That's why this private HowTo of dmarion (https://people.freebsd.org/~dmarion/beaglebone/creating_bootable_sd_card/) is irritating. It does not leave some free space at the beginning.

But the question about what sector beginning is really used is still unclear. The GENERICSD image is downloaded from freebsd.org and is also misaligned by the maintainer. If user use this image, they not only get bad performance, they destroy their SDcards much faster. But because it is unclear, I cannot prove what start sector is used for the 4k sector size boundary alignment thing.



SirDice said:


> Linux mostly adopted the MS-DOS naming convention. That only added to the utter confusion for newcomers to BSD.
> 
> With GPT things have gotten a little more sane. As it was new for all existing operating systems. With GPT disks are simply split up into partitions, you can have a maximum of 128 (from the top of my head). You can easily tell the difference by looking at the name; ada0p1 for example is the first partition on disk ada0. Because of the pX name you instantly know that disk is using the GPT partitioning scheme. A ada0s2a would be the first partition in the second slice of ada0 and you also know this has to be an MBR partitioning scheme.


Nice to read that GPT makes some things easier.
But because some new FreeBSD users start with embedded devices because of low hardware investments to use FreeBSD, most of the embedded devices use MS-DOS partition table layouts and this information about slices / (hidden) partitions are much needed to understand the basics.
At the beginning a users task is to write the distribution / OS to the SD card and make it bootable.
Most other things could be learned. Either by doing (trial and error via ssh) or simple wiki readings.



> And no, we didn't create that confusion. MSDOS came later than BSD. Our naming convention was there first.


Didn't know that. Interesting. I should watch a video about (Free)BSD.


----------



## SirDice (Jan 14, 2022)

Zwie said:


> Didn't know that. Interesting. I should watch a video about (Free)BSD.


FreeBSD is a direct descendant of the original AT&T/Bell Labs UNIX code. You can trace much of our code back to the early '70s. Some of the code in the FreeBSD source tree is older than I am (I was born in 1971). 





__





						bsd-family-tree « misc « share - src - FreeBSD source tree
					






					cgit.freebsd.org


----------



## covacat (Jan 14, 2022)

the boot space reservation applies only to sunxi boards


----------

