# rclone not working with W^X



## matt_k (May 6, 2021)

Hello,

I am trying to use W^X on my FreeBSD box. Basically, all the software I use daily either works, or works with `elfctl -e +wxneeded`, except for rclone.


```
~> rclone
exec_new_vmspace: mapping stack size 0x20000000 prot 0x7 failed mach error 2 errno 13
Abort trap
~> pls sysctl kern.elf64.allow_wx=1
kern.elf64.allow_wx: 0 -> 1
~> rclone
Usage:
  rclone [flags]
  rclone [command]i
...
...
...
```

rclone is also resistant to elfctl and apparently, it cannot be set.

```
root@fbsd:~ # elfctl -e +wxneeded /usr/local/bin/rclone
elfctl: NT_FREEBSD_FEATURE_CTL note not found
elfctl: NT_FREEBSD_FEATURE_CTL note not found
root@fbsd:~ # elfctl  /usr/local/bin/rclone
elfctl: NT_FREEBSD_FEATURE_CTL note not found
```

Is there anything else I can do, or should I just wait and not use W^X until this is somehow resolved? (I use rclone multiple times a day, so not using it is not an option).

Edit: sorry I forgot to mention that this is on FreeBSD 13.0-RELEASE, all packages up to date, every piece of sotware installed via pkg.


----------



## mark_j (May 11, 2021)

Interestingly, in here: https://www.freebsd.org/status/report-2021-01-2021-03/, under the section "Vulnerability Mitigations", Ed Maste says:

"We added support for enforcing a write XOR execute mapping policy. *It is enabled by setting the kern.elf64.allow_wx and/or kern.elf32.allow_wx sysctls to 0 (for 64-bit and 32-bit binaries, respectively)*. Binaries can indicate that they requre writeable and executable mappings by setting the NT_FREEBSD_FCTL_WXNEEDED ELF feature bit, set via elfctl."

Perhaps it's a misprint?

The man page is just as obtuse as ever: security(7)
Edit: In fact the man pages for R13 are still -Current, and so don't include the changes made to kern.elf{32,64}.allow_wx
See here, though: https://cgit.freebsd.org/src/commit/?id=2e1c94aa1fd582fb8ae0522f0827be719ff5fb67

(I am not running FreeBSD 13 yet, so I cannot confirm any of this.)


----------



## Alain De Vos (May 11, 2021)

Maybe it's an issue. In your case I would enter the issue.
Maybe there are alternatives for rclone ?


----------



## matt_k (May 11, 2021)

mark_j said:


> Binaries can indicate that they requre writeable and executable mappings by setting the NT_FREEBSD_FCTL_WXNEEDED ELF feature bit, set via elfctl."


Yes, but rclone does not have/does not allow manipulating this settable bit. There were some other programs on my system that had issue with W^X and I enabled this bit with no issue on them.


Alain De Vos said:


> Maybe it's an issue. In your case I would enter the issue.
> Maybe there are alternatives for rclone ?



Alain: perhaps there are, but my rclone is already an alternative to another program (a cloud provider's flatpak or snappack or whatever that is). EDIT: I contacted the FreeBSD rclone port maintainer.


----------



## mark_j (May 11, 2021)

Is this rclone a port? Can you compile with -Wl,-z,wxneeded?


----------



## matt_k (May 11, 2021)

mark_j said:


> Is this rclone a port? Can you compile with -Wl,-z,wxneeded?


I will try it when I get back to my FreeBSD box. It's a package installed via pkg.


----------



## mark_j (May 11, 2021)

If you have the time, inclination and motivation then use poudriere to build the port and test it in that jail.


----------



## iclis (May 19, 2021)

Fresh FreeBSD box here, installed bhyve via pkg. Got same error while trying to launch a newly created VM.

```
exec_new_vmspace: mapping stack size 0x20000000 prot 0x7 failed mach error 2 errno 13
Abort trap
```
Did same what was advised here - *sysctl kern.elf64.allow_wx=1* - it resolved the issue. 
Not sure what exactly it does and whether it's safe at all. Please advise.


----------



## SirDice (May 19, 2021)

iclis said:


> installed bhyve via pkg.


bhyve(8) is part of the base OS, you don't install it with a package.


----------



## _martin (May 19, 2021)

mark_j You highlighted the sentence


mark_j said:


> *It is enabled by setting the kern.elf64.allow_wx and/or kern.elf32.allow_wx sysctls to 0*





mark_j said:


> Perhaps it's a misprint?


It's kinda double negative. By setting the 'allow wx' to 0 you're turning ON the feature. At least I'm assuming you wanted to point that out with the highlighted text.


----------



## mark_j (May 19, 2021)

N


iclis said:


> Fresh FreeBSD box here, installed bhyve via pkg. Got same error while trying to launch a newly created VM.
> 
> ```
> exec_new_vmspace: mapping stack size 0x20000000 prot 0x7 failed mach error 2 errno 13
> ...



Not all things run with it enabled.


----------



## Alain De Vos (May 20, 2021)

I have "kern.elf64.allow_wx=0", and bhyve works justs fine ...
kern.elf64.allow_wx=1 is i think the default and allows address space to be and writeable and executable.


----------



## mark_j (May 20, 2021)

If I understand _martin correctly, that would make it off by default, as it's a double negative setting.

Unfortunately, I don't have a 13 machine to test on.


----------



## _martin (May 20, 2021)

mark_j said:


> If I understand _*[FONT=monospace]_martin[/FONT]*_ correctly, that would make it off by default, as it's a double negative setting.



I think the variable name is a bit confusing. Normally this feature is called "write xor exec", or W^X. But this variable behaves more like WRITE|EXEC.


```
# sysctl -d kern.elf64.allow_wx
kern.elf64.allow_wx: Allow pages to be mapped simultaneously writable and executable
#
```
So 1 enables to have write and exec pages, 0 disables it (and hence enables W^X).


----------



## matt_k (May 20, 2021)

Exactly.
=0 enables W^X protection, meaning you can't have write and execute pages in memory. I don't see the sysctl name being confusing. 
In the meantime, I contacted the rclone port maintainer, who said something along the lines that he will release the maintainership of rclone port and can't help here. 
So, I guess that's that. Can I somehow build the port with the aforementioned options even when I don't have poudriere, or it is not possible, as those options are not enabled in the port and `make config` does not list any options to build rclone with?


----------



## Alain De Vos (May 20, 2021)

If rclone needs write&execute memory pages, with w^x feature disabled, you must enable it to use it with kern.elf64.allow_wx=1
Just like if you want to compile jave you must include compat in the kernel.


----------



## matt_k (May 20, 2021)

Yes, but as I am writing above, I can't enable it with `elfctl` and that's what this thread is about.


----------



## _martin (May 20, 2021)

Alain De Vos said:


> If rclone needs w^x memory pages you must enable it to use it.


No, not correct. rclone requires W^X to be disabled (i.e. for some strange reason it requires rwx page like we are in 90s). For that you need to enable the kern.elf64.allow_wx tunable.
Confusion stems from what I wrote above.

matt_k
mark_j gave you an example how to do it:


mark_j said:


> Can you compile with -Wl,-z,wxneeded?


For the `elfctl` to work you need to have a special program header in ELF binary which may or may not be there. The link flags above do create that section.


----------



## matt_k (May 20, 2021)

_martin said:


> No, not correct. rclone requires W^X to be disabled (i.e. for some strange reason it requires rwx page like we are in 90s). For that you need to enable the kern.elf64.allow_wx tunable.
> Confusion stems from what I wrote above.
> 
> matt_k
> ...


Yes, thanks. So far I don't know how to do that, so some studying will be necessary. 

I guess learning poudriere is a worthwhile skill anyway. 
I will leave this thread open until I make it happen somehow.


----------



## _martin (May 20, 2021)

`rclone` is written in _go_ language, I don't know what are the differences in build process and what can be done to influence the outcome.

I've looked into the wxneeded linker option and I don't think that helps in this case. That option creates OPENBSD_WXNEED segment in the ELF file which seems to be something different to what `elfctl` is going for. `elfctl` is parsing the NOTE program header. That's the one that influences the behavior in the end when rwx page is requested and sysctl toggle is set to 0.


----------



## matt_k (May 20, 2021)

I checked all ELF files in /bin and /usr/local/bin for NT_FREEBSD_FEATURE_CTL note. Only four files on my system do not have this note.
fzf (written in go)
go (written in go)
gofmt (written in go)
rclone (written in go)

There is also one program written in go, that has this note and +wxneeded can be set and thats siggo (terminal signal client). But I built it as a user using the go build system, not through packages/ports. So I tried doing the same with rclone and the result is no CTL note again. 
I guess, something useful might come out of this -- I won't use programs written in go.


----------



## Alain De Vos (May 21, 2021)

golang does not compile.
Error,
exec_new_vmspace: mapping stack size 0x20000000 prot 0x7 failed mach error 2 errno 13
exec_new_vmspace: mapping stack size 0x20000000 prot 0x7 failed mach error 2 errno 13


----------



## _martin (May 21, 2021)

matt_k Yesterday was the first time in my life I saw _go_. I've seen some flags one can set during build process but a) they got ignored b) as stated above wouldn't help at all.
I'm trying to find a way to specify/include PT_NOTE program header in `go build` or something similar.


----------



## _martin (May 23, 2021)

Alain De Vos It doesn't compile because internally it uses bash (make.bash) which also uses wants rwx page for some reason. I really doubt there's valid reason for it (on Linux rclone doesn't use rwx page at all).

Anyway it seems FreeBSD's default linker (ld.lld(1)) is not capable of creating such entry. If I want to compile my own binaries that don't use libc I need to take care of it myself. Also it seems elfctl(1) only checks for NT_FREEBSD_FEATURE_CTL while kernel checks first for NT_FREEBSD_ABI_TAG and hence both entries in the note section need to be present. I first used only the former resulting in the file manageable by elfctl but ignored by kernel.

If I create this dummy file (inspired by /usr/src/lib/csu/common/feature_note.S):
note.S

```
.section .note.tag,"a", %note
        .p2align        2
        .4byte          8,4,1
        .asciz          "FreeBSD"
        .p2align        2
        .4byte          1300139

        .p2align        2
        .4byte          8,4,4
        .asciz          "FreeBSD"
        .p2align        2
        .4byte          0
```
and run `cc -c note.S` I get the object file I can link against my binary and achieve the needed result.

Now the question is: can this be used when building go itself and/or rclone ? That I don't know..


----------



## meena (Sep 2, 2021)

Hey folks! just dropping by to say I've filed a filed a bug with upstream.
This means that in due time (it's slated for 1.18), go will fix their tool-chain to produce binaries on FreeBSD which aren't automatically marked as RWX. (the automatic part is happening in our ld.so or kernel, as we're not getting the right clues from the binary)

We've also found out that you can circumvent this partially*, by building with `-buildmode=pie`. It then seems to use our native toolchain (ld?), and as such produces binaries that work under W^X, aslr, and so forth…

(*Partially in so far, as that we haven't found out how to build go itself as PIE.)

I've submitted a diff for review for changing the default build mode in the ports framework, but that's still a bit all over the place and who knows how many ports other than net/syncthing need to be touched.


----------

