# Ptrace, GDB, KDB and LLDB



## ParisHilton92 (Sep 6, 2021)

Hi all,
this is to ask clarification about the most appropriate tool to debug *userspace *applications.

I did some research by reading the source code, and I have several doubts. 

I drop here a list of points:

1- *GDB*: there is a gdb_main.c, but it doesn't use ptrace (as I expected). Rather, it relies on *kdb* to do its operations.
For instance, function kdb_cpu_set_singlestep(...). My impression is that kdb acts only on kernel side (indeed, it directly writes the architectural registers), so it cannot be used by userspace. Is this right?

2- *Ptrace*: it is implemented here, but it seems that any debugger uses it. How does it possible?

3- *LLDB*: my impression is that it is, again, a kernel-debugger. 

Can you make some clarity about it, please? 
Thanks


----------



## SirDice (Sep 6, 2021)

Both gdb(1) and lldb(1) are debuggers that can _also_ be used for kernel debugging.


----------



## ParisHilton92 (Sep 6, 2021)

Hello SirDice,
if gdb in BSD doesn't use ptrace, how can user-space applications be debugged? User-space applications cannot directly write registers .... this is what I see in gdb_main.c .....


----------



## SirDice (Sep 6, 2021)

```
root@molly:~ # sysctl -d security.bsd.unprivileged_proc_debug
security.bsd.unprivileged_proc_debug: Unprivileged processes may use process debugging facilities
```


----------



## ParisHilton92 (Sep 6, 2021)

SirDice said:


> ```
> root@molly:~ # sysctl -d security.bsd.unprivileged_proc_debug
> security.bsd.unprivileged_proc_debug: Unprivileged processes may use process debugging facilities
> ```



Can you explain better this answer please?


----------



## SirDice (Sep 6, 2021)

It means there's a security setting that allows 'regular' users to debug processes. If I recall correctly this is allowed by default but can be disabled (it's one of the security options you can set during the install too). Don't know the exact specifics on how GDB/LLDB exactly work, just that they do as I've used them to debug my own crappy creations too.


----------



## mark_j (Sep 6, 2021)

Gdb doesn't need ptrace, it has an entire infrastructure inside it using its binary file descriptor library to handle non-ptrace systems. Regardless that's some old source code you're looking at that has been completely rewritten. You might want to look on github's freebsd read-only mirror.

Look at sys/kern/sys_process.c for all the hooks.

Oh, and to answer your question:
1. Go try gdb or lldb yourself.
2. See 1.

Is lldb up to the level of gdb? No. Kernel debugging is still done via kgdb. Lldb is very iffy on armv7 and likely armv8 so stick to gdb in those cases. I'm pretty sure lldb is close to gdb on amd64 system; but I only use gdb. Other architectures are the domain of gdb because llvm concentrates on some ARM and Amd64 exclusively.
If you want to learn how gdb works, there's a 'gdb internals' pdf floating around the web. I will find the name of it if you wish.


----------



## mark_j (Sep 6, 2021)

SirDice said:


> It means there's a security setting that allows 'regular' users to debug processes. If I recall correctly this is allowed by default but can be disabled (it's one of the security options you can set during the install too). Don't know the exact specifics on how GDB/LLDB exactly work, just that they do as I've used them to debug my own crappy creations too.


Crappy creations are grist for the mill of gdb.


----------



## mark_j (Sep 6, 2021)

ParisHilton92 said:


> Hello SirDice,
> if gdb in BSD doesn't use ptrace, how can user-space applications be debugged? User-space applications cannot directly write registers .... this is what I see in gdb_main.c .....


Where do you see that?


----------



## ParisHilton92 (Sep 6, 2021)

mark_j said:


> Where do you see that?



Hello Mark,
I answer to this first:

Let's consider this code, here: 


> case 's': {    /* Step. */
> uintmax_t addr;
> register_t pc;
> if (!gdb_rx_varhex(&addr)) {
> ...



Function kdb_cpu_set_singlestep writes the register mdscr_el1.

From source code, I didn't manage to understand where is the entry point from the user-space since GDB for BSD doesn't user ptrace..


----------



## ParisHilton92 (Sep 6, 2021)

mark_j said:


> Gdb doesn't need ptrace, it has an entire infrastructure inside it using its binary file descriptor library to handle non-ptrace systems.



May you help me in understanding this infrastructure? 
I cannot understand how code in gdb/gdb_main.c (kernel-space code, since it accesses to registers) is called by user-space ... I miss something


----------



## SirDice (Sep 6, 2021)

ParisHilton92 said:


> (kernel-space code, since it accesses to registers)


Why do you think only the kernel has access to CPU registers?


----------



## ParisHilton92 (Sep 6, 2021)

SirDice said:


> Why do you think only the kernel has access to CPU registers?



I re-formulate because it's not my point  

My question is that I did not understand how can code in gdb_main.c debug user-space applications without using the BSD implementation of ptrace.

Citing the user "Gdb doesn't need ptrace, it has an entire infrastructure inside it using its binary file descriptor library to handle non-ptrace systems.". Ok, not ptrace, but there should be something to access to the virtual memory of another process: I miss this part..


----------



## mark_j (Sep 6, 2021)

If you want to know the hows and whys of FreeBSD's implementation of gdb into kernel specifically for ARM (which seems to be your question) then freebsd-hackers and/or one of the arm mailing lists would be your direction.

You only mention mdscr_el1 just recently, which is a hell of a lot of relevant information to miss, let alone this is specific to ARM debugging. That register, as you know, is a hardware debug register, yes?, so gdb plonks the breakpoint address in it and waits for the cpu to hit a PC that matches it, then over to gdb. So I'm still not seeing your problem.

The comment remains, you're better off going to the mailing lists and asking the guys who actually write the kernel code, not us end users.


----------



## mark_j (Sep 6, 2021)

ParisHilton92 said:


> Hello SirDice,
> if gdb in BSD doesn't use ptrace, how can user-space applications be debugged? User-space applications cannot directly write registers .... this is what I see in gdb_main.c .....


Yes, this is of course the kernel debug. This is NOT userland debugging.
I think, looking back on your questions you are confused about the structure of FreeBSD, and therefore all BSDs.

The code in /usr/src/sys is all kernel related. Stuff above it, say /usr/src/bin is userland etc. Gdb, the userland debugger is a *port*. Lldb comes as part of llvm in the main source tree (not sure about ARM though.)


----------



## ParisHilton92 (Sep 7, 2021)

Thanks mark_j!
It's true I am confused: I am trying to learn this world  

I am almost convinced that the userland debugger is included in the GDB project (just by reading the source code).
Everything you told me as been very useful. And thanks for the mailing lists too.

Can I ask you something more? I didn't understand what you mean with: 


> Gdb, the userland debugger is a *port*.


----------



## SirDice (Sep 7, 2021)

devel/gdb is a port, i.e. a third party application, it's not included with the base OS. lldb(1) is included with the base OS as part of the Clang compiler suite.


----------



## _martin (Sep 8, 2021)

I'm not aware of any other way of getting registers than ptrace syscall when debugging the process from user space. I don't have any Arm/FreeBSD, but on the amd64 (truncated for brevity):

```
$ truss -c gdb /bin/ls
(gdb) quit
syscall                     seconds   calls  errors
ptrace                  0.009397145     103       0
```
`security.bsd.unprivileged_proc_debug` is toggling the ptrace capability to non-root users.


----------



## _martin (Sep 8, 2021)

ParisHilton92 said:


> I cannot understand how code in gdb/gdb_main.c (kernel-space code, since it accesses to registers) is called by user-space ... I miss something


You missed this: gdb is not a kernel code at all. The whole gdb si running in userspce as a process. It uses ptrace syscall to attach, read and possibly modify the process it attaches to.


----------

