# Reading / Writing in serial port



## jjachuf (Nov 13, 2021)

Hi,

I am attempted to communicate in a bidirectional way with an Arduino device using the serial port, obviously in FreeBSD. I have to read what comes from the Arduino and also to write.

I tried with *Python* [1] and *Rust* [2] and in both cases I get errors, but the same code in Linux or Windows works fine. 
I was wondering if it is necessary to touch some variable with *sysctl* or something similar in FreeBSD, o there is any limitation in FreeBSD

I can do it in Linux, but I want to do it in FreeBSD because all the rest of the app is ready to deploy as a jail. 


[1] https://github.com/pyserial/pyserial/discussions/622
[2] https://gitlab.com/susurrus/serialport-rs/-/issues/114


----------



## SirDice (Nov 13, 2021)

Check the permissions on /dev/cuaU0:

```
root@molly:~ # ll /dev/cuau0*
crw-rw----  1 uucp  dialer  0x43 Nov  9 21:48 /dev/cuau0
crw-rw----  1 uucp  dialer  0x44 Nov  9 21:48 /dev/cuau0.init
crw-rw----  1 uucp  dialer  0x45 Nov  9 21:48 /dev/cuau0.lock
```
As you can see only the uucp user and the dialer group are able to read/write to the device. Simplest solution is just to add your user to the dialer group. `pw groupmod dialer -m <user>`


----------



## jjachuf (Nov 13, 2021)

SirDice said:


> Check the permissions on /dev/cuaU0:
> 
> ```
> root@molly:~ # ll /dev/cuau0*
> ...



Hi SirDice 

I'm running the app as root


----------



## SirDice (Nov 13, 2021)

You mentioned you're running this in a jail? Did you set devfs.rules correctly? Many devices are hidden away in a jail.


----------



## jjachuf (Nov 13, 2021)

SirDice said:


> You mentioned you're running this in a jail? Did you set devfs.rules correctly? Many devices are hidden away in a jail.


Thanks SirDice . Yes, that would be for the deployment, but now in development is running directly on the "host".
In the explanation I put in PySerial [1] I show that I can read and write in the first cycle, but when it is executed for the second time it gives error. Maybe it's a limitation of the PySerial itself
[1] https://github.com/pyserial/pyserial/discussions/622


----------



## ralphbsz (Nov 13, 2021)

In general, serial port access works fine. I use it several places on my machines, both FreeBSD and Linux, both from root and from user accounts, both in C and in Python (with pyserial). Common problems include permissions (see SirDice's message), but you're running as root so that should be OK.



jjachuf said:


> ... it gives error.


We can't help debug "error". There are zillions of possible errors. You need to tell us exactly what the error is, and what causes the error.

I would also suggest beginning with a super simple test case: Take a wire, and bridge pins 2 and 3 of the serial port connector on the computer, leaving all other pins unconnected (doesn't matter whether this is a 9-pin or 25-pin connector). That bridges TxD and RxD, meaning any data you transmit will be immediately received. Set the port for any baud rate, disable hardware flow control, and run a simple software test that you get data transmitted back again. The pyserial installation comes with a little terminal emulator program (I think it's called minicom something) which you can use for testing.


----------



## Tieks (Nov 13, 2021)

The second call to _reset_input_buffer()_ causes the problem. I never used pyserial, but it occurs to me that you may not need that call at all. Just try to run without it. 
If there is a need to explicitly empty that buffer at that point, you might take a look at _flushInput()_ instead.


----------



## jjachuf (Nov 13, 2021)

ralphbsz said:


> In general, serial port access works fine. I use it several places on my machines, both FreeBSD and Linux, both from root and from user accounts, both in C and in Python (with pyserial). Common problems include permissions (see SirDice's message), but you're running as root so that should be OK.
> 
> 
> We can't help debug "error". There are zillions of possible errors. You need to tell us exactly what the error is, and what causes the error.



Hi ralphbsz
Yes, so as not to repeat I had put what I mentioned in the PySerial repository: https://github.com/pyserial/pyserial/discussions/622


```
ser = serial.Serial('/dev/cuaU0')
while True:   
    ser.reset_input_buffer()
    ser_bytes = ser.readline()
    decoded_bytes = ser_bytes[0:len(ser_bytes) - 2].decode("utf-8")

    print('Bytes:', ser_bytes)
    print('UTF-8:', decoded_bytes)
  
    resp = ser.write(b'v')
    print('Resp:', resp)
    sleep(2)
```

The error is given when cleaning the buffer. The first cycle in the loop works fine and when he tries the heavy failure.

```
File "/home/jose/desarrollos/fav/desarollo/venv/lib/python3.8/site-packages/serial/serialposix.py", line 677, in _reset_input_buffer    termios.tcflush(self.fd, termios.TCIFLUSH) termios.error: (6, 'Device not configured')
```
If I do not run the ser.reset_input_buffer() line the same, execute the first cycle and in the second one gives another error

```
raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)
```


The next portion of code works fine. Every time a data is sent from Arduino, I can see it by console and it is not necessary run  ser_bytes = ser.readline()

```
ser = serial.Serial('/dev/cuaU0')
while True:
    ser_bytes = ser.readline()
    decoded_bytes = ser_bytes[0:len(ser_bytes) - 2].decode("utf-8")
    print(decoded_bytes)   
    sleep(2)
```

I have the problem when I read data and, based on these data, I write in the Arduino





ralphbsz said:


> I would also suggest beginning with a super simple test case: Take a wire, and bridge pins 2 and 3 of the serial port connector on the computer, leaving all other pins unconnected (doesn't matter whether this is a 9-pin or 25-pin connector). That bridges TxD and RxD, meaning any data you transmit will be immediately received. Set the port for any baud rate, disable hardware flow control, and run a simple software test that you get data transmitted back again. The pyserial installation comes with a little terminal emulator program (I think it's called minicom something) which you can use for testing.


Ok I will see

Thanks for your answer


----------



## jjachuf (Nov 13, 2021)

Tieks said:


> The second call to _reset_input_buffer()_ causes the problem. I never used pyserial, but it occurs to me that you may not need that call at all. Just try to run without it.
> If there is a need to explicitly empty that buffer at that point, you might take a look at _flushInput()_ instead.


Hi Tieks 

I answered this to ralphbsz , it is necessary to clean the buffer because (in Linux and Windows) there is always the last reading and does not stop to wait for the Arduino to send you data. And if I do not run it, it error:
_And about flushInput():_
*reset_input_buffer()*
Flush input buffer, discarding all its contents.
Changed in version 3.0: renamed from flushInput()



> raise SerialException( serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)


----------



## jjachuf (Nov 14, 2021)

Seeing the PySerial code has differences according to the OS. Then I tried with Linuxlator but I get the same result as native to FreeBSD


----------



## ralphbsz (Nov 14, 2021)

Let's untangle this. You are getting two separate errors. If you call reset_input_buffer() on every iteration of the loop, then the second time around, it gives you "device not configured". That is nothing but the error string corresponding to errno ENXIO, which literally means: you are trying to do something to a device that does not exist, or you are asking a device to do something it is not capable of. Common examples include: You open a device by name (in your case /dev/cuaU0), but by the time the call comes around, the USB service has disconnected it, and it no longer actually exists. There are other causes, such as sending a device an ioctl that it can never execute, such as sending a SCSI disk the command to "eject page" (which is a good command for SCSI printers, but disks don't do that), or sending a device an ioctl that it temporarily impossible (like asking a tape drive to rewind the tape, but no tape is physically present). Now, I have no idea why calling reset_input_buffer() would cause that, other than that the device has been disconnected. Suggestion: Look in your log for disconnection events. Maybe add to your python code to do an "ls -l /dev/cuaU0" and "stty -a < /dev/cuaU0", to see that the device is in good shape.

Second error: If you don't use reset_input_buffer(), you get "device reports readiness to read but returned no data". That's not an OS error, it's a pyserial internal error. It happens when you have to programs using the same serial port, and both are reading from the same input (which happens regularly to me, if I forget to stop another program). And it can happen if the device is disconnected while you are using it.

My suspicion is that debugging device disconnects might help.


----------



## Tieks (Nov 14, 2021)

jjachuf said:


> it is necessary to clean the buffer because (in Linux and Windows) there is always the last reading and does not stop to wait for the Arduino to send you data.


So you need to run this code on Windows, Linux and FreeBSD, is that correct? 
It seems to me that _ser.readline()_ on FreeBSD behaves differently. On FreeBSD it stops executing until there is for data from the other side, whereas in Windows and Linux it keeps executing without returning data. Is that right?


----------



## jjachuf (Nov 14, 2021)

Tieks said:


> So you need to run this code on Windows, Linux and FreeBSD, is that correct?



No, I need it in FreeBSD, but if it does not work I'll go for Linux. I tried in Linux and Windows to rule out that it is my code, both work fine.


Tieks said:


> It seems to me that _ser.readline()_ on FreeBSD behaves differently. On FreeBSD it stops executing until there is for data from the other side, whereas in Windows and Linux it keeps executing without returning data. Is that right?


If it's like you say, in Linux and Windows, the last data is in the buffer and therefore considers it a new reading, as if it came from the arduino


----------



## jjachuf (Nov 14, 2021)

ralphbsz said:


> Let's untangle this. You are getting two separate errors. If you call reset_input_buffer() on every iteration of the loop, then the second time around, it gives you "device not configured". That is nothing but the error string corresponding to errno ENXIO, which literally means: you are trying to do something to a device that does not exist, or you are asking a device to do something it is not capable of. Common examples include: You open a device by name (in your case /dev/cuaU0), but by the time the call comes around, the USB service has disconnected it, and it no longer actually exists. There are other causes, such as sending a device an ioctl that it can never execute, such as sending a SCSI disk the command to "eject page" (which is a good command for SCSI printers, but disks don't do that), or sending a device an ioctl that it temporarily impossible (like asking a tape drive to rewind the tape, but no tape is physically present). Now, I have no idea why calling reset_input_buffer() would cause that, other than that the device has been disconnected. Suggestion: Look in your log for disconnection events. Maybe add to your python code to do an "ls -l /dev/cuaU0" and "stty -a < /dev/cuaU0", to see that the device is in good shape.



Now I'm going to try what you say. Something I noticed is that after error the device /dev/cuaU0 does not exist and now has the name /dev/cuaU1.  Then I have to disconnect from the USB and reconnect me so that it will be /dev/cuaU0 again



ralphbsz said:


> Second error: If you don't use reset_input_buffer(), you get "device reports readiness to read but returned no data". That's not an OS error, it's a pyserial internal error. It happens when you have to programs using the same serial port, and both are reading from the same input (which happens regularly to me, if I forget to stop another program). And it can happen if the device is disconnected while you are using it.
> 
> My suspicion is that debugging device disconnects might help.


----------



## jjachuf (Nov 14, 2021)

without
ser.reset_input_buffer()



> ==================================================
> 
> 
> iteration 1
> ...



with
ser.reset_input_buffer()



> ==================================================
> 
> 
> iteration 1
> ...



In both cases in the iteration 2 appears /dev/cuaU1


----------



## jjachuf (Nov 14, 2021)

with
ser.reset_input_buffer()

and  when I do not write about the Arduino
*I do not run the line:*
resp = ser.write('v'.encode('ascii'))



> ==================================================
> 
> 
> iteration 1
> ...



*The port /dev/cuaU1 is not created*
for some reason when I write the second port is created
Jose


----------



## ralphbsz (Nov 14, 2021)

Look at the dmesg output to maybe get an idea *why* the device is disconnected.

My guess would be: insufficient power connection.


----------



## jjachuf (Nov 15, 2021)

ralphbsz said:


> Look at the dmesg output to maybe get an idea *why* the device is disconnected.
> 
> My guess would be: insufficient power connection.



This is before run [resp = ser.write('v'.encode('ascii'))]



> jose@lenovo:~ % dmesg | grep uchcom
> uchcom0 on uhub0
> uchcom0: <vendor 0x1a86 USB2.0-Ser, rev 1.10/2.54, addr 4> on usbus0
> uchcom0: CH340 detected




and this after run 



> jose@lenovo:~ % dmesg | grep uchcom
> uchcom0 on uhub0
> uchcom0: <vendor 0x1a86 USB2.0-Ser, rev 1.10/2.54, addr 4> on usbus0
> uchcom0: CH340 detected
> ...


I do not see the reason


----------



## jjachuf (Nov 15, 2021)

I contribute something else, then that the app fails and stops, if I do an ls in dev obtain only cuaU1. cuaU0 disappeared



> # ls -l /dev/cuaU*
> crw-rw----  1 uucp  dialer  0x1d2 15 nov.  17:50 /dev/cuaU1
> crw-rw----  1 uucp  dialer  0x1d3 15 nov.  17:50 /dev/cuaU1.init
> crw-rw----  1 uucp  dialer  0x1d4 15 nov.  17:50 /dev/cuaU1.lock





> # usbconfig
> ugen0.1: <0x8086 XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
> ugen0.2: <Logitech USB Receiver> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (98mA)
> ugen0.3: <vendor 0x8087 product 0x0a2a> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
> ...


----------



## ralphbsz (Nov 15, 2021)

jjachuf said:


> I do not see the reason


But it is clear what happens: When running your program, it causes cua0 to be disconnected, and then immediately reconnect (see below). Now, I agree with you: no idea why this happens, but it is clear that it does happen.

Debugging the why is going to be interesting. I would start by making sure you are not using flaky cables and connectors, nor cables that are too small. I've seen lots of weird USB problems, which are caused by bad connections, in particular on the power lines.

Also: Many USB to serial adapters use FTDI chips. Those chips are often cloned in China, usually badly. Cheap Chinese serial adapter clones cause all manners of funny results. What I do now is to throw any serial adaptor that doesn't work in the trash, and replace them with ones from reputable sources (such as Adafruit). This could be part of your problem, but it doesn't have to be.



jjachuf said:


> I contribute something else, then that the app fails and stops, if I do an ls in dev obtain only cuaU1. cuaU0 disappeared


That makes perfect sense: Clearly the USB system can not create two devices with the same name; there can't be two /dev/cuaU0 that refer to different chips. To prevent confusion, the system will also not create a new /dev/cuaU0 for a small timeout period after the old one has been removed. So if a chip has been removed, and another chip inserted, it makes sense to give the second one a different name.

This is a symptom of a much more generic design problem: USB devices are not always discovered in the same order, so their device names can be unpredictable. You can't rely on one particular device always being /dev/cuaU0, if other USB serial things may get attached. What I do in my code is that when looking for a specific piece of hardware, I iterate over all /dev/cuaU* things, see whether they are already in use (if yes, leave them alone), if not ask them for some individual characteristics, and then use the one that matches.


----------



## jjachuf (Nov 16, 2021)

Hi ralphbsz


ralphbsz said:


> But it is clear what happens: When running your program, it causes cua0 to be disconnected, and then immediately reconnect (see below). Now, I agree with you: no idea why this happens, but it is clear that it does happen.


I agree with you. I mean it's detached, but I do not see the reason.




ralphbsz said:


> Debugging the why is going to be interesting. I would start by making sure you are not using flaky cables and connectors, nor cables that are too small. I've seen lots of weird USB problems, which are caused by bad connections, in particular on the power lines.
> 
> Also: Many USB to serial adapters use FTDI chips. Those chips are often cloned in China, usually badly. Cheap Chinese serial adapter clones cause all manners of funny results. What I do now is to throw any serial adaptor that doesn't work in the trash, and replace them with ones from reputable sources (such as Adafruit). This could be part of your problem, but it doesn't have to be.


Some notes about this:
1) In the same laptop (and same usb port) with Linux works fine. I was wondering if the power handling in the USB port does SO or it is something of the Motherboard itself
2) This week my customer will bring me another device to replace the one I'm using

You have added or changed something with sysctl or in sysctl.conf?


----------



## ralphbsz (Nov 16, 2021)

jjachuf said:


> Some notes about this:
> 1) In the same laptop (and same usb port) with Linux works fine.


Some power management of USB ports can be done in software, but not on all USB ports. Old traditional USB2.0 ports had Gnd and +5V hardwired to the computer supply. Modern USB-C ports have very complex power management. And in the middle is a big grey zone.



> You have added or changed something with sysctl or in sysctl.conf?


No, completely stock FreeBSD 12.2, and USB serial hasn't changed behavior for me since version 8 or 9.


----------



## jjachuf (Nov 16, 2021)

ralphbsz In your app you do reading and writing on the device?


----------



## msplsh (Nov 16, 2021)

So, you're getting ENXIO when termios uses ioctl to send TIOCFLUSH to the descriptor...  Digging through the source, I think you need to take heart the comment in the source for this driver, uchcom.c

"Driver for WinChipHead CH341/340, the worst USB-serial chip in the world"


----------



## ralphbsz (Nov 16, 2021)

jjachuf said:


> ralphbsz In your app you do reading and writing on the device?


Yes, and there is more than one program. I talk to Omega industrial controllers (which have the USB chip built in), to Weeder data acquisition boards, to a HAI UPB PIM, and finally to a Dell modem. The last one is read only in production (I use it just to monitor caller ID), but I've written to it to configure it, by hand with a program like minicom.

The only problem I see once bad USB adapters are eliminated is "device reports readiness to read but returned no data". This happens if two programs are accessing the same serial device at the same time. In most cases, I have eliminated this possibility (by locking the devices so they can only be used by one reader/writer), but I haven't had time to clean up the UPB PIM software, so it occasionally happens.


----------



## kpedersen (Nov 16, 2021)

When testing new RS232 devices, I tend to short RX and TX and then use `cu` or some code I wrote a while back to see if bytes get through. They all seem to say they use some random Prolific chip but none of them are really the same, often some are dead.

But you say that you are experiencing the device vanishing when using it with FreeBSD? There isn't too much I can recommend for that. Perhaps try it in VirtualBox running on FreeBSD and use the USBv1.1 passthrough? Luckily v1.1 won't be the bottleneck in this case.

It is strange that it works with Linux but not FreeBSD. Possibly it could be a power thing but usb / serial adapters specifically do tend to use very little.


----------



## bobmc (Nov 17, 2021)

I tried "#picocom -b 115200 /dev/cuaU0" and that worked for a randomly selected USB port which was connected to a MEGA2560 (extended Arduino) printing DS18B20 sensor data. It might be helpful to check picocom.c which uses open() to bind the USB serial port. It's 2200 lines of code.

I use the mega2560 because it has 3 UART channels. Most Arduino variants have UART for control and no free UART for application.


----------



## jjachuf (Nov 17, 2021)

Thanks everyone for your help. I found this [1] that avoids the error I reported. So it says this setting is by default in windows and linux

*sysctl hw.usb.no_cs_fail=1*
Now I'm going to try with the full app to see that everything works fine, but with the portion of code that came testing no longer the error occurs

[1] https://lists.freebsd.org/pipermail/freebsd-stable/2021-February/093236.html


----------



## kpedersen (Nov 17, 2021)

jjachuf said:


> Thanks everyone for your help. I found this [1] that avoids the error I reported.


Wow, good spot. I now have a box full of previously suspected broken adapters to now check with this!


----------



## jjachuf (Nov 17, 2021)

kpedersen said:


> Wow, good spot. I now have a box full of previously suspected broken adapters to now check with this!


kpedersen Then you tell us how it turned out


----------



## jjachuf (Dec 30, 2021)

Hello, I wanted to add some notes to the thread. They brought me another device, same features, and works fine, without needing to use the modification with *sysctl*
Some problem had the first one I used


----------

