Termux (Android) support #285
Description
Activity
This seems reasonable.
Besides ctypes bindings for the
libusb_wrap_sys_device(added in libusb 1.0.23) andlibusb_get_deviceAPIs, I think we should have a PyUSB interface that abstracts these details.Of the top of my mind I'm not sure where to place or how to name this interface. But are you willing to give it a shot and send me a patch/open a pull request?
Archiving: original discussion on the libusb mailing list.
Reacted by Andrzej RaczkowskiCan do.
But I was also not sure where to place this, as it is a different approach to the standard enumerate devices → select device to work with sequence.
Would this be good:
usb.core.[device_]from_fd(fd: int) -> usb.core.Device+extend libusb1 (other backends are not on Android)
I am also not sure whether this is only for Android or if we can natively get USB file descriptors on other systems from other programs? to work with...
I have to check later but I think
dev.set_configuration()did not work for me on Termux.Do we need more detailed lib functions?
class _LibUSB(usb.backend.IBackend): # ... # fd -> handle @methodtrace(_logger) def wrap_sys_device(self, sys_dev): handle = _libusb_device_handle() _check(self.lib.libusb_wrap_sys_device(self.ctx, sys_dev, byref(handle))) return handle # handle -> devid @methodtrace(_logger) def get_device(self, handle): return self.lib.libusb_get_device(handle)
Or how it most probably will only be used?
class _FileDescriptorDeviceHandle(object): def __init__(self, fd, ctx): # get handle from file descriptor self.handle = _libusb_device_handle() _check(_lib.libusb_wrap_sys_device(ctx, fd, byref(self.handle))) # get device (id?) from handle self.devid = _lib.libusb_get_device(self.handle) class _LibUSB(usb.backend.IBackend): # ... # fd --> dev (like _DeviceHandle) @methodtrace(_logger) def get_device_from_fd(self, fd): return _FileDescriptorDeviceHandle(fd, self.ctx)
I have not found a mention about the need to free the handles etc. Android libusb1 doc only mentions the creation of a temporary device...
Working on it: master...Querela:master
On my device:
git clone https://github.com/Querela/pyusb.git && cd pyusb python3 -m venv venv source venv/bin/activate pip install -e . termux-usb -l termux-usb -r -e ./tools/termux_device_info.py /dev/bus/usb/001/002
DEVICE ID 10c4:ea60 on Bus 001 Address 002 ================= bLength : 0x12 (18 bytes) bDescriptorType : 0x1 Device bcdUSB : 0x110 USB 1.1 bDeviceClass : 0x0 Specified at interface bDeviceSubClass : 0x0 bDeviceProtocol : 0x0 bMaxPacketSize0 : 0x40 (64 bytes) idVendor : 0x10c4 idProduct : 0xea60 bcdDevice : 0x100 Device 1.0 iManufacturer : 0x1 Silicon Labs iProduct : 0x2 CP2102 USB to UART Bridge Controller iSerialNumber : 0x3 0001 bNumConfigurations : 0x1 CONFIGURATION 1: 100 mA ================================== bLength : 0x9 (9 bytes) bDescriptorType : 0x2 Configuration wTotalLength : 0x20 (32 bytes) bNumInterfaces : 0x1 bConfigurationValue : 0x1 iConfiguration : 0x0 bmAttributes : 0x80 Bus Powered bMaxPower : 0x32 (100 mA) INTERFACE 0: Vendor Specific =========================== bLength : 0x9 (9 bytes) bDescriptorType : 0x4 Interface bInterfaceNumber : 0x0 bAlternateSetting : 0x0 bNumEndpoints : 0x2 bInterfaceClass : 0xff Vendor Specific bInterfaceSubClass : 0x0 bInterfaceProtocol : 0x0 iInterface : 0x2 CP2102 USB to UART Bridge Controller ENDPOINT 0x81: Bulk IN =============================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x81 IN bmAttributes : 0x2 Bulk wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x0 ENDPOINT 0x1: Bulk OUT =============================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x1 OUT bmAttributes : 0x2 Bulk wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x0Last time I tested Termux and it does work with libusb (requires root) on my Android TV boxes. So yes it will be nice to see pyusb working under Termux.
The output above works without root on my device (under Android 10).
I wrote a simple serial (uart?) wrapper around this and I can catch serial usb2ttl output, e. g. from my DSO138mini. gh:Querela/termux-usb-pythonGreat. I will try this when I have some time, probably over the weekend.
Just notice that libusb was moved from termux-root-packages to the standard packages repo three months ago.
https://github.com/termux/termux-packages/tree/master/packages/libusbYes, I saw it only afterwards but I kind of remember that I saw the Termux issue some time ago when it was not yet implemented.
(With root this sys_device wrapping may not even be neccessary but as non-root user, we can only request a USB file descriptor from Android.)Experiments etc.:
pySerial doesn't work for me because it requires device paths/names to open the USB descriptor. Maybe I can patch it up so that it will work with PyUSB but it natively tries to use hidapi. (Since the device handles should be the same, it may be interchangeable? I'm not sure.)
I also have a problem where programs try to set termios controls/settings and that will not work with the PyUSB handle. My research was also not successful, so I'm not sure how to do this. One user in the Termux issue (?) suggestedremserialbut I kind of doubt it works. Andsocatwas not that newbie friendly ...
Since I wrote my own UART (?) serial with PyUSB + my patch, I know it works but I'm stuck on getting the Android USB FD to work with other tools ...😊 oops nm, I found it in your PR.
Also note that if termux/termux-api#349 is fixed then no special handling for termux will be required in pyusb as the termux build of libusb will handle everything for you.
Reacted by Rupak Kumar Yadav8 remaining items
@normanr You are of course right. I think libusb/libusb#830 and libusb/libusb#874 are the right direction for libusb Android support. Please give them a try. The method in libusb pull request 830 is already usable now (it is just a README updates with an example).
As for this particular patch, it covers a niche use cases for Termux users. I am actually using Termux myself and I do not really need to use the normal Android libusb support at all as I do now know Java/Android development myself and have no interests to know about either of them. (I am not a programmer myself but I know a little bit about C and Python).
Unfortunately I tried https://github.com/Querela/pyusb/commits/master and it does not seem to work under my Huawei P20 running Android 10.
The following two commands failed (hang). I simply do not have access to "/dev/bus/usb/" directory.
termux-usb -l termux-usb -r -e ./tools/termux_device_info.py /dev/bus/usb/001/002If I do not use the above two commands, it still failed.
~ $ git clone https://github.com/Querela/pyusb pyusb_termux Cloning into 'pyusb_termux'... remote: Enumerating objects: 2207, done. remote: Total 2207 (delta 0), reused 0 (delta 0), pack-reused 2207 Receiving objects: 100% (2207/2207), 1.02 MiB | 5.07 MiB/s, done. Resolving deltas: 100% (1281/1281), done. ~ $ cd pyusb_termux/ ~/pyusb_termux $ python3 -m venv venv ~/pyusb_termux $ source venv/bin/activate (venv) ~/pyusb_termux $ pip install -e . Obtaining file:///data/data/com.termux/files/home/pyusb_termux Installing collected packages: pyusb Running setup.py develop for pyusb Successfully installed pyusb-1.0.2 WARNING: You are using pip version 21.1.1; however, version 21.1.2 is available. You should consider upgrading via the '/data/data/com.termux/files/home/pyusb_termux/venv/bin/python3 -m pip install --upgrade pip' command. (venv) ~/pyusb_termux $ python ./tools/termux_device_info.py Traceback (most recent call last): File "/data/data/com.termux/files/home/pyusb_termux/./tools/termux_device_info.py", line 38, in <module> fd = int(sys.argv[1]) IndexError: list index out of rangeI tried my Mi Box S Android TV box running Android TV OS 9.0 as well and "termux-usb -l" just does not list anything. So it seems to me that I do not have the right Android device to carry out the test.
Then I used another Android 7 Box with root option (H96 Max RK3328 based box, need to run as root to have the write permissions), plain libusb/pyusb work as expected. However, termux-usb or tools/termux_device_info.py (from https://github.com/Querela/pyusb/blob/master/tools/termux_device_info.py) does not work.
The error message is the same with root or without root.
(venv) ~/pyusb_termux $ python ./tools/termux_device_info.py /dev/bus/usb/005/007 Traceback (most recent call last): File "/data/data/com.termux/files/home/pyusb_termux/./tools/termux_device_info.py", line 38, in <module> fd = int(sys.argv[1]) IndexError: invalid literal for int() with base 10: /dev/bus/usb/005/007All in all, it seems to me for #287 to work, "termux-usb -l" should work and it seems to me that means the right permission and existence of "/dev/bus/usb" which does not seem to be the case for my Android devices (non rooted).
And this bridge seems to rely on a libusb fork. Technically you do not need to use a libusb fork as the latest libusb version should have the stuff in place already.
https://github.com/normanr/termux-libusb-bridge/blob/main/termux-libusb-bridge.py
termux/termux-api#349But I think fundamentaly right now libusb needs a user provided helper (java) to work under Android non rooted device.
libusb/libusb#830The following libusb pull request is supposed to get rid of that user helper.
libusb/libusb#874termux_device_info.py accepts a file descriptor, not a path. When it's run via termux-usb the -e option will execute the specified command with a file descriptor referring to the device as its argument. You can not see the files because selinux does not allow it, only the java api's have access to the usb devices (and can be used open and return a file descriptor for them).
So if you want to test without termux-usb (like on a non-android device), you need to do something like:
./termux_device_info.py 9 9</dev/bus/usb/005/007(i.e. use a shell redirect to open the usb device and pass the fd number to the process). You can see the effect by runningtermux-usb -e echo /dev/bus/usb/005/007and seeing that it prints out a file descriptor number.The latest libusb does not have the right-stuff in place yet (as far as I know) to support opening usb devices on non-rooted devices without having to change the source code of the binary that you're using. #287 has the same problem (the source code of the binary needs to be changed to explicitly support reading the fd from the command line and opening it and giving it to libusb).
libusb/libusb#874 doesn't get rid of the helper it just proposes to add some of the java glue required to get libusb working with non-rooted devices into libusb itself. It still requires the program that's opening the usb device to link to java and call the right init functions to make everything work. That only works for programs you have the source code to, and you're willing to change to make it work. The nice thing about AnotherTerm's helper is that it "just-works" for existing binaries.
It could be nice to expose the wrap_sys_device API in pyusb, but it does require code changes to make use of it. I have some pending changes in a working tree, mostly the suggestions I made on #287, if you want I can look at committing and pushing them somewhere (not as a full PR, but so that they're not 100% lost).
Reacted by mcuee and Andrzej RaczkowskiIt could be nice to expose the wrap_sys_device API in pyusb, but it does require code changes to make use of it. I have some pending changes in a working tree, mostly the suggestions I made on #287, if you want I can look at committing and pushing them somewhere (not as a full PR, but so that they're not 100% lost).
@normanr
Yes that would be good so that your work is not lost. Thanks.Not so sure if the patched libusb version will help. I will try it under Termux and unrooted Android.
Ref: https://github.com/green-green-avk/AnotherTerm-scripts/blob/master/install-patched-libusb-1.0.23.sh
On my device:
git clone https://github.com/Querela/pyusb.git && cd pyusb python3 -m venv venv source venv/bin/activate pip install -e . termux-usb -l termux-usb -r -e ./tools/termux_device_info.py /dev/bus/usb/001/002
DEVICE ID 10c4:ea60 on Bus 001 Address 002 ================= bLength : 0x12 (18 bytes) bDescriptorType : 0x1 Device bcdUSB : 0x110 USB 1.1 bDeviceClass : 0x0 Specified at interface bDeviceSubClass : 0x0 bDeviceProtocol : 0x0 bMaxPacketSize0 : 0x40 (64 bytes) idVendor : 0x10c4 idProduct : 0xea60 bcdDevice : 0x100 Device 1.0 iManufacturer : 0x1 Silicon Labs iProduct : 0x2 CP2102 USB to UART Bridge Controller iSerialNumber : 0x3 0001 bNumConfigurations : 0x1 CONFIGURATION 1: 100 mA ================================== bLength : 0x9 (9 bytes) bDescriptorType : 0x2 Configuration wTotalLength : 0x20 (32 bytes) bNumInterfaces : 0x1 bConfigurationValue : 0x1 iConfiguration : 0x0 bmAttributes : 0x80 Bus Powered bMaxPower : 0x32 (100 mA) INTERFACE 0: Vendor Specific =========================== bLength : 0x9 (9 bytes) bDescriptorType : 0x4 Interface bInterfaceNumber : 0x0 bAlternateSetting : 0x0 bNumEndpoints : 0x2 bInterfaceClass : 0xff Vendor Specific bInterfaceSubClass : 0x0 bInterfaceProtocol : 0x0 iInterface : 0x2 CP2102 USB to UART Bridge Controller ENDPOINT 0x81: Bulk IN =============================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x81 IN bmAttributes : 0x2 Bulk wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x0 ENDPOINT 0x1: Bulk OUT =============================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x1 OUT bmAttributes : 0x2 Bulk wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x0Trying this, I get:
Traceback (most recent call last): File "/data/data/com.termux/files/home/source/pyusb/./tools/termux_device_info.py", line 31, in <module> main(fd) File "/data/data/com.termux/files/home/source/pyusb/./tools/termux_device_info.py", line 19, in main device = usb.core.device_from_fd(fd) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/data/com.termux/files/home/source/pyusb/usb/core.py", line 1298, in device_from_fd raise NoBackendError('No backend available') usb.core.NoBackendError: No backend availabletermux-usb -lgives a list with/dev/bus/usb/001/002as expected.Any idea why
import usb.backend.libusb1 as libusb1; libusb1.get_backend()would be returning None? I tried with bothpkg install libusbwhich is currently at version1.0.26-1and uninstalling that and usingpip install libusb==1.0.23b7as that's the version mentioned near the start of this issue, but neither seem to provide a backend that's not None.Any ideas?
Also were there any blockers for #287?
You may need to put libusb-1.0.0.so in the right place in order for pyusb to work. Termux does not use the default location like
/usr/local/libor/usr/lib.Also were there any blockers for #287?
I do not think #287 is ready. Please try it and post your test results. By right it should work with stock libusb-1.0.26 from termux and not a third party fork of libusb-1.0.23.
And I think the right way to use libusb under Android is using the documented ways from libusb. I am not so sure how it can be adopted by Termux though as I do not know much about Android.
https://github.com/libusb/libusb/tree/master/android/examplesThere is also an on-going improvement for libusb under Android.
Reacted by Adam Suban-LoewenYou may need to put libusb-1.0.0.so in the right place in order for pyusb to work. Termux does not use the default location like
/usr/local/libor/usr/lib.Thanks, yeah I tried stuff like https://stackoverflow.com/a/71102974/2561747 to specify the location to no avail.
But compiling stock libusb-1.0.23 worked, following this procedure termux/termux-packages#1714 (comment) but using the official release for version 1.0.23. I guess there may have been a backwards-incompatible change or regression since that version. But now the PR works and I can print device info, and read and write small bits of data.
I keep getting timeout errors when writing large amounts of data (even with very long timeouts), but I'm not sure if that's from the pyusb fork or the application code.
Hi,
In Termux you only receive a USB file descriptor (number bound to single process).
Any operations like
usb.core.find(..)won't work.I wrote a workaround/fix that extends the libusb1 interface and provides the
Deviceobject given the file descriptor.See here: https://github.com/Querela/termux-usb-python/blob/e6166ded4555d988365e6609b91dec4f5ebb8fcd/usblib.py#L94
I only tested it on the most current Android (10) and Termux version ...
Usage examples are in my Repo.
Can we implement this in your library?
Enumerating devices would maybe work with subprocesses but working with a specific device is only possible in a process(?) callback and does not seem feasable. I guess ..?