upydev
Command line tool for MicroPython devices
uPydev is an acronym of ‘MicroPython device’, and it is intended to be a command line tool to make easier the development, prototyping and testing process of devices based on boards running MicroPython. It is intended to be cross-platform and connection agnostic (Serial, WiFi and Bluetooth Low Energy).
Lincense: MIT
Documentation: https://upydev.readthedocs.io.
upydev
Command line tool
uPydev is an acronym of ‘MicroPython device’, and it is intended to be a command line tool to make easier the development, prototyping and testing process of devices based on boards running MicroPython. It is intended to be cross-platform and connection agnostic (Serial, WiFi and Bluetooth Low Energy).
Lincense: MIT
Documentation: https://upydev.readthedocs.io.
Features
Tools to allow configuration, management, communication and control of MicroPython devices
Command line Autocompletion
File IO operations (upload, download one or multiple files, recursively sync directories…)
SHELL-REPL: Serial, WiFi (WebREPL/WebSecureREPL) and Bluetooth Low Energy
OTA* Firmware updates WiFi (TCP/SSL), BLE (*esp32 only)
Custom commands for debugging, testing and prototyping
Custom tasks yaml files that can be played like ansible
Run tests in device with pytest and parametric tests or benchmarks using yaml files
Group mode to operate with multiple devices
Installing
Install upydev by running:
$ pip install upydev
To update to the latest version available:
$ pip install --upgrade upydev
To get development version:
$ pip install https://github.com/Carglglz/upydev/tree/develop.zip
To get help, use h or help command :
$ upydev help
Or see help about a specific command $ upydev [COMMAND] -h:
$ upydev put -h
usage: put [-h] [-dir DIR] [-rst] file/pattern/dir [file/pattern/dir ...]
upload files to device
positional arguments:
file/pattern/dir indicate a file/pattern/dir to upload
optional arguments:
-h, --help show this help message and exit
-dir DIR path to upload to
-rst to soft reset after upload
Getting started
If using a device with no MicroPython preinstalled (esp8266, esp32) first follow MicroPython getting started from official docs to see how to install it for the first time.
After that the device should be up and running for the next step.
Requirement
- Needs REPL to be accessible:
- > Wireless Devices:
WiFi: Needs WebREPL enabled in the device see WebREPL: a prompt over-wifi and WebREPL: web-browser interactive prompt
Bluetooth Low Energy: Needs BleREPL enabled in the device. 1
- > Serial Devices:
USB: Connected through USB data cable.
- 1
This is still experimental and for esp32 requires
ble_advertising.py,ble_uart_peripheral.py,ble_uart_repl.pyto be uploaded to the device. These scripts can be found in upyutils directory and they come from micropython examples. Finally to enable it add the following tomain.py:import ble_uart_repl ble_uart_repl.start()
Create a configuration file
Save the address and password/baudrate of a connected device so this won’t be required for future interaction.
Note
If device address is unknown use
$ upydev scan [OPTION]whereOPTIONcan be-srfor SERIAL,-ntfor WiFi or-blfor Bluetooth Low Energy.
e.g.
$ upydev scan -sr Serial Scan: SerialDevice/s found: 2 ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ PORT ┃ DESCRIPTION ┃ MANUFACTURER ┃ ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ /dev/cu.usbmodem387E386731342 ┃ Pyboard Virtual Comm Port in FS Mode ┃ MicroPython ┃ ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ /dev/cu.SLAB_USBtoUART ┃ CP2104 USB to UART Bridge Controller ┃ Silicon Labs ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Upydev will use local working directory configuration unless it does not find any or manually indicated with -g option which is the global configuration flag.
To save configuration in working directory:
$ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE]
where ADDRESS must be a valid :
- 2
IP can be a valid dhcp_hostname e.g.
esp_dev.local(must be set with e.g.nic.config(hostname="esp_dev"))- 3
-pis set to 115200 by default, so it is not necessary unless using a different baudrate- 4
It will depend on OS system (e.g. Linux uses MAC format ‘XX:XX:XX:XX:XX:XX’, and macOS uses UUID format ‘XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX’)
Default device name is upydevice, to set a custom name use -@ flag as
$ upydev config -t 192.168.1.53 -p mypass -@ mycustomdevice
To check configuration upydev or upydev check
$ upydev Device: mycustomdevice Address: 192.168.1.53, Device Type: WebSocketDevice
Or to get more information if the device is online
$ upydev -i Device: mycustomdevice WebSocketDevice @ ws://192.168.1.53:8266, Type: esp32, Class: WebSocketDevice Firmware: MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32 (MAC: 30:ae:a4:23:35:64, RSSI: -45 dBm)
To save configuration globally use
-gflag:$ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE] -ge.g.
$ upydev config -t 192.168.1.53 -p mypass -g
To save configuration in a global group use
-ggflag:$ upydev config -t [DEVICE ADDRESS] -p [PASSWORD/BAUDRATE] -gg -@ mydevicee.g.
$ upydev config -t 192.168.1.53 -p mypass -gg -@ mydevice
[Optional]
Use register command to register a device as a shell function.
This defines the function in ~/.bashrc or ~/.profile
$ upydev register -@ mydevicefunction mydevice() { upydev "$@" -@ mydevice; } function _argcomp_upydev() { _python_argcomplete upydev; } complete -o bashdefault -o default -o nospace -F _argcomp_upydev mydevice$ source ~/.profileNow
mydevicewill accept any args and pass them to upydev, as well as autocompletion of args, e.g.$ mydevice Device: mydevice Address: 192.168.1.53, Device Type: WebSocketDevice
Or if the device is connected. 5
$ mydevice -i Device: mydevice WebSocketDevice @ ws://192.168.1.53:8266, Type: esp32, Class: WebSocketDevice Firmware: MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32 (MAC: 30:ae:a4:23:35:64, Host Name: mydevice, RSSI: -45 dBm)
- 5
Check this using
pingorprobe, e.g.$ mydevice ping PING mydevice.local (192.168.1.53): 56 data bytes 64 bytes from 192.168.1.53: icmp_seq=0 ttl=255 time=5.303 ms 64 bytes from 192.168.1.53: icmp_seq=1 ttl=255 time=218.701 ms 64 bytes from 192.168.1.53: icmp_seq=2 ttl=255 time=39.224 ms 64 bytes from 192.168.1.53: icmp_seq=3 ttl=255 time=62.249 ms ^C --- mydevice.local ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 5.303/81.369/218.701/81.835 ms $ mydevice probe Reaching mydevice... mydevice -> WebSocketDevice @ mydevice.local -> OK [✔]
Finally to enter device shell-repl mode do:
$ upydev shl@mydevice
shell-repl @ mydevice
WebREPL connected
WARNING: ENCRYPTION DISABLED IN THIS MODE
MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
Type help() for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@mydevice:~ $
or if the device is registered
$ mydevice shl
shell-repl @ mydevice
WebSecREPL with TLSv1.2 connected
TLSv1.2 @ ECDHE-ECDSA-AES128-CCM8 - 128 bits Encryption
MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
Type help() for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@mydevice:~ $
Note
To enable WebSocket over TLS or wss check WebSocket (ws) / WebSocket Secure (wss) TLS
Once the device is configured see Usage documentation to check which modes and tools are available.
Or if you are working with more than one device continue with the following section to create a group configuration.
Create a GROUP file
Make a global group of uPy devices named “UPY_G” to enable redirection to a specific device so next time any command can be redirected to any device within the group
Use mkg as $ upydev mkg UPY_G -g -devs [NAME] [ADDRESS] [PASSWORD/BAUDRATE/DUMMY] [NAME2]... 6
to create and add more than one device at once. e.g.
$ upydev mkg UPY_G -g -devs esp_room1 192.168.1.42 mypass esp_room2 192.168.1.54 mypass2
- 6
Every device must have a name, address and password/baudrate/dummy data (in case of ble) so the args can be parsed properly.
or use config and -gg flag as mentioned above to add one device at a time.
$ upydev config -t 192.168.1.42 -p mypass -gg -@ esp_room1
WebSocketDevice esp_room1 settings saved in global group!
$ upydev config -t 192.168.1.54 -p mypass -gg -@ esp_room2
WebSocketDevice esp_room2 settings saved in global group!
To see the devices saved in this global group, use gg.
$ upydev gg GROUP NAME: UPY_G # DEVICES: 2 ┣━ esp_room1 -> WebSocketDevice @ 192.168.1.42 ┗━ esp_room2 -> WebSocketDevice @ 192.168.1.54
Now any command can be redirected to one of these devices with the -@ 7 option :
$ upydev info -@ esp_room1 WebSocketDevice @ ws://192.168.1.42:8266, Type: esp32, Class: WebSocketDevice Firmware: MicroPython v1.12-63-g1c849d63a on 2020-01-14; ESP32 module with ESP32 (MAC: 80:7d:3a:80:9b:30, RSSI: -51 dBm)
- 7
Option
-@has autocompletion on tab so hit tab and see what devices are available
Note
To add or remove devices from this group use mgg, and -gg flag which is the same
as -G UPY_G.
Add
$ upydev mgg -gg -add [NAME] [PASSWORD] [PASSWORD/BAUDRATE/DUMMY] [NAME2]...Remove
$ upydev mgg -gg -rm [NAME] [NAME2]...
[Optional]
Finally use register command to register a group as a shell function.
This defines the function in ~/.bashrc or ~/.profile
$ upydev register devsg -@ pybV1.1 espdev oble
#UPYDEV GROUP devsg
function devsg() { upydev "$@" -@ pybV1.1 espdev oble; }
function _argcomp_upydev() { _python_argcomplete upydev; }
complete -o bashdefault -o default -o nospace -F _argcomp_upydev devsg
$ source ~/.profile
Now devsg will accept any args and pass them to upydev, as well as
autocompletion of args, e.g.
$ devsg
Device: pybV1.1
Address: /dev/tty.usbmodem3370377430372, Device Type: SerialDevice
Device: espdev
Address: espdev.local, Device Type: WebSocketDevice
Device: oble
Address: 00FEFE2D-5983-4D6C-9679-01F732CBA9D9, Device Type: BleDevice
$ devsg -i
Device: pybV1.1
SerialDevice @ /dev/tty.usbmodem3370377430372, Type: pyboard, Class: SerialDevice
Firmware: MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; PYBv1.1 with STM32F405RG
Pyboard Virtual Comm Port in FS Mode, Manufacturer: MicroPython
(MAC: 3c:00:3d:00:02:47:37:30:38:37:33:33)
Device: espdev
WebSocketDevice @ ws://192.168.1.53:8266, Type: esp32, Class: WebSocketDevice
Firmware: MicroPython v1.18-42-g30b6ce86b-dirty on 2022-01-27; ESP32 module with ESP32
(MAC: 30:ae:a4:23:35:64, Host Name: espdev, RSSI: -55 dBm)
Device: oble
BleDevice @ 00FEFE2D-5983-4D6C-9679-01F732CBA9D9, Type: esp32 , Class: BleDevice
Firmware: MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; 4MB/OTA BLE module with ESP32
(MAC: ec:94:cb:54:8e:14, Local Name: oble, RSSI: -50 dBm)
Usage
usage: $ upydev ACTION [options]
Upydev can handle three types of directives:
- An ACTION from Mode / Tools:
Utilities to manage/control a device. e.g.
$ upydev config -t /dev/tty.usbmodem387E386731342 -@ pyblite,$ upydev check,$ upydev put my_script.py
- A predefined MicroPython command:
This always requires a connected device and it translates into MicroPython code snippets. e.g.
$ upydev info,$ upydev mem,$ upydev df
- “Raw” or unregistered commands:
Commands that are not registered in upydev are sent directly to the device as in a REPL. e.g.
$ upydev "led.on()",$ upydev "print('Hello')",$ upydev "import mylib; mylib.dothis()"
Mode/Tools
- > Help
To see help on any mode, tool or command.
ACTIONS:
help,h,dm,fio,fw,kg,rp,sh,db,gp,gc.- > Device Management
To manage configuration of a device/group of devices.
ACTIONS :
config,check,set,register,lsdevs,mkg,mgg,mksg,see,gg- > File IO operations
To upload/download files to/from a device.
ACTIONS:
put,get,dsync,install,update_upyutils- > Firmware
To list, get or flash the firmware of a device.
ACTIONS:
fwr,flash,ota,mpyx- > Keygen
To generate SSL key-certs and random WebREPL passwords.
ACTIONS:
kg rsa,kg wr,kg ssl,rsa sign,rsa verify,rsa auth- > REPL
To enter the REPL.
ACTIONS:
repl,rpl- > SHELL-REPL:
To enter shell-repl.
ACTIONS:
shell,shl,shl-config,set_wss,jupyterc- > Debugging
To debug device connection, run scripts or run interactive test with pytest.
ACTIONS:
ping,probe,scan,run,timeit,stream_test,sysctl,log,pytest setup,pytest,play- > Group Command Mode
To operate with a group of devices.
OPTIONS:
-G,-GP
upy Commands
A set of commands to control or configure the device.
Mode/Tools
Help
ACTIONS:
help,h,dm,fio,fw,kg,rp,sh,db,gp,gc,docs,udocs,mdocs.
- help:
to see help about upydev. (Same as
h, or use-hto see information about optional args too)
- dm:
to see help about device management.
- fio:
to see help on file input/ouput operations.
- fw:
to see help on firmware operations.
- kg:
to see help on keygen operations.
- rp:
to see to see help on REPL modes.
- sh:
to see help on SHELL-REPL modes.
- db:
to see help on debugging operations.
- gp:
to see help on group mode options.
- gc:
to see General commmands help.
Note
To see help about a any ACTION/COMMAND,
use -h after that ACTION/COMMAND as : $ upydev ACTION -h
Note
To see docs about upydev, upydevice or MicroPython use docs, udocs,
mdocs, respectively. Use a keyword as second argument for keyword search.
Device Management
ACTIONS :
config,check,set,register,lsdevs,mkg,mgg,mksg,see,gg
- config:
To save upy device settings (see
-t,-p,-g,-@,-gg), so the target and password arguments wont be required any more. A-ggflag will add the device to the global group (UPY_G) (-ttarget-ppassword-gglobal directory-@device name-ggglobal group)
- check:
To check current device information or with
-@entry point if stored in the global group. Use-iflag if device is online/connected to get more info.
- set:
To set current device configuration from a device saved in the global group with
-@entry point
- register:
To register a device name as a shell function so it can be called from the command line and pass any args to
upydev. This adds the function in~/.profileor~/.brashrcor any other config file indicated with-soption
- lsdevs:
To see which devices are registered, this also defines
lsdevsas a shell function so it can be called directly
- mkg:
To make a group of devices to send commands to.
- mgg:
To manage a group of devices to send commands to. Use
-Gfor the name of the group and-addoption to add devices (indicate a name, ip and the password of each board) or-rmto remove devices (indicated by name)
- mksg:
To make a subset group of an existing group. Use -f for the name of the subgroup, -G for the name of parent group and -devs option to indicate the names of the devices to include.
- see:
To get specific info about a devices group
- gg:
To see global group
File IO operations
ACTIONS:
put,get,dsync,install,update_upyutils
- put:
To upload a file/files/pattern to device.
- get:
To download a file/files/pattern from device.
- dsync:
To recursively sync a folder/files/pattern from/to device filesystem. See dsync examples
Note
dsyncneedsshasum.pyin device, and if using-dflag it needsupysh.pyin device or use-fgflag. Also-rfflag needsupysh2.pyin device. Usedsync -hto see help about flag options.
dsyncexpects current working directory./to be at the same level of device filesystem current working directory, e.g. root/directory as default. so to sync host cwd./use:$ upydev dsyncor to sync device cwd into host cwd use:$ upydev dsync -d.
- install:
Install libs to
/libpath with upip.
- update_upyutils:
To update the latest versions of sync_tool.py, nanoglob.py, shasum.py, upylog.py, upynotify.py, upysecrets.py, upysh2.py, uping.py, time_it.py, uptime.py, cycles.py, wss_repl.py and wss_helper.py.
Firmware
ACTIONS:
fwr,flash,ota,mpyx
- fwr:
To list or get available firmware versions.
- flash:
To flash a firmware file in the device.
- ota:
To do an OTA firmware update. This needs
ota.pyorotable.py.
- mpyx:
To froze a module/script , and save some RAM, it uses mpy-cross tool (mpy-cross must be available in $PATH) e.g.
$ upydev mpyx [FILE].py,$ upydev mpyx [FILE].py [FILE2].py,$ upydev mpyx *.py.
Keygen
ACTIONS:
kg rsa,rsa sign,rsa verify,rsa auth,kg wr,kg ssl
- kg rsa:
To generate RSA-2048 bit key that will be shared with the device (it is unique for each device) use
-tfkeyto send this key to the device (use only if connected directly by USB, the AP of the device or a “secure” wifi e.g. local/home). Use-rkeyoption to remove private key from host (only store public key). To generate a host key pair usekg rsa host. Then the public key will be sent to the device so it can verify or authenticate the host signature.
- rsa sign:
To sign a file with device RSA key,
$ upydev rsa sign [FILE]. To sign a file with host RSA key:$ upydev rsa sign host [FILE]
- rsa verify:
To verify a signature of a file made with device RSA key :
$ upydev rsa verify [FILE]. To verify in device a signature made with host RSA key:$ upydev rsa verify host [FILE]
- rsa auth:
To authenticate a device with RSA encrypted challenge(Public Keys exchange must be done first)
- kg wr:
To “refresh” the WebREPL password with a new random password derivated from the RSA key previously generated. A token then is sent to the device to generate the same password from the RSA key previously uploaded. This won’t leave any clues in the TCP Websocekts packages of the current WebREPL password. (Only the token will be visible; check this using wireshark) (This needs upysecrets.py).
$ upydev kg wr,$ upydev keygen wr
- kg ssl:
Note
See WebSocket (ws) / WebSocket Secure (wss) TLS to check how to use these commands properly
- dev (default):
gen (default): So
$ upydev kg ssl # same as $ upydev kg ssl dev gen. To generate ECDSA key and certificate to enable SSL sockets This needs a ROOT CA key first, generated with$ upydev kg ssl CA. Use-tfkeyto upload this key to the device (use only if connected directly by USB, the AP of the device or a “secure” wifi e.g. local/home). Use-to [serial devname]flag with-tfkeyto transfer keys by USB/Serial.$ upydev kg sslexport: To export a device certificate to current working directory (cwd)
add: To add a device certificate from cwd to upydev verify locations path
status: To check datetime validity of a device certificate.
- CA:
gen (default): So
$ upydev kg ssl CA # same as $ upydev kg ssl CA gen. To generate ECDSA ROOT CA key and certificate to enable SSL sockets This needs a password, which will be required to generate and sign device/host certificates. Use-tfkeyto upload this ROOT CA cert to the device (use only if connected directly by USB, the AP of the device or a “secure” wifi e.g. local/home). Use-to [serial devname]flag with-tfkeyto transfer keys by USB/Serial.export: To export a ROOT CA key/cert to current working directory (cwd)
add: To add a ROOT CA key/cert from cwd to upydev verify locations path
status: To check datetime validity of a ROOT CA certificate.
- host:
gen (default): So
$ upydev kg ssl host # same as $ upydev kg ssl host gen. To generate a HOST ECDSA key and certificate to enable SSL sockets This needs a ROOT CA key first, generated with$ upydev kg ssl CA. This is needed so the device can authenticate the client (host).status: To check datetime validity of the HOST certificate.
REPL
ACTIONS:
repl,rpl,
- repl/rpl:
- To enter one of the following depending of upydevice type:
WebSocketDevice –> WebREPL/WebSecREPL (with
-wssflag)SerialDeivce –> Serial REPL
SHELL-REPL
ACTIONS:
shell,shl,shl-config,set_wss,jupyterc
- shell/shl:
To enter shell-repl
e.g.
$ upydev shl,$ upydev shl@mydeviceIt has autocompletion on TAB for available devices.
- shl-config:
To configure shell-repl prompt colors.
- set_wss:
To toggle between WebSecureREPL and WebREPL, to enable WebSecureREPL do
$ upydev set_wss, to disable$ upydev set_wss -wss
- jupyterc:
To run MicroPython upydevice kernel for jupyter console, CTRL-D to exit, %%lsmagic to see magic commands and how to connect to a device. Hit tab to autcomplete magic commands and MicroPython/Python code. (This needs jupyter and Jupyter Upydevice kernel to be installed)
Debugging
ACTIONS:
ping,probe,scan,run,timeit,stream_test,sysctl,log,pytest setup,pytest,play
- ping:
pings the target to see if it is reachable, CTRL-C to stop
- probe:
To test if a device is reachable, use
-ggflag for global group and-devsto filter which ones.
- scan:
To scan for devices, use with
-srfor serial,-ntfor network, or -bl for bluetooth low energy, if no flag provided it will do all three scans.
- run :
Same as
import [SCRIPT], where[SCRIPT]is indicated as second argument or by-foption (script must be in upydevice or in sd card indicated by-soption and the sd card must be already mounted as ‘sd’). Supports CTRL-C to stop the execution and exits nicely. e.g.$ upydev run myscript.py
- timeit:
To measure execution time of a module/script indicated as second argument or by
-foption This is an implementation of https://github.com/peterhinch/micropython-samples/tree/master/timed_function e.g.$ upydev timeit myscript.py
- stream_test:
To test download speed (from device to host). Default test is 10 MB of random bytes are sent in chunks of 20 kB and received in chunks of 32 kB. To change test parameters use
-chunk_tx,-chunk_rx, and-total_size.
- sysctl :
To start/stop a script without following the output. To follow initiate wrepl/srepl as normal, and exit with CTRL-x (webrepl) or CTRL-A,X (srepl) TO START: use
-start[SCRIPT_NAME], TO STOP: use-stop[SCRIPT_NAME]
- log:
To log the output of a upydevice script, indicate script with
-foption, and the sys.stdout log level and file log level with-dslevand-dflev(defaults are debug for sys.stdout and error for file). To log in background use -daemon option, then the log will be redirected to a file with level-dslev. To stop the ‘daemon’ log mode use -stopd and indicate script with-foption. ‘Normal’ file log and ‘Daemon’ file log are under .upydev_logs folder in $HOME directory, named after the name of the script. To follow an on going ‘daemon’ mode log, use-followoption and indicate the script with-foption.
- pytest setup:
To set
pytest.iniandconftest.pyin current working directory to enable selection of specific device with-@entry point.
- pytest:
To run upydevice test with pytest, do
$ upydev pytest setupfirst. e.g.$ upydev pytest mydevicetest.py. See Making Test for devicesNote
pytestandpytest-benchmarkrequired. Install with$ pip install pytest pytest-benchmark
- play:
To play custom tasks in ansible playbook style, e.g.
$ upydev play mytask.yaml. See Using tasks files. To add/remove/list tasks in upydev useadd,rm,list. Tasks will be stored in~/.upydev_playbooks
Group Mode
OPTIONS:
-G,-GPTo send a command to multiple devices in a group (made with make_group command)
To target specific devices within a group add
-devsoption as-devs [DEV NAME] [DEV NAME] ...or use-@ [DEV NAME] [DEV NAME] ...which has autocompletion on tab and accepts group names, * wildcards or brace expansion.e.g.
$ upydev check -@ esp\*,$ upydev check -@ esp{1..3}
Note
upydev will use local working directory group configuration unless it does not find any or manually indicated with -g option
- COMMAND MODE OPTION:
- -G :
$ upydev ACTION -G GROUPNAME [opts]or$ upydev ACTION -gg [opts]for global group. This sends the command to one device at a time- -GP:
$ upydev ACTION -GP GROUPNAME [opts]or$ upydev ACTION -ggp [opts]for global group. For parallel/non-blocking command execution using multiprocessing
Upy Commands
General Commands
info: to get device system info
id: to get device unique id
upysh: to enable the upy shell in the device (then do
$ upydev manto access upysh manual info)reset: to do a device soft reset
kbi: sends CTRL-C signal to stop an ongoing loop, to be able to access repl again
uhelp: MicroPython device help
umodules: MicroPython help(‘modules’)
ls: ls improved with multiple dirs and pattern matching
cat: cat improved with multiple files and pattern matching
mem: to get device RAM memory info
du: to get the size of file in root dir (default) or sd with
-s sdoption. If no file name indicated with-foption, prints all filesdf: to get memory info of the file system, (total capacity, free, used) (default root dir,
-soption to change)tree: to get directory structure in tree format (requires
upysh2.py)ifconfig: to get device network interface configuration info if station is enabled and connected to an AP
net scan: to do a WiFi scan and get available AP nearby.
net on: to enable STA interface
net off: to disable STA interface
net config: to configure and connect to an AP, must provide essid and password (see
-wp)net status: to get STA state. It returns True if enabled, False if disabled
ap on: to enable AP
ap off: to disable AP
ap status: AP state ; returns True if enabled, False if disabled
ap config: AP configuration of essid and password with authmode WPA/WPA2/PSK, (see
-ap), needs first that the AP is enabled (do$ upydev ap_on)ap scan: scan devices connected to AP; returns number of devices and mac address
i2c config: to configure the i2c pins (see
-i2c, defaults are SCL=22, SDA=23)i2c scan: to scan i2c devices (must config i2c pins first)
set rtc localtime: to pass host localtime and set upy device rtc
set rtc ntptime: to set rtc from server, (see
-utcfor time zone)datetime: to get date and time (must be set first, see above commands)
set hostname: to set hostname of the device for dhcp service (needs wpa_supplicant.py)
set localname: to set localname of the device for ble gap/advertising name (needs ble_uart_peripheral.py)
shasum -c: to check shasum file
shasum: to compute hash SHA-256 of a device file
sd: commands to manage an sd
diff: use git diff between device’s [~file/s] and local file/s
enable_sh: upload required files so shell is fully operational
uconfig: set or check config (from _config.py files in device)
uptime: shows how long the device has been running (requires
uptime.py)cycles: shows how many times the device has been rebooted (requires
cycles.py)load: run one or more local scripts in device. (accepts “*” wildcard and brace expansions)
SD
sd enable: to enable/disable the LDO 3.3V regulator that powers the SD module use
-pooption to indicate the Pin.sd init: to initialize the sd card; (spi must be configured first) it creates an sd object and mounts it as a filesystem.
sd deinit: to unmount sd card.
sd auto: experimental command, needs a sd module with sd detection pin. Enable an Interrupt with the sd detection pin, so it mounts the sd when is detected, and unmount the sd card when it is extracted.
Note
These commands needs sdcard.py in the device, see upyutils directory in upydev github repo. sd_auto also needs SD_AM.py in the device.
Serial
To use this mode connect the device by USB to the computer and use one of the following commands:
If the device is not previously configured do:
$ upydev shl -t [serial port] -p [baudrate] (default is 115200)
To configure a serial device see
To access SERIAL SHELL-REPL
$ upydev shl
To configure a serial device in the global group ‘UPY_G’ see
Then the device can be accessed with:
$ upydev shl -@ foo_device
or
$ upydev shl@foo_device
shell-repl @ foo_device
SerialREPL connected
MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; ESP32 module with ESP32
Type "help()" for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@foo_device:~ $
WebSocket (ws) / WebSocket Secure (wss) TLS
To enable WebREPL see WebREPL: a prompt over-wifi and WebREPL: web-browser interactive prompt
See WebSocketDevice development setup
Steps to enable WebSecureREPL mode
Put
wss_repl.pyandwss_helper.pyin the device: to do this useupdate_upyutilscommand as:
$ upydev update_upyutilsThis will upload :
sync_tool.py
nanoglob.py
shasum.py(to enable dsync)
upylog.py
upynotify.py
uping.py
upysh.py(to enable dsync)
upysecrets.py(to enable random WebREPL passwords generation)
upysh2.py(to enable ‘tree’ and ‘du’ command)
wss_repl.py(to enable WebSecureREPL)
wss_helper.py(to enable WebSecureREPL)
Generate ROOT CA ECDSA private key and self-signed ROOT CA certificate then upload the ROOT CA certificate to the device:
To generate the key do:
$ upydev kg ssl CA -tfkey-tfkeyoption is to send the ROOT CA certificate to the device.This will ask to set a passphrase, Do not forget it because it will be needed to generate HOST and device keys/certificates
Generate HOST ECDSA private key and ROOT CA signed HOST certificate :
To generate the key do:
$ upydev kg ssl hostThis will ask for the ROOT CA key passphrase to sign HOST certificate and then set another passphrase for HOST key, Do not forget it because it will be needed to log into WebSecureREPL
Generate device ECDSA private key and ROOT CA signed device certificate then upload them to the device:
To generate the key do:
$ upydev kg ssl -tfkey-tfkeyoption is to send the key/cert to the device (so use this if connected directly to the AP of the device or a “secure” wifi e.g. local/home) If not connected to a “secure” wifi upload the key (it is stored in upydev.__path__) by USB/Serial connection.This will ask for the ROOT CA key passphrase to be able to sign device certificate.
- At this point there should be in the host verify locations path
upydev.\__path__: ROOT CA key/cert pair
HOST key/cert pair
device cert.
- And in the device:
ROOT CA cert
device key/cert pair.
Note
This setup of ROOT CA–>HOST/device certificate chain enables TLS mutual authentication, and if the ROOT CA key/cert pair is exported to another host, it can generate its own HOST key/cert pair so it can perform a TLS mutual authentication too. This enables multihost support.
Enable WebSecREPL/WebSecureREPL in device
Replace
import webreplforimport wss_replandwebrepl.start()bywss_repl.start(ssl=True), inboot.pyormain.py.
After these steps WebSecureREPL or WebREPL over wss is now available:
$ upydev shl
Or if the global group UPY_G is configured already, any device in the global group
can be accessed with this mode using:
$ upydev shl@[DEVICE]
e.g.
mbp@cgg:~$ upydev shl@esp_room1
Enter passphrase for key 'HOST_key@6361726c6f.pem':
WebSecREPL with TLSv1.2 connected
TLSv1.2 @ ECDHE-ECDSA-AES128-CCM8 - 128 bits Encryption
MicroPython v1.18-165-g795370ca2-dirty on 2022-03-01; ESP32 module with ESP32
Type "help()" for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@esp_room1:~ $
Note
Once WebSecREPL is enabled, device configuration can be updated with host passphrase
as -p [password]:[passphrase] so it’s not needed for logging anymore.
$upydev config -t esp_room1.local -p mypasswd:mypassphr -@ esp_room1 -gg
WebSecureREPL protocol
TLSv1.2 @ ECDHE-ECDSA-AES128-CCM8 - 128 bits Encryption
Cipher suite ECDHE-ECDSA-AES128-CCM8 (recommended for embedded devices):
ECDSA private keys: Generated with SECP256R1 (a.k.a prime256v1 or P-256) see RFC-5480
Bluetooth Low Energy
This is still experimental and for esp32 requires ble_advertising.py, ble_uart_peripheral.py, ble_uart_repl.py to be uploaded
to the device. These scripts can be found in upyutils directory and they come from micropython examples.
Finally to enable it add the following to main.py:
import ble_uart_repl
ble_uart_repl.start()
To configure a ble device see
To access BLE SHELL-REPL
$ upydev shl
To configure a ble device in the global group ‘UPY_G’ see
Then the device can be accessed with:
$ upydev shl -@ ble_device
or
$ upydev shl@ble_device
shell-repl @ ble_device
BleREPL connected
MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; ESP32 module with ESP32
Type "help()" for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
esp32@ble_device:~ $
By default advertising and GAP name is set using device platform and unique id.
e.g. advertising name esp32-74 and GAP name ESP32@7c9ebd3d9df4
To set a custom name use set localname [NAME] in upydev or shell-repl as
esp32@ble_device:~ $ set localname bledev
# Now reset the device, then check localname with $ upydev -i -@ ble_device
# or connect to shell-repl mode and use command info e.g.
esp32@ble_device:~ $ info
BleDevice @ 00FEFE2D-5983-4D6C-9679-01F732CBA9D9, Type: esp32 , Class: BleDevice
Firmware: MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; 4MB/OTA BLE module with ESP32
(MAC: ec:94:cb:54:8e:14, Local Name: bledev, RSSI: -76 dBm)
shell-repl
The shell-repl Mode allows to toggle between SHELL and REPL modes (Use CTRL-s to do this)
The REPL mode has two limitations:
It is not listening actively for output (This means that if a timer/hardware interrupt callback print something it will not appear in the repl).
To define a function/class or make a loop use the paste mode. (CTRL-E)
However the original WebREPL/Serial REPL can be accessed from shell with
replcommande.g.
esp32@esp_room1:~ $ repl
<-- Device esp_room1 MicroPython -->
Use CTRL-x to exit, Use CTRL-k to see custom wrepl Keybdings
Password:
WebREPL connected
>>>
MicroPython v1.12 on 2019-12-20; ESP32 module with ESP32
Type "help()" for more information.
>>>
>>>
Note
To see help use -h
esp32@esp_room1:~ $ -h
usage: command [options]
This means that if the first argument [command] is not a registered
command it is redirected to the underlying system shell. To redirect a command
to the system shell use %command
Shell for MicroPython devices
Do CTRL-k to see keybindings info
[command] -h to see further help of any command
optional arguments:
-h, --help show this help message and exit
-v show program's version number and exit
commands:
** use enable_sh to upload required files for filesystem cmds
{ls,head,cat,mkdir,touch,cd,mv,pwd,rm,rmdir,du,tree,df,mem,exit,vim,run,reload,info,id,uhelp,modules,uping,rssi,net,ifconfig,ap,i2c,set,datetime,shasum,upip,timeit,update_upyutils,lcd,lsl,lpwd,ldu,docs,mdocs,ctime,enable_sh,diff,config,sd,uptime,cycles,load,repl,getcert,jupyterc,pytest,put,get,dsync,debugws,fw,mpyx,ota,upy-config,install}
ls list files or directories
head display first lines of a file
cat concatenate and print files
mkdir make directories
touch create a new file
cd change current working directory
mv move/rename a file
pwd print current working directory
rm remove file or pattern of files
rmdir remove directories or pattern of directories
du display disk usage statistics
tree list contents of directories in a tree-like format
df display free disk space
mem show ram usage info
exit exit upydev shell
vim use vim to edit files
run run device's scripts
reload reload device's scripts
info prints device's info
id prints device's unique id
uhelp prints device's help info
modules prints device's frozen modules
uping device send ICMP ECHO_REQUEST packets to network hosts
rssi prints device's RSSI (WiFi or BLE)
net manage network station interface (STA._IF)
ifconfig prints network interface configuration (STA._IF)
ap manage network acces point interface (AP._IF)
i2c manage I2C interface
set set device's configuration {rtc, hostname, localname}
datetime prints device's RTC time
shasum shasum SHA-256 tool
upip install or manage MicroPython libs
timeit measure execution time of a script/function
update_upyutils update upyutils scripts
lcd change local current working directory
lsl list local files or directories
lpwd print local current working directory
ldu display local disk usage statistics
docs see upydev docs at https://upydev.readthedocs.io/en/latest/
mdocs see MicroPython docs at docs.micropython.org
ctime measure execution time of a shell command
enable_sh upload required files so shell is fully operational
diff use git diff between device's [~file/s] and local file/s
config set or check config (from *_config.py files)#
sd commands to manage an sd
uptime prints device's uptime since latest boot (requires uptime.py)
cycles prints device's cycle count (requires cycles.py)
load run local script in device
repl enter WebREPL
getcert get device's certificate if available
jupyterc enter jupyter console with upydevice kernel
pytest run tests on device with pytest (use pytest setup first)
put upload files to device
get download files from device
dsync recursively sync a folder from/to device's filesystem
debugws toggle debug mode for websocket debugging
fw list or get available firmware from micropython.org
mpyx freeze .py files using mpy-cross. (must be available in $PATH)
ota to flash a firmware file using OTA system
upy-config enter upy-config dialog
install install libraries or modules with upip to ./lib
Note
To see keybindings info do CTRL-k: This will print the following info
* Autocompletion keybindings:
- tab to autocomplete device file / dirs names / raw micropython (repl commands)
- shift-tab to autocomplete shell commands
- shift-right to autocomplete local file / dirs names
- shift-left to toggle local path in prompt
* CTRL - keybindings:
- CTRL-x : to exit shell/repl
- CTRL-p : toggle RAM STATUS right aligned message (USED/FREE)
- CTRL-e : paste vim mode in repl
- CTRL-d : ends vim paste mode in repl and execute buffer
- CTRL-c : KeyboardInterrupt, in normal mode, cancel in paste mode
- CTRL-b : prints MicroPython version and sys platform
- CTRL-r : to flush line buffer
- CTRL-n : shows mem_info()
- CTRL-y : gc.collect() shortcut command
- CTRL-space : repeats last command
- CTRL-o, Enter : to enter upy-config dialog
- CTRL-t : runs temp buffer ('_tmp_script.py' in cwd)
- CTRL-w : prints device info
- CTRL-a : set cursor position at the beggining
- CTRL-j : set cursor position at the end of line
- CTRL-f : toggle autosuggest mode (Fish shell like)(use right arrow to complete)
- CRTL-s : toggle shell mode to navigate filesystem (see shell commands)
- CTRL-k : prints the custom keybindings (this list)
Some examples of these commands:
esp32@esp_room1:~ $ df
Filesystem Size Used Avail Use% Mounted on
Flash 2.0 MB 636.0 KB 1.4 MB 31.4 % /
esp32@esp_room1:~ $ cd lib
esp32@esp_room1:~/lib$ ls
client.py logging.py
protocol.py ssl_repl.py
sync_tool.py upylog.py
upynotify.py upysecrets.py
upysh2.py
esp32@esp_room1:~/lib$ mem
Memory Size Used Avail Use%
RAM 116.188 KB 17.984 KB 98.203 KB 15.5 %
esp32@esp_room1:~/lib$ cd
esp32@esp_room1:~ $ cd test_sync_dir
esp32@esp_room1:~/test_sync_dir$ tree
.
├── dirA
│ ├── dirB
│ │ └── file3.py
│ └── file2.py
├── THETESTCODE.py
├── file1b.py
└── othe_dir
3 directories, 4 files
esp32@esp_room1:~/test_sync_dir$ cat THETESTCODE.py
# This is a MicroPython script
# define a function in edit mode now
def my_test_func():
print('This is a function defined in edit mode with tab indentation')
for i in range(10):
my_test_func()
for i in range(5):
print('test finish')
esp32@esp_room1:~/test_sync_dir$ run THETESTCODE.py
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
This is a function defined in edit mode with tab indentation
test finish
test finish
test finish
test finish
test finish
esp32@esp_room1:~/test_sync_dir$ exit -r
Rebooting device...
Done!
logout
Connection to esp_room1 closed.
Note
enable_sh command installs shasum.py, upysh.py, upysh2.py and
nanoglob.py which are necesary for filesystem related commands like ls, dsync,
tree, etc.
Note
Shell-repl mode can be suspended in background with CTRL-z, to resume do $ fg
or list background jobs with $ jobs
Note
Shell-repl mode saves command history between sessions, so using autosuggestion
(enabled with CTRL-f) speeds up typing.
Note
Commands can be concatenated with && and there is some of support for pipes
>, >>, | and brace expansions {}.
Note
To toggle local path press shift-left, and if the local path happens to be
a git repo, current branch will be displayed in prompt. If checking out to a
different branch, re-toggle local path to reflect the new branch.
e.g.
my_project:/esp32@mydevice:~ [develop] $
Upyutils
This collection of scripts needs to be uploaded to the device to use some commands.
Bluetooth Low Energy
ble_advertising.py: helper module to do bluetooth low energy advertising easier
ble_uart_peripheral.py: helper module for bluetooth low energy UART GATT profile
ble_uart_repl.py: module to enable bluetooth low energy REPL through UART GATT profile
otable.py: module to enable bluetooth low energy DFU mode in esp32 and do OTA firmware updates.
Config
config: (__init__.py, configfuncs.py, params.py) config module to easily configure/set parameters as named tuples. See Config examples
Cryptography
rsa: module to perform asymmetric encryption with RSA algorithms
qrcode.py: module to be used with uqr lib and do pretty prints in terminal.
shasum.py: module to mimic shasum (compute hash SHA256 of file or check shafiles)
upysecrets.py: module to generate random passwords.
Develop
buzzertools.py: some utility methods to drive a buzzer (beep, alarm, and hardware interrupts)
dac_signal_gen.py: this lib is to generate signals with the on board DAC (sine and square waves)
test_code.py: a example test code script
time_it.py: a script to measure execution time of other scripts, implemented from, @peterhinch timed_fuction
upylog.py: a modified version of MicroPython logging module, with time format logging and log to file option (default file: ‘error.log’)
upynotify.py : a module to notify events with beeps (buzzer needed) and blinks. This is useful for “physical debugging”.
uptime.py: To set and get how long the device has been running.
cycles.py: To set and get how many times a device has been rebooted.
ursyslogger.py: To enable remote logging to a server using rsyslog (to be used with upylog.py)
IRQ
irq_controller.py: a module to controll hardware interrupts
Motor
dcmotor.py: this lib is to control a dc motor, with a dc motor driver (tested on DRV8871 )
servo.py: a lib to control servos, source: @deshipu
stepper.py: this lib is to control a stepper motor, with a stepper motor driver (tested on A4988 )
Network
mqtt_client.py: a tiny wrapper to add automatic print message callback
ota.py: a module to do OTA firmware updates for esp32
socket_client_server.py: a tiny wrapper to test clients/servers with tcp sockets
ssl_repl.py: a module to enable SSL repl
ssl_socket_client_server.py: a tiny wrapper to test clients/servers with SSL wrapped tcp sockets
sync_tool.py: this script is to get large files from the upy device faster (files > 100 kB)
uping.py: to make the device send ICMP ECHO_REQUEST packets to network hosts, (this adds statistics, continuous ping, and esp8266 compatibility) (original uping.py from @shawwwn )
wifiutils.py: to make easier to save/load wifi configuration (STA and AP ) and connect to an access point or enable its own.
wpa_supplicant.py: a module to mimic wpa_supplicant function, (connect to closest AP based on wpa_supplicant.config file)
wss_helper.py: to enable WebSecureREPL (server and client handshakes)
wss_repl.py: tiny wrapper of webrepl.py module to enable WebSecureREPL
nwatchdog.py: network watchdog class to check device is connected to an AP.
SD
SD_AM.py: This script is used with sd_auto command, see sd_auto for more info
sdcard.py: a lib to read/write to an sd card using spi interface, source @peterhinch
Sensors
ads1115.py: this lib is for the ADC ads1115 module, @robert-hh
bme280.py: This lib is for the ‘weather’ sensor BME280 , source: @robert-hh
ina219.py: this lib is for the INA219 voltage/current/power sensor, source: @chrisb2
lsm9ds1.py: this lib is for the IMU lsm9ds1, source: @hoihu
init_ADS.py: a tiny wrapper to add some methods ( read, stream, log, test…)
init_BME280.py: a tiny wrapper to add some methods ( read, stream, log, test…)
init_INA219.py: a tiny wrapper to add some methods ( read, stream, log, test…)
init_IMU.py: a tiny wrapper to add some methods ( read, stream, log, test…)
Shell
upysh2.py: upysh extesion with tree, du, and rm -r commands.
upysh.py: upysh custom version with ls, cat extended with pprint output and matching patterns.
nanoglob.py: glob module to match any pattern in device filesystem.
Upylog
MicroPython logging module with time format (predefined) and log to file support.
upylog.basicConfig (level=INFO, filename=_filename, stream=None, format=None, sd=False)
level: same as logging, by default upylog.INFO
options: [upylog.DEBUG, upylog.INFO, upylog.WARNING, upylog.ERROR, upylog.CRITICAL] or strings “DEBUG”, “INFO”…
filename: by default ‘error.log’
stream: by default sys.stderr
format: by default “LVL_MSG”, this will print [NAME] [LEVEL] MESSAGE
options:
“LVL_MSG” –> [NAME] [LEVEL] MESSAGE
“TIME_LVL_MSG” –> TIME [NAME] [LEVEL] MESSAGE
sd: default False; if set to True, ‘error.log’ file will be store in a mounted sd named ‘sd’
upylog.getLogger (name, log_to_file=False, rotate=_rotate)
name: the name of the logger.
log_to_file: wether save log output in the log file.
rotate: max size in bytes of the log file (default 2000).
Note
When max size is reached the log file will be renamed [logfile].log.1 and continue logging to [logfile].log, which means that if at all there would be two log files with max size of aprox. rotate bytes.
Note
If using time format, don’t forget to set the RTC, otherwise it will start to count from default EPOCH.
Note
By default Logger log level and file log level will be the same but both can be changed
with for log level Logger.setLevel(level) and file log level Logger.setLogfileLevel(level).
Example:
import upylog
upylog.basicConfig(level=upylog.INFO, format='TIME_LVL_MSG')
errlog = upylog.getLogger("errorlog_test", log_to_file=True) # This log to file 'error.log';
log = upylog.getLogger("log_test") # This just prints to sys.stdout
log.debug("Test message: %d(%s)", 100, "foobar")
log.info("Test message2: %d(%s)", 100, "foobar")
log.warning("Test message3: %d(%s)")
log.error("Test message4")
log.critical("Test message5")
upylog.info("Test message6")
try:
1/0
except Exception as e:
log.exception(e, "Exception Ocurred") # This error is not logged to file
pass
try:
5/0
except Exception as e:
errlog.exception(e, "Exception Ocurred") # This error IS logged to file
pass
Output:
2019-11-14 21:37:45 [log_test] [INFO] Test message2: 100(foobar)
2019-11-14 21:37:45 [log_test] [WARN] Test message3: %d(%s)
2019-11-14 21:37:45 [log_test] [ERROR] Test message4
2019-11-14 21:37:45 [log_test] [CRIT] Test message5
2019-11-14 21:37:45 [None] [INFO] Test message6
2019-11-14 21:37:45 [log_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 14, in <module>
ZeroDivisionError: divide by zero
2019-11-14 21:37:45 [errorlog_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 20, in <module>
ZeroDivisionError: divide by zero
# To see if the error was logged to file
>>> cat('error.log')
2019-11-14 21:37:45 [errorlog_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 20, in <module>
ZeroDivisionError: divide by zero
Note
To enable remote logging with
ursyslogger.RsysLoggeradd remote logger to log with:
...
>>> rsyslog = RsysLogger("server.local", port=514, hostname="mydevice", t_offset="+01:00")
>>> log.remote_logger = rsyslog
>>> log.info("Remote hello")
2022-09-15 10:06:04 [esp32@mydevice] [INFO] Remote hello
and in server e.g.
$ tail -F mydevice.local.log
Sep 15 10:06:04 mydevice.local esp32@mydevice Remote hello
Upynotify
A module to notify events with beeps (buzzer needed) and blinks. This is useful for physical debugging
Class:
NOTIFYER (buzz_pin, led_pin, fq=4000, on_time=150, n_times=2, off_time=150, timer=None, period=5000)
buzz_pin: the pin number to drive the buzzer
led_pin: the pin number to drive the led
fq: Frequency of the PWM to drive the buzzer
on_time: time in ms that the buzzer or led is on when calling the methods
buzzer_call(1)orblink_call(1)off_time: time in ms that the buzzer or led is off when calling the methods
buzzer_call(1)orblink_call(1)n_times: number of times a cycle of on-off is repeated when calling the methods
buzzer_call(1)orblink_call(1)timer: the number of the Timer to use, if using notify method. (This allows to schedule notifications and repeat it every period milliseconds)
period: time in milliseconds to pass to the Timer.
Methods of NOTIFYER class:
buzz_beep (beep_on_time, n_times, beep_off_time, fq, led=True)
To make the buzz beep with the indicated parameters
led_blink (led_on_time, n_times, led_off_time)
To make the buzz blink with the indicated parameters
notify (use=’buzz’, mode=’SHOT’, timeout=5000, on_init=None)
To make a notification using
buzzorledindicated by use, in the modeSHOT(one time after timeout in ms) orPERIODIC(each time after timeout in ms, until stopped). Allows to execute a function on_init, for example , print or log a message.
HOWTO
DEBUG
SERIAL DEVICES
While working with new serial devices be aware of:
Use a USB data cable, otherwise the device will be powered on but won’t be accessible.
If using a device with no native USB support (e.g. esp32, esp8266) most likely it will have a CP210x USB to UART Bridge Virtual COM Port (VCP) chip (or something similar), so the up to date drivers will be needed. (e.g. Silicon labs VCP Drivers) 1
If using Linux, it may be necessary to add user to dialout group to allow access to the USB device. 2
$ sudo usermod -a -G dialout $USER
- 1
This may require reset the computer so the drivers can be loaded properly.
- 2
See How do I allow a non-default user to use serial device ttyUSB0?
WEBSOCKET DEVICES (WIFI)
While working with devices connected over WiFi, in order to succeed sending upydev commands make sure that there is a reliable connection between the host and the device and that the wifi signal strength (rssi) in the device is above -80 (below -80 performance could be inconsistent)
A ‘Reliable’ connection means that there is no packet loss (use ping or upydev ping command to check)
See Received signal strength indication and Mobile Signal Strength Recommendations
TRACKING PACKETS
To see if “command packets” are sent and received or lost use Wireshark and filter the ip of the device.
SEE WHAT’S GOING ON UNDER THE HOOD:
ℹ️ Host and the device must be connected.
In a terminal window open a ‘serial repl’ with upydev srepl –port [USBPORT] command
In another window use upydev normally. Now in the terminal window with the serial repl you can see which commands are sent.
Working with dchp_hostname instead of IP
To do this activate station interface and set
dhcp_hostnamebefore connecting to the WLAN, e.g. in MicroPython REPL/script>>> import network >>> wlan = network.WLAN(network.STA_IF) >>> wlan.active(True) >>> wlan.config(dhcp_hostname='esp_dev')After connecting, the device should be reachable as
esp_dev.local, do$ ping esp_dev.local,avahi-resolve --name esp_dev.localor$ sudo resolvectl query esp_dev.localto check that it is working as expected.$ ping esp_dev.local PING esp_dev.local (192.168.1.73): 56 data bytes 64 bytes from 192.168.1.73: icmp_seq=0 ttl=255 time=2.403 ms 64 bytes from 192.168.1.73: icmp_seq=1 ttl=255 time=219.618 ms 64 bytes from 192.168.1.73: icmp_seq=2 ttl=255 time=239.348 ms ^C --- esp_dev.local ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 2.403/153.790/239.348/107.349 ms $ avahi-resolve --name esp_dev.local esp_dev.local 192.168.1.73 $ sudo resolvectl query esp_dev.local esp_dev.local: 192.168.1.73 -- link: enp5s0Now is possible to use this
dhcp_hostnamefor-targument while configuring a device e.g.$ upydev config -t esp_dev.local -p mypass -@ esp_dev -gg WebSocketDevice esp_dev settings saved in global group! $ upydev check -@ esp_dev Device: esp_dev Address: esp_dev.local, Device Type: WebSocketDevice $ upydev check -@ esp_dev -i Device: esp_dev WebSocketDevice @ ws://192.168.1.73:8266, Type: esp32, Class: WebSocketDevice Firmware: MicroPython v1.12-63-g1c849d63a on 2020-01-14; ESP32 module with ESP32 (MAC: 30:ae:a4:1e:73:f8, RSSI: -38 dBm) $ upydev ping -@ esp_dev PING esp_dev.local (192.168.1.73): 56 data bytes 64 bytes from 192.168.1.73: icmp_seq=0 ttl=255 time=56.655 ms 64 bytes from 192.168.1.73: icmp_seq=1 ttl=255 time=75.751 ms ^C --- esp_dev.local ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 56.655/66.203/75.751/9.548 ms
Note
Be aware some systems default ping use ipv6 first, and fallback to ipv4 while
resolving mDNS names, which may cause some delay. Use ping -4 instead which will use
ipv4 directly and resolve the name faster.
BLUETOOTH LOW ENERGY DEVICES
WEBSOCKET DEVICES (WIFI) THROUGH ZEROTIER GLOBAL AREA NETWORK
See ZeroTier Global Area Network
Although there is no library to directly connect a microcontroller to a zerotier network, a raspberry pi can be used as a bridge to make it possible. So install zerotier in your computer and in the raspberry pi.
Setup a zerotier network, add both your computer and the raspberry pi. (guide) Now add the rules for port fordwarding e.g. for WebREPL port (8266) in the raspberry pi and device with IP 192.168.1.46
First enable port forwarding by editing /etc/sysctl.conf and uncomment
net.ipv4.ip_forward=1
And reload
$ sudo sysctl -p
net.ipv4.ip_forward = 1
Then set the rules with iptables
$ sudo iptables -t nat -A PREROUTING -p tcp --dport 8266 -j DNAT --to-destination 192.168.1.46:8266
$ sudo iptables -t nat -A POSTROUTING -j MASQUERADE
And if using a firewall e.g. ufw
$ sudo ufw allow 8266
$ sudo ufw route allow in on ztrta7qtbo out on wlan0 to 192.168.1.46 port 8266 from any
$ sudo ufw reload
Where ztrta7qtbo is the zerotier interface (check this and its IP with ifconfig) Now connecting to the raspberry pi zerotier IP and port 8266 should redirect the traffic to the microcontroller port 8266 (WebREPL), e.g.
$ upydev config -t 142.64.115.62 -p mypass -gg -@ zerotdevice
Where 142.64.115.62 is the IP of the raspberry pi zerotier interface.
To configure shell-repl with WebSecureREPL through zerotier network do the same as above but with port 8833.
To enable ota firmware updates (e.g your computer has a zerotier IP 142.64.115.75)
$ sudo iptables -t nat -A PREROUTING -p tcp --dport 8014 -j DNAT --to-destination 142.64.115.75:8014
$ sudo iptables -t nat -A POSTROUTING -j MASQUERADE
And if using a firewall e.g. ufw
$ sudo ufw allow 8014
$ sudo ufw route allow in on wlan0 out on ztrta7qtbo to 142.64.115.75 port 8014 from any
$ sudo ufw reload
Note
If $ sudo zerotier-cli info shows this error:
Error connecting to the ZeroTier service:
Please check that the service is running and that TCP port 9993 can be contacted via 127.0.0.1.
Add this rule $ sudo iptables -t nat -I POSTROUTING -o lo -j ACCEPT
Now shell-repl mode is available using -zt option: e.g.
$ upydev shl@zerotdevice -zt 142.64.115.75/192.168.1.79
Where 192.168.1.79 is the IP of the raspberry pi in the local area network.
Or configure a device with the -zt option so it is not required anymore, e.g.
$ upydev config -t 142.64.115.62 -p mypass -gg -@ zerowpy -zt 142.64.115.75/192.168.1.79
WebSocketDevice zerotdevice settings saved in global group!
WebSocketDevice zerotdevice settings saved in ZeroTier group!
Now to access the shell-repl mode through zerotier network:
$ upydev shl@zerotdevice
Note
To allow ping and probe work correctly instead of pinging the raspberry pi,
add the ssh alias of the raspberry pi and the local ip or mDNS name of the device to -zt option as :[ALIAS]/[DEVICE_IP] e.g. :
$ upydev config -t 142.64.115.62 -p mypass -gg -@ zerowpy -zt 142.64.115.75/192.168.1.79:rpi/192.168.1.46 # OR upydev config -t 142.64.115.62 -p mypass -gg -@ zerowpy -zt 142.64.115.75/192.168.1.79:rpi/weatpy.local
This expects the raspberry pi to be accesible through ssh [ALIAS], and the keys added to the ssh-agent.
See ssh add keys and ssh alias
Now ping and probe should actually reach the device through raspbery pi ping, e.g.:
$ upydev ping -@ zerowpy
TESTING DEVICES WITH PYTEST
upydevice device classes allow to test MicroPython code in devices interactively with pytest, e.g. button press, screen swipes, sensor calibration, actuators, servo/stepper/dc motors , etc. Under tests directory there are example tests to run with devices. e.g.
$ upydev pytest test_esp_serial.py -@ sdev
Running pytest with Device: sdev
============================================================= test session starts =============================================================
platform darwin -- Python 3.7.9, pytest-6.1.0, py-1.9.0, pluggy-0.13.1
rootdir: /Users/carlosgilgonzalez/Desktop/MICROPYTHON/TOOLS/upydevice/test, configfile: pytest.ini
collected 7 items
test_esp_serial.py::test_devname PASSED
test_esp_serial.py::test_platform
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:14 [pytest] [ESP32] : Running SerialDevice test...
22:34:14 [pytest] [ESP32] : DEV PLATFORM: esp32
SerialDevice @ /dev/tty.SLAB_USBtoUART, Type: esp32, Class: SerialDevice
Firmware: MicroPython v1.16 on 2021-06-24; ESP32 module with ESP32
CP2104 USB to UART Bridge Controller, Manufacturer: Silicon Labs
(MAC: 30:ae:a4:23:35:64)
22:34:14 [pytest] [ESP32] : DEV PLATFORM TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_blink_led LED: ON
LED: OFF
LED: ON
LED: OFF
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:17 [pytest] [ESP32] : BLINK LED TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_run_script
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:17 [pytest] [ESP32] : RUN SCRIPT TEST: test_code.py
2000-01-01 00:53:30 [log_test] [INFO] Test message2: 100(foobar)
2000-01-01 00:53:30 [log_test] [WARN] Test message3: %d(%s)
2000-01-01 00:53:30 [log_test] [ERROR] Test message4
2000-01-01 00:53:30 [log_test] [CRIT] Test message5
2000-01-01 00:53:30 [None] [INFO] Test message6
2000-01-01 00:53:30 [log_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 14, in <module>
ZeroDivisionError: divide by zero
2000-01-01 00:53:30 [errorlog_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 20, in <module>
ZeroDivisionError: divide by zero
22:34:18 [pytest] [ESP32] : RUN SCRIPT TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_raise_device_exception
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:18 [pytest] [ESP32] : DEVICE EXCEPTION TEST: b = 1/0
[DeviceError]:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: divide by zero
22:34:18 [pytest] [ESP32] : DEVICE EXCEPTION TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_reset
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:18 [pytest] [ESP32] : DEVICE RESET TEST
Rebooting device...
Done!
22:34:18 [pytest] [ESP32] : DEVICE RESET TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_disconnect
---------------------------------------------------------------- live log call ----------------------------------------------------------------
22:34:18 [pytest] [ESP32] : DEVICE DISCONNECT TEST
22:34:18 [pytest] [ESP32] : DEVICE DISCONNECT TEST: [✔]
Test Result: PASSED
============================================================== 7 passed in 5.20s ==============================================================
IDE INTEGRATION with PLATFORMIO TERMINAL
ATOM
To do this go to Atom Settings –> Packages –> Then search for platformio-ide-terminal and click on Settings. Here go to Custom Texts section: (There are up to 8 “custom texts” or commands that can be customised) These custom text will be pasted an executed in the Terminal when called. And this can be done with keybindings or key-shortcuts. For example:
To automate upload the current file:
In Custom text 1 write: upydev put -f $F
To automate run the current file:
In Custom text 2 write: upydev run -f $F
To automate open the wrepl:
In Custom text 3 write: upydev wrepl
To automate diagnose:
In Custom text 4 write: upydev diagnose
Now configure the Keybindings, to do this go to Settings –> Keybindings –> your keymap file
Then in keymap.cson add: (This is an example, the key combination can be changed)
'atom-workspace atom-text-editor:not([mini])': 'ctrl-shift-d': 'platformio-ide-terminal:insert-custom-text-4' 'ctrl-cmd-u': 'platformio-ide-terminal:insert-custom-text-1' 'ctrl-cmd-x': 'platformio-ide-terminal:insert-custom-text-2' 'ctrl-cmd-w': 'platformio-ide-terminal:insert-custom-text-3'Save the file and now when pressing these key combinations should paste the command and run it in the Terminal.
Visual Studio Code
Using tasks and adding the shortcut in keybinds.json file for example:
Task:
"version": "2.0.0", "tasks": [ { "label": "upydev_upload", "type": "shell", "command": "upydev", "args": ["put", "-f", "${file}"], "options": { "cwd": "${workspaceFolder}"}, "presentation": { "echo": true, "reveal": "always", "focus": true, "panel": "shared", "showReuseMessage": true, "clear": false }, "problemMatcher": [] }]
Keybinding.json
{ "key": "ctrl+cmd+u",
"command": "workbench.action.tasks.runTask",
"args": "upydev_upload"}
About
Source code used as a reference
Other tools are:
Requirements (modules)
Tested on
OS:
MacOS X (Mojave 10.14.5-6, Big Sur 11.6.1)
Raspbian GNU/Linux 10 (Buster)
Ubuntu 20.04.2 LTS (GNU/Linux 5.8.0-55-generic x86_64)
BOARDS
Esp32 (Huzzah feather, Geekcreit)
Esp8266 Huzzah feather
Pyboard Lite v1.0
Pyboard v1.1
DEVELOPER TOOLS
Under Upyutils folder
Upylog MicroPython logging module with time format (predefined) and log to file support.
Upynotify MicroPython module with NOTIFYER class to notify events with beeps (needs buzzer) and blinks.
Examples
Using Config module
The config module allows to create and modify configuration
dinamically in the device using *_config.py files.
Note
Config parameter values accept numbers, strings and booleans True, False.
To use strings, they must be double + single quoted e.g. a="'INFO'".
Using config module as upydev command
Use $ upydev uconfig, (see help with $ upydev uconfig -h)
Warning
Note the u to avoid aliasing with config command.
Warning
config module expects / to be the root path of the device, if this is not the case add a
root_path.py file with this content:
RPATH = '/flash' # change flash for the root path of your device
e.g.
$ # Add a configuration named test
$ upydev uconfig add test
$ # Add parameters and values to test configuration.
$ upydev uconfig test: a=1 b=True c="'DEBUG'"
test -> a=1, b=True, c='DEBUG'
$ # Check configuration files now
$ upydev uconfig
test
$ # Check test configuration file
$ upydev uconfig test -y # pretty print flag
test:
b: True
c: DEBUG
a: 1
After this there should be a file in device cwd ./ named test_config.py
To access test device configuration, e.g. in main.py or REPL.
>>> from test_config import TEST
>>> dir(TEST)
['__class__', 'b', 'c', 'a']
>>> TEST.a
1
>>> TEST.b
True
>>> TEST.c
'DEBUG'
Using config module in shell-repl mode
Same as above but this time is config command, e.g
$ upydev shl
shell-repl @ pybV1.1
SerialREPL connected
MicroPython v1.19.1-217-g5234e1f1e on 2022-07-29; PYBv1.1 with STM32F405RG
Type "help()" for more information.
- CTRL-k to see keybindings or -h to see help
- CTRL-s to toggle shell/repl mode
- CTRL-x or "exit" to exit
pyboard@pybV1.1:~ $ config add foo
pyboard@pybV1.1:~ $ config foo: a=1 b=2 c=3
foo -> a=1, b=2, c=3
pyboard@pybV1.1:~ $ config
foo
pyboard@pybV1.1:~ $ config foo -y
foo:
b: 2
c: 3
a: 1
Using config module in a device script
To add a configuration (only needed one time)
from config import add_param
add_param('foo')
This adds a function named set_foo in config.params module.
To set foo configuration parameters
from config.params import set_foo
set_foo(a=1, b=2, c=3)
Which creates a foo_config.py file with a FOO named tuple.
>>> from foo_config import FOO
>>> print(FOO)
FOOCONFIG(a=1, c=3, b=2)
>>> FOO.a
1
>>> FOO.b
2
>>> FOO.c
3
Using config module in device development
Using config module in device main.py allows to set for example
different run-time modes e.g.
pyboard@pybV1.1:~ $ config add mode
pyboard@pybV1.1:~ $ config mode: app=False
Now in device main.py
from mode_config import MODE
import my_app
if MODE.app:
my_app.run()
else:
# this is debug mode
print('Device ready to debug')
# or
my_app.run_debug()
Or set log levels e.g. in combination with upylog.py
pyboard@pybV1.1:~ $ config add log
pyboard@pybV1.1:~ $ config log: level="'INFO'"
Now in device main.py
import upylog
from log_config import LOG
upylog.basicConfig(level=LOG.level, format="TIME_LVL_MSG")
log = upylog.getLogger("pyb", log_to_file=True, rotate=1000)
log.logfile = "debug.log"
log.info(f"Device ready")
log.debug("Just some debug info") # this will not print anything
After a soft reset:
MPY: sync filesystems
MPY: soft reboot
2022-08-16 16:18:39 [pyb] [INFO] Device ready
MicroPython v1.19.1-217-g5234e1f1e on 2022-07-29; PYBv1.1 with STM32F405RG
Type "help()" for more information.
Using dsync command
Note
To enable dsync command use $ upydev update_upyutils shasum.py upysh.py upysh2.py
otherwise only dsync -f option would be avaible (which will force sync host current
working directory into device current working directory)
dsync expects current working directory ./ to be at the same level of device current
working directory ./ which by default is usually root / directory.
So let’s consider this example project: my_project with the following structure
my_project$ tree
.
├── README.md
├── configfiles.config
└── src
├── boot.py
├── main.py
└── lib
└── foo.py
And device filesystem
.
├── boot.py
├── main.py
└── lib
├── shasum.py
└── upysh.py
So to sync src with device filesystem cd into src and use $ upydev dsync
Note
Use
-nto make a dry-run so you can see what would be synced.Use
-i file/pattern file/pattern..to ignore any unwanted file.Use
-pto see diff between modified files. (requiresgitto be available in $PATH)Use
-rfto remove any file/dir in device filesystem that is not in current host dir structure. (requiresupysh2.pyin device)
If using dsync from shell-repl mode -n flag will save the list of files/dirs
to sync so it can be viewed again with dsync -s, or dsync -s -app to show and apply.
my_project$ cd src
src$ upydev dsync
dsync: syncing path ./:
dsync: dirs: OK[✔]
dsync: syncing new files (1):
- ./lib/foo.py [0.02 kB]
./lib/foo.py -> mydevice:./lib/foo.py
./lib/foo.py [0.02 kB]
▏███████████████████████████████████████████▏ - 100 % | 0.02/0.02 kB | 0.02 kB/s | 00:01/00:01 s
1 new file, 0 files changed, 0 files deleted
To sync from device to host use -d flag.
src$ upydev dsync -d
dsync: path ./:
dsync: dirs: OK[✔]
dsync: syncing new files (2):
- ./lib/shasum.py [5.83 kB]
- ./lib/upysh.py [10.00 kB]
mydevice:./lib/shasum.py -> ./lib/shasum.py
./lib/shasum.py [5.83 kB]
▏███████████████████████████████████████████▏ - 100 % | 5.83/5.83 kB | 9.23 kB/s | 00:00/00:00 s
mydevice:./lib/upysh.py -> ./lib/upysh.py
./lib/upysh.py [10.00 kB]
▏███████████████████████████████████████████▏ - 100 % | 10.00/10.00 kB | 16.48 kB/s | 00:00/00:00 s
2 new files, 0 files changed, 0 files deleted
Now host and device filesystem are fully synced.
src $ tree
.
├── boot.py
├── main.py
└── lib
├── foo.py
├── shasum.py
└── upysh.py
src $ upydev tree
.
├── boot.py
├── main.py
└── lib
├── foo.py
├── shasum.py
└── upysh.py
Note
tree command needs module upysh2.py, that can be uploaded with
$ upydev update_upyutils upysh2.py. And in this case was already frozen
in firmware so that’s why it doesn’t appear in device filesystem.
dsync accepts multiple files/dirs/ or pattern that will filter what to sync
and speed up the syncing process, e.g.
# only sync lib dir
src $ upydev dsync lib
dsync: syncing path ./lib:
dsync: dirs: OK[✔]
dsync: files: OK[✔]
# only sync .py and .html files
src $ upydev dsync "*.py" "*.html"
dsync: syncing path ./*.py, ./*.html:
dsync: dirs: none
dsync: dirs: none
dsync: files: OK[✔]
Using tasks files
It is possible to create custom tasks yaml files so they can be played like in ansible.
using play command, check some examples in upydev/playbooks
e.g. consider this task file mytask.yaml:
---
- name: Example playbook
hosts: espdev, gkesp32, pybV1.1, oble
tasks:
- name: Load upysh
command: "import upysh"
- name: Check device filesystem
command: "ls"
- name: Check memory
command: "mem"
- name: Check info
command: "info && id"
- name: Raw MP command
command: "import time;print('hello');led.on();time.sleep(1);led.off()"
command_nb: "led.on();time.sleep(1);led.off()"
- name: Test load script
wait: 5
load: sample.py
And script sample.py
import time
for i in range(10):
print(f"This is a loaded script: {i}")
time.sleep(0.5)
First set the name of the file, in this case Example playbook, then set the devices
or hosts in which the tasks will be run.
Note
devices must be already saved in the global group (see save device in global group)
Finally add tasks using name, and the command to be run.
Directives
- Accepted directives are:
name: To indicate the playbook name or a task name
hosts: List of hosts (devices) (if none, it will use upydev config)
tasks: To indicate the list of tasks to execute
command: A command to be executed as in
shell-replmode or REPL command.command_nb: A raw MicroPython command to be executed in non-blocking way.
command_pl: A command to be executed in parallel (if using multiple devices).
reset: To reset the device before executing the task.
wait: To wait x seconds before executing the task.
load: To load a local script (in cwd or task file directory) and execute in device.
load_pl: To load a local script and execute in parallel (if using multiple devices).
include: To filter which hosts will be included in that task.
ignore: To filter which hosts will be ignored in that task.
Tip
command:This directive accepts commands that are available in
shell-replmode (see shell-repl), so several commands can be concatenated with&&. Note that it can also accept raw MicroPython commands.
command_nb:This directive means non-blocking so it will send the command and won’t wait for the output. Also only raw MicroPython commands are accepted.
command_plandload_pl:Won’t work with
BleDevicesso they will be ignored or may raise an error. Only exception is forpytestcommand which will work for all devices.
hosts:If directive not present,
playwill use current upydev config e.g.mytask_no_hosts.yaml--- - name: Example playbook with no hosts indicated tasks: - name: Load upysh command: "from upysh import ls" - name: Check device filesystem command: "ls" - name: Check memory command: "mem"
$ upydev play mytask_no_hosts.yaml -@ pybv1.1 $ # OR $ pyb play mytask_no_hosts.yaml $ # OR $ mydevgroup play mytask_no_hosts.yaml $ # OR $ upydev play mytask_no_hosts.yaml -@ pybv1.1 espdev gkesp32
To run the tasks file do:
$ upydev play playbooks/mytask.yaml
PLAY [Example playbook]
**********************************************************************************************************************************
TASK [Gathering Facts]
**********************************************************************************************************************************
ok [✔]: [pybV1.1]
ok [✔]: [gkesp32]
ok [✔]: [espdev]
ok [✔]: [oble]
TASK [Load upysh]
**********************************************************************************************************************************
[pybV1.1]: import upysh
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: import upysh
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: import upysh
----------------------------------------------------------------------------------------------------------------------------------
[oble]: import upysh
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
TASK [Check device filesystem]
**********************************************************************************************************************************
[pybV1.1]: ls
_tmp_script.py boot.py debug.log
debug.log.1 DIR_TEST dstest
dummy.py hostname.py lib
log_config.py main.py mpy_test.py
nemastepper.py new_dir new_file.py
pospid.py pospid_steppr.py README.txt
root_path.py servo_serial.py settings_config.py
stepper.py test_code.py test_file.txt
test_main.py test_secp256k1.py test_to_fail.py
testnew.py udummy.py upy_host_pub_rsa6361726c6f73.key
upy_pub_rsa3c003d000247373038373333.key upy_pv_rsa3c003d000247373038373333.key
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: ls
appble.py base_animations.py ble_flag.py
boot.py dummy.py ec-cacert.pem
error.log hostname.py http_client_ssl.py
http_server_ssl.py http_server_ssl_ecc_pem.py http_ssl_test.py
lib localname.py main.py
microdot.mpy myfile.txt myfile.txt.sign
ROOT_CA_cert.pem server.der size_config.py
ssl_auth.py SSL_cert_exp.pem SSL_certificate7c9ebd3d9df4.der
SSL_certificate7c9ebd569e5c.der ssl_config.py ssl_context_rsa.py
ssl_ecc_auth.py ssl_flag.py SSL_key7c9ebd3d9df4.der
ssl_rsa_auth.py test_code.py test_ssl_context_client.py
test_to_fail.py udummy.py upy_host_pub_rsa6361726c6f73.key
upy_host_pub_rsaacde48001122.key upy_pub_rsa7c9ebd3d9df4.key upy_pv_rsa7c9ebd3d9df4.key
webrepl_cfg.py wpa_supplicant.config wpa_supplicant.py
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: ls
ap_.config appble.py blemode_config.py
boot.py buzzertools.py dummy.py
ec-cacert.pem ec-cakey.pem error.log
hello_tls_context.py hostname.py lib
main.py main.py.sha256 microdot.mpy
remote_wifi_.config ROOT_CA_cert.pem rsa_cert.der
size_config.py src_boot.py src_main.py
SSL_cert_exp.pem SSL_certificate30aea4233564.der ssl_config.py
SSL_key30aea4233564.der SSL_key_exp.der SSL_key_exp.pem
test_code.py test_config.py test_ssl_context_client.py
test_to_fail.py webrepl_cfg.py wifi_.config
wpa_supplicant.config wpa_supplicant.py
----------------------------------------------------------------------------------------------------------------------------------
[oble]: ls
_tmp_script.py ble_flag.py boot.py dummy.py
error.log lib localname.py main.py
main2.py nofile.py nofile2.py size_config.py
test_code.py test_to_fail.py testble.py
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
TASK [Check memory]
**********************************************************************************************************************************
[pybV1.1]: mem
Memory Size Used Avail Use%
RAM 102.336 kB 11.728 kB 90.608 kB 11.5 %
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: mem
Memory Size Used Avail Use%
RAM 123.136 kB 18.576 kB 104.560 kB 15.1 %
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: mem
Memory Size Used Avail Use%
RAM 111.168 kB 52.576 kB 58.592 kB 47.3 %
----------------------------------------------------------------------------------------------------------------------------------
[oble]: mem
Memory Size Used Avail Use%
RAM 111.168 kB 23.120 kB 88.048 kB 20.8 %
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
TASK [Check info]
**********************************************************************************************************************************
[pybV1.1]: info && id
SerialDevice @ /dev/tty.usbmodem3370377430372, Type: pyboard, Class: SerialDevice
Firmware: MicroPython v1.19.1-217-g5234e1f1e on 2022-07-29; PYBv1.1 with STM32F405RG
Pyboard Virtual Comm Port in FS Mode, Manufacturer: MicroPython
(MAC: 3c:00:3d:00:02:47:37:30:38:37:33:33)
ID: 3c003d000247373038373333
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: info && id
WebSocketDevice @ wss://192.168.1.66:8833, Type: esp32, Class: WebSocketDevice
Firmware: MicroPython v1.19.1-321-gb9b5404bb on 2022-08-24; 4MB/OTA SSL module with ESP32
(MAC: 7c:9e:bd:3d:9d:f4, Host Name: gkesp32, RSSI: -69 dBm)
ID: 7c9ebd3d9df4
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: info && id
WebSocketDevice @ wss://192.168.1.53:8833, Type: esp32, Class: WebSocketDevice
Firmware: MicroPython v1.19.1-304-g5b7abc757-dirty on 2022-08-23; ESP32 module with ESP32
(MAC: 30:ae:a4:23:35:64, Host Name: espdev, RSSI: -49 dBm)
ID: 30aea4233564
----------------------------------------------------------------------------------------------------------------------------------
[oble]: info && id
BleDevice @ 00FEFE2D-5983-4D6C-9679-01F732CBA9D9, Type: esp32 , Class: BleDevice
Firmware: MicroPython v1.18-128-g2ea21abae-dirty on 2022-02-19; 4MB/OTA BLE module with ESP32
(MAC: ec:94:cb:54:8e:14, Local Name: oble, RSSI: -63 dBm)
ID: ec94cb548e14
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
TASK [Raw MP command]
**********************************************************************************************************************************
[pybV1.1]: import time;print('hello');led.on();time.sleep(1);led.off()
hello
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: import time;print('hello');led.on();time.sleep(1);led.off()
hello
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: import time;print('hello');led.on();time.sleep(1);led.off()
hello
----------------------------------------------------------------------------------------------------------------------------------
[oble]: import time;print('hello');led.on();time.sleep(1);led.off()
hello
----------------------------------------------------------------------------------------------------------------------------------
[pybV1.1]: led.on();time.sleep(1);led.off()
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: led.on();time.sleep(1);led.off()
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: led.on();time.sleep(1);led.off()
----------------------------------------------------------------------------------------------------------------------------------
[oble]: led.on();time.sleep(1);led.off()
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
TASK [Test load script]
**********************************************************************************************************************************
WAIT: DONE
----------------------------------------------------------------------------------------------------------------------------------
[pybV1.1]: loading playbooks/sample.py
This is a loaded script: 0
This is a loaded script: 1
This is a loaded script: 2
This is a loaded script: 3
This is a loaded script: 4
This is a loaded script: 5
This is a loaded script: 6
This is a loaded script: 7
This is a loaded script: 8
This is a loaded script: 9
Done!
----------------------------------------------------------------------------------------------------------------------------------
[gkesp32]: loading playbooks/sample.py
This is a loaded script: 0
This is a loaded script: 1
This is a loaded script: 2
This is a loaded script: 3
This is a loaded script: 4
This is a loaded script: 5
This is a loaded script: 6
This is a loaded script: 7
This is a loaded script: 8
This is a loaded script: 9
Done!
----------------------------------------------------------------------------------------------------------------------------------
[espdev]: loading playbooks/sample.py
This is a loaded script: 0
This is a loaded script: 1
This is a loaded script: 2
This is a loaded script: 3
This is a loaded script: 4
This is a loaded script: 5
This is a loaded script: 6
This is a loaded script: 7
This is a loaded script: 8
This is a loaded script: 9
Done!
----------------------------------------------------------------------------------------------------------------------------------
[oble]: loading playbooks/sample.py
This is a loaded script: 0
This is a loaded script: 1
This is a loaded script: 2
This is a loaded script: 3
This is a loaded script: 4
This is a loaded script: 5
This is a loaded script: 6
This is a loaded script: 7
This is a loaded script: 8
This is a loaded script: 9
Done!
----------------------------------------------------------------------------------------------------------------------------------
**********************************************************************************************************************************
It is also possible to filter which tasks to run on each device using
include or ignore directives, e.g.
---
- name: Example playbook
hosts: espdev, gkesp32, pybV1.1, oble
tasks:
- name: Raw MP Command
command: "import time;print('hello');led.on();time.sleep(1);led.off()"
include: pybV1.1
- name: Raw MP Command Parallel
command_nb: "led.on();time.sleep(2);led.off()"
ignore: pybV1.1
$ upydev play playbooks/mytask_pll.yaml
PLAY [Example playbook]
*******************************************************************************************************************************
TASK [Gathering Facts]
*******************************************************************************************************************************
ok [✔]: [pybV1.1]
ok [✔]: [gkesp32]
ok [✔]: [espdev]
ok [✔]: [oble]
TASK [Raw MP Command]
*******************************************************************************************************************************
[pybV1.1]: import time;print('hello');led.on();time.sleep(1);led.off()
hello
-------------------------------------------------------------------------------------------------------------------------------
*******************************************************************************************************************************
TASK [Raw MP Command Parallel]
*******************************************************************************************************************************
[gkesp32]: led.on();time.sleep(2);led.off()
-------------------------------------------------------------------------------------------------------------------------------
[espdev]: led.on();time.sleep(2);led.off()
-------------------------------------------------------------------------------------------------------------------------------
[oble]: led.on();time.sleep(2);led.off()
-------------------------------------------------------------------------------------------------------------------------------
*******************************************************************************************************************************
Tip
It is possible to add these tasks and its loaded scripts so they can be run from anywhere, using add, rm and list.
add: add a task file (
.yaml) or script (.py) to upydev, e.g.rm: remove a task or script file from upydev.
list: list available tasks in upydev.
tasks files and scripts are stored in ~/.upydev_playbooks
Let’s consider this example with battery.yaml and battery.py
---
- name: Check Battery State
tasks:
- name: Battery
load: battery.py
command: "battery"
from machine import ADC
bat = ADC(Pin(35))
bat.atten(ADC.ATTN_11DB)
class Battery:
def __init__(self, bat=bat):
self.bat = bat
def __repr__(self):
volt =((self.bat.read()*2)/4095)*3.6
percentage = round((volt - 3.3) / (4.23 - 3.3) * 100, 1)
return f"Battery Voltage : {round(volt, 2)}, V; Level:{percentage} %"
battery = Battery()
Adding this to upydev tasks
$ upydev play add battery.*
battery.yaml added to upydev tasks.
battery.py added to upydev tasks scripts.
$ upydev play battery # will run battery.yaml which loads battery.py in device and get battery state
PLAY [Check Battery State]
*******************************************************************************************************************************
HOSTS TARGET: [espdev]
HOSTS FOUND : [espdev]
*******************************************************************************************************************************
TASK [Gathering Facts]
*******************************************************************************************************************************
ok [✔]: [espdev]
TASK [Battery]
*******************************************************************************************************************************
[espdev]: loading battery.py
Done!
-------------------------------------------------------------------------------------------------------------------------------
[espdev]: battery
Battery Voltage : 4.2, V; Level:96.8 %
-------------------------------------------------------------------------------------------------------------------------------
*******************************************************************************************************************************
And after that it is possible to do:
$ upydev battery
Battery Voltage : 4.19, V; Level:95.9 %
Making Test for devices with upydev/upydevice + pytest
Simple tests definitions
Using upydevice/test/ as a template is easy to create custom tests for a device, to be run interactively, which can range from entire modules to single functions, e.g.
Note
pytest and pytest-benchmark required. Install with
$ pip install pytest pytest-benchmark
Consider test test_blink_led from test_esp_serial.py
This will test led on() and off() functions:
def test_blink_led():
TEST_NAME = 'BLINK LED'
if dev.dev_platform == 'esp8266':
_ESP_LED = 2
elif dev.dev_platform == 'esp32':
_ESP_LED = 13
_led = dev.cmd("'led' in globals()", silent=True, rtn_resp=True) # define led if not already defined
if not _led:
dev.cmd('from machine import Pin; led = Pin({}, Pin.OUT)'.format(_ESP_LED))
for i in range(2):
dev.cmd('led.on();print("LED: ON")')
time.sleep(0.2)
dev.cmd('led.off();print("LED: OFF")')
time.sleep(0.2)
try:
assert dev.cmd('not led.value()', silent=True,
rtn_resp=True), 'LED is on, should be off'
do_pass(TEST_NAME)
print('Test Result: ', end='')
except Exception as e:
do_fail(TEST_NAME)
print('Test Result: ', end='')
raise e
or testing a module test_code.py in device that will test upylog.py logging functions.
Consider test test_run_script from test_esp_serial.py
def test_run_script(): # the name of the test for pytest
TEST_NAME = 'RUN SCRIPT' # the name of the test to display in log
log.info('{} TEST: test_code.py'.format(TEST_NAME))
dev.wr_cmd('import test_code', follow=True)
try:
assert dev.cmd('test_code.RESULT', silent=True,
rtn_resp=True) is True, 'Script did NOT RUN'
dev.cmd("import sys,gc;del(sys.modules['test_code']);gc.collect()") # reloads module
do_pass(TEST_NAME)
print('Test Result: ', end='')
except Exception as e:
do_fail(TEST_NAME)
print('Test Result: ', end='')
raise e
So running this test
test $ upydev pytest test_esp_serial.py
Running pytest with Device: sdev
=========================================== test session starts ===========================================
platform darwin -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/carlosgilgonzalez/Desktop/MY_PROJECTS/MICROPYTHON/TOOLS/upydevice_.nosync/test, configfile: pytest.ini
plugins: benchmark-3.4.1
collected 7 items
test_esp_serial.py::test_devname PASSED
test_esp_serial.py::test_platform
---------------------------------------------- live log call ----------------------------------------------
20:09:36 [pytest] [sdev] [ESP32] : Running SerialDevice test...
20:09:36 [pytest] [sdev] [ESP32] : DEV PLATFORM: esp32
SerialDevice @ /dev/cu.usbserial-016418E3, Type: esp32, Class: SerialDevice
Firmware: MicroPython v1.19.1-285-gc4e3ed964-dirty on 2022-08-12; ESP32 module with ESP32
CP2104 USB to UART Bridge Controller, Manufacturer: Silicon Labs
(MAC: 30:ae:a4:23:35:64)
20:09:37 [pytest] [sdev] [ESP32] : DEV PLATFORM TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_blink_led LED: ON
LED: OFF
LED: ON
LED: OFF
---------------------------------------------- live log call ----------------------------------------------
20:09:39 [pytest] [sdev] [ESP32] : BLINK LED TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_run_script
---------------------------------------------- live log call ----------------------------------------------
20:09:39 [pytest] [sdev] [ESP32] : RUN SCRIPT TEST: test_code.py
2022-08-17 19:09:38 [log_test] [INFO] Test message2: 100(foobar)
2022-08-17 19:09:38 [log_test] [WARN] Test message3: %d(%s)
2022-08-17 19:09:38 [log_test] [ERROR] Test message4
2022-08-17 19:09:38 [log_test] [CRIT] Test message5
2022-08-17 19:09:38 [None] [INFO] Test message6
2022-08-17 19:09:38 [log_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 14, in <module>
ZeroDivisionError: divide by zero
2022-08-17 19:09:38 [errorlog_test] [ERROR] Exception Ocurred
Traceback (most recent call last):
File "test_code.py", line 20, in <module>
ZeroDivisionError: divide by zero
20:09:40 [pytest] [sdev] [ESP32] : RUN SCRIPT TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_raise_device_exception
---------------------------------------------- live log call ----------------------------------------------
20:09:40 [pytest] [sdev] [ESP32] : DEVICE EXCEPTION TEST: b = 1/0
[DeviceError]:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: divide by zero
20:09:40 [pytest] [sdev] [ESP32] : DEVICE EXCEPTION TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_reset
---------------------------------------------- live log call ----------------------------------------------
20:09:40 [pytest] [sdev] [ESP32] : DEVICE RESET TEST
Rebooting device...
Done!
20:09:41 [pytest] [sdev] [ESP32] : DEVICE RESET TEST: [✔]
Test Result: PASSED
test_esp_serial.py::test_disconnect
---------------------------------------------- live log call ----------------------------------------------
20:09:41 [pytest] [sdev] [ESP32] : DEVICE DISCONNECT TEST
20:09:41 [pytest] [sdev] [ESP32] : DEVICE DISCONNECT TEST: [✔]
Test Result: PASSED
============================================ 7 passed in 5.08s ============================================
Advanced tests definitions using yaml files
It is possible to use parametric test generation using yaml files e.g.
consider test_dev.py in upydev/tests.
Defining a test in a yaml file with the following directives:
Test Directives
name: The name of the test
hint: Info about the test, description, context, etc.
reset: To reset the device (
softorhard) before running the test.load: To load and execute a local file in device (.e.g
test_basic_math.py)command: The command to run the test in device.
args: To pass argument to the test function in device.
kwargs: To pass keyword arguments to the test function in device.
result: The command to get test result.
exp: Expected result to assert.
exp_type: Expected type of result to assert.
assert_op: Assert operation if other than
==.assert_itr: Assert elements of iterable result (
any, orall).benchmark: To run a benchmark of the function (device time). (
pytest-benchmarkplugin required)bench_host: To capture benchmark time of device + host (total time)
diff: To compute diff between device and host benchmark times (i.e. interface latency)
follow: To follow device benchmark output only (host+device time).
rounds: Rounds to run the function if doing a benchmark.
unit: To specify units if the measure is other than time in seconds. (i.e sensors)
network: To run network tests, (currently only
iperf3:server,iperf3:client)ip: IP to use in network tests, (
localip, ordevip)reload: To reload a script in device so it can be run again .e.g reload
foo_testmodule if command wasimport foo_test.
Note
load can be a command too, .e.g import mytestlib although it won’t return anything (only stdout).
Tip
Some directives are mutually exclusive, e.g. the 3 types of tests would be:
Assert Test: using command, result, exp (with options like exp_type, assert_op, assert_itr)
Benchmark Test: using benchmark with rounds and options like bench_host, diff, follow, unit…
Network Test: using network, command, ip to run network tests.
The directives that should work with any type of test are the rest ( name, load, args, kwargs, hint, reload, reset )
---
- name: "sum"
load: ./dev_tests/test_basic_math.py
command: "a = do_sum"
args: [1, 1]
result: a
exp: 2
- name: "diff"
command: "a = do_diff"
args: [1, 1]
result: a
exp: 0
- name: "product"
command: "a = do_product"
args: [2, 2]
result: a
exp: 4
- name: "division"
command: "a = do_div"
args: [1, 2]
result: a
exp: 0.5
def do_sum(a, b):
return a + b
def do_diff(a, b):
return a - b
def do_div(a, b):
return a / b
def do_product(a, b):
return a * b
tests $ upydev pytest test_load_basic_math.yaml
Running pytest with Device: pybV1.1
===================================================== test session starts =====================================================
platform darwin -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/carlosgilgonzalez/Desktop/MY_PROJECTS/MICROPYTHON/TOOLS/upydev_.nosync/tests, configfile: pytest.ini
plugins: benchmark-3.4.1
collected 7 items
test_dev.py::test_devname PASSED
test_dev.py::test_platform
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:44 [pytest] [pybV1.1] [PYBOARD] : Running SerialDevice test...
17:06:44 [pytest] [pybV1.1] [PYBOARD] : DEV PLATFORM: pyboard
17:06:44 [pytest] [pybV1.1] [PYBOARD] : DEV PLATFORM TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[sum]
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:44 [pytest] [pybV1.1] [PYBOARD] : Running [sum] test...
17:06:44 [pytest] [pybV1.1] [PYBOARD] : Loading ./dev_tests/test_basic_math.py file...
17:06:44 [pytest] [pybV1.1] [PYBOARD] : Command [a = do_sum(*[1, 1])]
17:06:45 [pytest] [pybV1.1] [PYBOARD] : expected: 2 --> result: 2
17:06:45 [pytest] [pybV1.1] [PYBOARD] : sum TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[diff]
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Running [diff] test...
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Command [a = do_diff(*[1, 1])]
17:06:45 [pytest] [pybV1.1] [PYBOARD] : expected: 0 --> result: 0
17:06:45 [pytest] [pybV1.1] [PYBOARD] : diff TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[product]
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Running [product] test...
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Command [a = do_product(*[2, 2])]
17:06:45 [pytest] [pybV1.1] [PYBOARD] : expected: 4 --> result: 4
17:06:45 [pytest] [pybV1.1] [PYBOARD] : product TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[division]
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Running [division] test...
17:06:45 [pytest] [pybV1.1] [PYBOARD] : Command [a = do_div(*[1, 2])]
17:06:45 [pytest] [pybV1.1] [PYBOARD] : expected: 0.5 --> result: 0.5
17:06:45 [pytest] [pybV1.1] [PYBOARD] : division TEST: [✔]
Test Result: PASSED
test_dev.py::test_disconnect
-------------------------------------------------------- live log call --------------------------------------------------------
17:06:45 [pytest] [pybV1.1] [PYBOARD] : DEVICE DISCONNECT TEST
17:06:45 [pytest] [pybV1.1] [PYBOARD] : DEVICE DISCONNECT TEST: [✔]
Test Result: PASSED
====================================================== 7 passed in 1.76s ======================================================
Note
pytest command will by default use test_dev.py if only yaml files indicated
Running Benchmarks with pytes-benchmark
See pytest-benchmark documentation
To write a benchmark test use benchmark directive to indicate a function that will be called rounds times (default 5). Consider this example:
---
- name: System Check
hint: "Device CPU frequency:"
command: "import machine;machine.freq()"
- name: Pystone Benchmark
hint: Run 500 loops, returns time in seconds to complete a run.
load: "import pystone_lowmem"
benchmark: "pystone_lowmem.main"
args: [500, True]
reload: "pystone_lowmem"
Where the function pystone_lowmem.main(500,True) will perform a 500 loops run and
return the time that it took in seconds.
Tip
Use of time.ticks_ms/time.ticks_us and time.ticks_diff to obtain the
time that it takes to run any function and return time in seconds e.g.
def benchmark_this(func, *args, **kwargs):
t0 = time.ticks_ms()
result = func(*args, **kwargs)
delta = time.ticks_diff(time.ticks_ms(), t0)
return delta/1e3 # delta/1e6 if using time.ticks_us
Running test_benchmark/test_pystone_bmk.yaml benchmark with different devices
and saving benchmark results
$ pyb pytest test_benchmark/test_pystones_bmk.yaml --benchmark-save=pyb_pystones
...
$ gk32 pytest test_benchmark/test_pystones_bmk.yaml --benchmark-save=gk32_pystones
...
$ sdev pytest test_benchmark/test_pystones_bmk.yaml --benchmark-save=sdev_pystones
...
$ oble pytest test_benchmark/test_pystones_bmk.yaml --benchmark-save=oble_pystones
...
It is possible to compare benchmark results e.g.
$ pytest-benchmark compare "*pystone*"
--------------------------------------------------------------------------------------------------- benchmark 'device': 4 tests ---------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[gkesp32@esp32] (0002_gk32_py) 188.0000 (1.0) 197.0000 (1.0) 192.8000 (1.0) 4.0249 (4.50) 195.0000 (1.0) 6.7500 (5.40) 2;0 5.1867 (1.0) 5 1
test_dev[Pystone Benchmark]:[pybV1.1@pyboard] (0001_pyb_pys) 262.0000 (1.39) 264.0000 (1.34) 263.4000 (1.37) 0.8944 (1.0) 264.0000 (1.35) 1.2500 (1.0) 1;0 3.7965 (0.73) 5 1
test_dev[Pystone Benchmark]:[oble@esp32] (0003_oble_py) 264.0000 (1.40) 267.0000 (1.36) 265.2000 (1.38) 1.3038 (1.46) 265.0000 (1.36) 2.2500 (1.80) 1;0 3.7707 (0.73) 5 1
test_dev[Pystone Benchmark]:[sdev@esp32] (0004_sdev_py) 282.0000 (1.50) 292.0000 (1.48) 288.4000 (1.50) 3.9115 (4.37) 289.0000 (1.48) 4.7500 (3.80) 1;0 3.4674 (0.67) 5 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
To see device firmware use --group-by=param
$ pytest-benchmark compare "*pystone*" --group-by=param
---------- benchmark 'Pystone Benchmark @ esp32 micropython-v1.18-128-g2ea21abae-dirty on 2022-02-19 4MB/OTA BLE module with ESP32': 1 tests -----------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[oble@esp32] (0003_oble_py) 264.0000 267.0000 265.2000 1.3038 265.0000 2.2500 1;0 3.7707 5 1
--------------------------------------------------------------------------------------------------------------------------------------------------------
------------ benchmark 'Pystone Benchmark @ esp32 micropython-v1.19.1-304-g5b7abc757-dirty on 2022-08-23 ESP32 module with ESP32': 1 tests -------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[sdev@esp32] (0004_sdev_py) 282.0000 292.0000 288.4000 3.9115 289.0000 4.7500 1;0 3.4674 5 1
--------------------------------------------------------------------------------------------------------------------------------------------------------
-------------- benchmark 'Pystone Benchmark @ esp32 micropython-v1.19.1-321-gb9b5404bb on 2022-08-24 4MB/OTA SSL module with ESP32': 1 tests --------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[gkesp32@esp32] (0002_gk32_py) 188.0000 197.0000 192.8000 4.0249 195.0000 6.7500 2;0 5.1867 5 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------
----------------- benchmark 'Pystone Benchmark @ pyboard micropython-v1.19.1-217-g5234e1f1e on 2022-07-29 PYBv1.1 with STM32F405RG': 1 tests ----------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[pybV1.1@pyboard] (0001_pyb_pys) 262.0000 264.0000 263.4000 0.8944 264.0000 1.2500 1;0 3.7965 5 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
To see the command/hint/context of the benchmark use --group-by=param:cmd
$ pytest-benchmark compare "*pys*" --group-by=param:cmd
benchmark "cmd={'name': 'Pystone Benchmark', 'hint': 'Run 500 loops, returns time in seconds to complete a run.', 'load': 'import pystone_lowmem', 'benchmark': 'pystone_lowmem.main(benchtm=True)', 'reload': 'pystone_lowmem'}": 4 tests
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dev[Pystone Benchmark]:[gkesp32@esp32] (0002_gk32_py) 188.0000 (1.0) 197.0000 (1.0) 192.8000 (1.0) 4.0249 (4.50) 195.0000 (1.0) 6.7500 (5.40) 2;0 5.1867 (1.0) 5 1
test_dev[Pystone Benchmark]:[pybV1.1@pyboard] (0001_pyb_pys) 262.0000 (1.39) 264.0000 (1.34) 263.4000 (1.37) 0.8944 (1.0) 264.0000 (1.35) 1.2500 (1.0) 1;0 3.7965 (0.73) 5 1
test_dev[Pystone Benchmark]:[oble@esp32] (0003_oble_py) 264.0000 (1.40) 267.0000 (1.36) 265.2000 (1.38) 1.3038 (1.46) 265.0000 (1.36) 2.2500 (1.80) 1;0 3.7707 (0.73) 5 1
test_dev[Pystone Benchmark]:[sdev@esp32] (0004_sdev_py) 282.0000 (1.50) 292.0000 (1.48) 288.4000 (1.50) 3.9115 (4.37) 289.0000 (1.48) 4.7500 (3.80) 1;0 3.4674 (0.67) 5 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
It is possible to benchmark measurements other than time, i.e. to benchmark sensor measurements.
Use unit directive in yaml file to indicate the unit or measurement and unit, e.g.
unit: "V" or unit: "voltage:V". This also can be set at the command line with
--unit option.
Let’s consider this example to take measurements with an ADC sensor ADS1115
---
- name: i2c_config
load: "from machine import I2C, Pin"
command: "i2c=I2C"
args: "[1]"
kwargs: "{'scl': Pin(22), 'sda': Pin(23)}"
- name: i2c_scan
command: "addr=i2c.scan()"
result: "i2c.scan()"
exp: [72]
exp_type: list
- name: ads_config
command: "from ads1115 import ADS1115;sensor=ADS1115(i2c,
addr[0], 1); sensor.set_conv(7, channel1=0)"
- name: ads_read
command: "mv = sensor.raw_to_v(sensor.read())"
result: mv
exp: 0
assert_op: "<="
exp_type: float
- name: ADS1115 Benchmark
hint: Test ADS1115 ADC sensor
load: "import time"
benchmark: "[(time.time_ns(), sensor.raw_to_v(sensor.read())) for i in range(100)]"
unit: "voltage:V"
rounds: 1
$ espd pytest test_ads/test_ads_bmk.yaml --benchmark-save=espd_ads1115 --benchmark-save-data
Running pytest with Device: espdev
Comparing against benchmarks from: Darwin-CPython-3.7-64bit/0022_espd_ads1115.json
===================================================================================================================== test session starts =====================================================================================================================
platform darwin -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/carlosgilgonzalez/Desktop/MY_PROJECTS/MICROPYTHON/TOOLS/upydev_.nosync/tests, configfile: pytest.ini
plugins: benchmark-3.4.1
collected 8 items
test_dev.py::test_devname PASSED
test_dev.py::test_platform
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:13 [pytest] [espdev] [ESP32] : Running WebSocketDevice test...
23:35:13 [pytest] [espdev] [ESP32] : Device: esp32
23:35:13 [pytest] [espdev] [ESP32] : Firmware: micropython v1.19.1-304-g5b7abc757-dirty on 2022-08-23; ESP32 module with ESP32
23:35:13 [pytest] [espdev] [ESP32] : DEV PLATFORM TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[i2c_config]
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:13 [pytest] [espdev] [ESP32] : Running [i2c_config] test...
23:35:13 [pytest] [espdev] [ESP32] : Loading from machi... snippet
paste mode; Ctrl-C to cancel, Ctrl-D to finish
=== from machine import I2C, Pin
23:35:14 [pytest] [espdev] [ESP32] : Command [i2c=I2C(*[1], **{'scl': Pin(22), 'sda': Pin(23)})]
23:35:14 [pytest] [espdev] [ESP32] : i2c_config TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[i2c_scan]
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:14 [pytest] [espdev] [ESP32] : Running [i2c_scan] test...
23:35:14 [pytest] [espdev] [ESP32] : Command [addr=i2c.scan()]
23:35:15 [pytest] [espdev] [ESP32] : expected: list --> result: <class 'list'>
23:35:15 [pytest] [espdev] [ESP32] : expected: [72] == result: [72]
23:35:15 [pytest] [espdev] [ESP32] : i2c_scan TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[ads_config]
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:15 [pytest] [espdev] [ESP32] : Running [ads_config] test...
23:35:15 [pytest] [espdev] [ESP32] : Command [from ads1115 import ADS1115;sensor=ADS1115(i2c, addr[0], 1); sensor.set_conv(7, channel1=0)]
23:35:16 [pytest] [espdev] [ESP32] : ads_config TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[ads_read]
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:16 [pytest] [espdev] [ESP32] : Running [ads_read] test...
23:35:16 [pytest] [espdev] [ESP32] : Command [mv = sensor.raw_to_v(sensor.read())]
23:35:17 [pytest] [espdev] [ESP32] : expected: float --> result: <class 'float'>
23:35:17 [pytest] [espdev] [ESP32] : expected: 0 <= result: 0.5788927
23:35:17 [pytest] [espdev] [ESP32] : ads_read TEST: [✔]
Test Result: PASSED
test_dev.py::test_dev[ADS1115 Benchmark]
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:17 [pytest] [espdev] [ESP32] : Running [ADS1115 Benchmark] test...
23:35:17 [pytest] [espdev] [ESP32] : Loading import tim... snippet
paste mode; Ctrl-C to cancel, Ctrl-D to finish
=== import time
23:35:18 [pytest] [espdev] [ESP32] : Hint: Test ADS1115 ADC sensor
23:35:18 [pytest] [espdev] [ESP32] : Benchmark Command [[(time.time_ns(), sensor.raw_to_v(sensor.read())) for i in range(100)]]
[(715559717849154000, 0.5791427), (715559717862555000, 0.5806427), (715559717872601000, 0.5815177), (715559717882546000, 0.5813928), (715559717892413000, 0.5820178), (715559717902349000, 0.5815177), (715559717912478000, 0.5811427), (715559717922413000, 0.5811427), (715559717932291000, 0.5812678), (715559717942212000, 0.5816427), (715559717952415000, 0.5815177), (715559717962345000, 0.5813928), (715559717972224000, 0.5813928), (715559717982118000, 0.5810177), (715559717992065000, 0.5815177), (715559718002000000, 0.5812678), (715559718011880000, 0.5811427), (715559718021805000, 0.5816427), (715559718031787000, 0.5808928), (715559718041728000, 0.5811427), (715559718051652000, 0.5815177), (715559718061669000, 0.5813928), (715559718071616000, 0.5813928), (715559718081553000, 0.5811427), (715559718091440000, 0.5811427), (715559718101334000, 0.5813928), (715559718111284000, 0.5813928), (715559718121221000, 0.5811427), (715559718131099000, 0.5808928), (715559718140997000, 0.5808928), (715559718150945000, 0.5811427), (715559718160970000, 0.5813928), (715559718170853000, 0.5817678), (715559718180773000, 0.5810177), (715559718190747000, 0.5808928), (715559718200688000, 0.5813928), (715559718210567000, 0.5807677), (715559718220463000, 0.5817678), (715559718230410000, 0.5805177), (715559718240350000, 0.5808928), (715559718250236000, 0.5810177), (715559718260293000, 0.5808928), (715559718270243000, 0.5811427), (715559718280178000, 0.5811427), (715559718290054000, 0.5811427), (715559718299951000, 0.5808928), (715559718309908000, 0.5805177), (715559718319844000, 0.5811427), (715559718329721000, 0.5806427), (715559718339619000, 0.5812678), (715559718349568000, 0.5813928), (715559718359641000, 0.5813928), (715559718370132000, 0.5813928), (715559718380101000, 0.5810177), (715559718390056000, 0.5807677), (715559718399996000, 0.5813928), (715559718409886000, 0.5810177), (715559718419775000, 0.5810177), (715559718429732000, 0.5816427), (715559718439670000, 0.5811427), (715559718449554000, 0.5808928), (715559718459455000, 0.5813928), (715559718469707000, 0.5811427), (715559718479695000, 0.5811427), (715559718489586000, 0.5815177), (715559718499520000, 0.5812678), (715559718509505000, 0.5805177), (715559718519437000, 0.5813928), (715559718529372000, 0.5808928), (715559718539332000, 0.5808928), (715559718549288000, 0.5811427), (715559718559336000, 0.5810177), (715559718569478000, 0.5807677), (715559718579373000, 0.5813928), (715559718589322000, 0.5810177), (715559718599262000, 0.5812678), (715559718609149000, 0.5806427), (715559718619041000, 0.5816427), (715559718628992000, 0.5812678), (715559718638925000, 0.5812678), (715559718648820000, 0.5812678), (715559718658717000, 0.5818928), (715559718668752000, 0.5808928), (715559718678688000, 0.5808928), (715559718688632000, 0.5807677), (715559718698531000, 0.5813928), (715559718708483000, 0.5808928), (715559718718414000, 0.5816427), (715559718728302000, 0.5808928), (715559718738192000, 0.5806427), (715559718748135000, 0.5812678), (715559718758075000, 0.5813928), (715559718768006000, 0.5807677), (715559718778015000, 0.5808928), (715559718787963000, 0.5816427), (715559718797895000, 0.5811427), (715559718807782000, 0.5812678), (715559718817676000, 0.5811427), (715559718827620000, 0.5808928), (715559718837561000, 0.5808928)]
23:35:20 [pytest] [espdev] [ESP32] : ADS1115 Benchmark TEST: [✔]
Test Result: PASSED
test_dev.py::test_disconnect
------------------------------------------------------------------------------------------------------------------------ live log call ------------------------------------------------------------------------------------------------------------------------
23:35:20 [pytest] [espdev] [ESP32] : DEVICE DISCONNECT TEST
23:35:20 [pytest] [espdev] [ESP32] : DEVICE DISCONNECT TEST: [✔]
Test Result: PASSED
Saved benchmark data in: /Users/carlosgilgonzalez/Desktop/MY_PROJECTS/MICROPYTHON/TOOLS/upydev_.nosync/tests/.benchmarks/Darwin-CPython-3.7-64bit/0023_espd_ads1115.json
------------------------------------------------------- benchmark 'device': 1 tests -------------------------------------------------------
Name (voltage in mV) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------
test_dev[ADS1115 Benchmark]:[espdev@esp32] 579.1427 582.0178 581.1515 0.3732 581.1427 0.5000 23;1 1.7207 100 1
-------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
===================================================================================================================== 8 passed in 13.40s ======================================================================================================================
Tip
benchmark directive accepts single value, a list of values or a list of 2 values tuples, where the first value is a time value and the second is the measurement to benchmark.
Note
To save benchmark results (i.e not only the stats) use --benchmark-save=[NAME] --benchmark-save-data
Data will be saved in .benchmarks/[SYSTEM PLATFORM]/xxxx_[NAME].json, e.g.
import json
from matplotlib import pyplot as plt
import sys
file = sys.argv[1]
with open(file, 'r') as rp:
report = json.load(rp)
data = report['benchmarks'][0]['stats']['data']
time_stamp = report['benchmarks'][0]['extra_info']['vtime']
# from absolute timestamps in ns to relative time in seconds
t_vec = [(t-time_stamp[0])/1e9 for t in time_stamp]
plt.plot(t_vec, data)
plt.ylabel("Voltage ($V$)")
plt.xlabel("Time ($s$)")
plt.show()
$ ./plot.py .benchmarks/Darwin-CPython-3.7-64bit/0023_espd_ads1115.json
Device development setups
SerialDevice
The easiest way to develop is having the device directly connected to the computer by USB. It allows a fast develop/test/fix/deploy cycle. It is also possible to make the device act as a peripheral so in can be integrated and controlled from the computer through a simple script, command line tool (like upydev) or even a GUI app. This is also possible using wireless connections, but this one has the lowest latency and the best performance.
To help with a fast development cycle, there are some tools/short-cuts/keybindings in shell-repl that allows
to load code from file into the device buffer to be executed. This is done using a tmp file _tmp_script.py in cwd.
In shell mode: Pressing
CTRL-twill load the contents of_tmp_script.pyin device buffer and execute it. e.g. the file_tmp_script.pywith content:
import time
for i in range(10):
print(f"hello: {i}")
time.sleep(0.1)
Pressing CTRL-t
esp32@sdev:~ $ Running Buffer...
hello: 0
hello: 1
hello: 2
hello: 3
hello: 4
hello: 5
hello: 6
hello: 7
hello: 8
hello: 9
In repl mode: Pressing
CTRL-ewill create/open file_tmp_script.pyto be modified invim. After saving and exit, the content will be loaded in device buffer. Next, pressingCTRL-dwill execute the buffer orCTRL-cto cancel. e.g.
Pressing CTRL-e, saving and exit, then CTRL-d:
Temp Buffer loaded do CTRL-D to execute or CTRL-C to cancel
>>> Running Buffer...
hello: 0
hello: 1
hello: 2
hello: 3
hello: 4
hello: 5
hello: 6
hello: 7
hello: 8
hello: 9
>>>
Using load command in shell mode: This allows to load and execute local scripts in device. This loads a local file content in device buffer and executes it.
esp32@sdev:~ $ load dummy.py
This is a dummy file for testing purpose
Tip
Device buffer is limited so if the file is too big it may be better to upload the file to the device or split the file in smaller ones.
Note
This is also avaible in the shell-repl for WebSocketDevices and BleDevices,
however latency will be higher due to the nature of wireless connections, e.g
higher latency of BleDevices if using a bluetooth headset at the same time.
WebSocketDevice
In upyutils/network there are some modules that may be of help when developing devices that needs to be connected and mantain a reliable connection.
Using wpa_supplicant.py module allows to the define a configuration file
wpa_supplicant.config with known AP networks ssid:passwords and its function
e.g.
{"my_ssid": "my_pass", "my_ssid2": "my_pass2"}
setup_network() will scan and connect to the closest known AP and return True if
connected.
import wpa_supplicant
if wpa_supplicant.setup_network():
print("Connected")
# Now for example RTC can be set with ntptime.settime()
else:
# Enable device AP instead
print("Enabling AP")
As a bonus to set mDNS host name of the device, add a file named hostname.py with
the name e.g. NAME = "mydevice" and it will be set by wpa_supplicant.setup_network()
too. This allows to use mydevice.local instead of device IP address.
$ ping mydevice.local
PING mydevice.local (192.168.1.53): 56 data bytes
64 bytes from 192.168.1.53: icmp_seq=0 ttl=255 time=100.093 ms
64 bytes from 192.168.1.53: icmp_seq=1 ttl=255 time=21.592 ms
64 bytes from 192.168.1.53: icmp_seq=2 ttl=255 time=239.554 ms
^C
--- mydevice.local ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 21.592/120.413/239.554/90.135 ms
In case a device needs to be moved (e.g. is powered by battery or to change device location) A network watchdog can be useful to reset and connect to a new AP or schedule a reconnection attemp.
Module nwatchdog.py defines a Network watchdog class that will init a WatchDog Timer
WDT with a timeout of n+10 and a hardware Timer that will check every n seconds if WLAN is connected,
and feed the WDT if True. Therefore if WLAN is for any reason disconnected the
watchdog will not be fed and it will trigger a reset.
e.g. in combination with wpa_supplicant.py and config module.
from wpa_supplicant import setup_network
from nwatchdog import WatchDog
from watchdog_config import WATCHDOG
from log_config import LOG
import ntptime
import upylog
from hostname import NAME
upylog.basicConfig(level=LOG.level, format='TIME_LVL_MSG')
log = upylog.getLogger(NAME, log_to_file=True, rotate=5000)
if setup_network():
if WATCHDOG.enabled:
wlan = network.WLAN(network.STA_IF)
watch_dog = WatchDog(wlan)
watch_dog.start()
log.info('Network WatchDog started!')
# Set time
try:
ntptime.settime()
except Exception as e:
log.exception(e, "NTP not available")
else:
# set AP
Module ursyslogger.py allows to forward logging messages to a remote host
using rsyslog . Configure rsyslog in remote server
to enable remote logging using TCP, see remote logging with rsyslog.
Then add RsysLogger to log.
...
>>> from ursyslogger import RsysLogger
>>> rsyslog = RsysLogger("server.local", port=514, hostname="mydevice", t_offset="+01:00")
>>> log.remote_logger = rsyslog
>>> log.info("Remote hello")
2022-09-15 10:06:04 [esp32@mydevice] [INFO] Remote hello
Then check in remote server e.g.
$ tail -F mydevice.local.log
Sep 15 10:06:04 mydevice.local esp32@mydevice Remote hello
BleDevice
Once the device is running BleREPL with NUS profile (Nordic UART Service), it is possible
to connect and send commands as with other devices. However due to the nature of
Bluetooth Low Energy, the computer needs to scan first and then connect, which
depending on the advertising period of the device, it may take a bit. This is why connecting
to the device using shell-repl mode is the best way to work. (e.g in case the device cannot
be connected using USB/Serial i.e. no physical access.)
Using config module it is possible to set different operation modes that will switch
between:
Custom ble app/profile (e.g.
Temeperature SensorProfile)Debug Mode, running
BleREPLwithNUSProfile.Bootloader Mode, running
DFUProfile to do OTA firmware updates.
Set mode config with, (in shell-repl)
esp32@oble:~ $ config add mode
esp32@oble:~ $ config mode: app=True blerepl=False dfu=False
mode -> app=True, blerepl=False, dfu=False
and in main.py:
from mode_config import MODE
if MODE.app:
print('App mode')
import myapp
myapp.run()
elif MODE.blerepl:
print('Debug mode')
import ble_uart_repl
ble_uart_repl.start()
elif MODE.dfu:
print('DFU mode')
from otable import BLE_DFU_TARGET
ble_dfu = BLE_DFU_TARGET()
Note
Note that while running in app or dfu mode to switch to another mode, it
should be done by setting mode config using config module and then rebooting the device, using a custom writable characteristic in case of app mode, and in case of dfu mode after a timeout with no connections
or OTA update successfully done (ideally switching to debug / blerepl mode to perform tests. After that
set config to app mode and reboot)