29 Commits

Author SHA1 Message Date
Kuba Szczodrzyński
a21c07fa03 [release] v0.5.0
Some checks failed
Lint check / Lint with clang-format (push) Has been cancelled
Lint check / Lint with black (push) Has been cancelled
PlatformIO Publish / publish (push) Has been cancelled
2022-06-02 23:06:11 +02:00
Kuba Szczodrzyński
bdffa7ef53 [core] Add UF2-based uploader 2022-06-02 23:05:35 +02:00
Kuba Szczodrzyński
22d40825bb [tools] Extract common utilities to separate modules 2022-06-02 22:22:23 +02:00
Kuba Szczodrzyński
50f26f546c [core] Use term "family" instead of "platform" 2022-06-02 22:22:19 +02:00
Kuba Szczodrzyński
9c7ea46ec3 [core] Build UF2 OTA image after linking 2022-06-02 22:21:51 +02:00
Kuba Szczodrzyński
5b4cf53d8a [tools] uf2ota: embed build date in extension tags 2022-06-02 14:14:16 +02:00
Kuba Szczodrzyński
79a701a4d4 [tools] uf2ota: fix for Python 3.7, fix Windows path compatibility 2022-06-02 14:12:03 +02:00
Kuba Szczodrzyński
81897e634c [realtek-ambz] Export both OTA images after linking 2022-06-01 21:41:04 +02:00
Kuba Szczodrzyński
3e11da4dd4 [docs] Add resources page 2022-05-31 12:06:24 +02:00
Kuba Szczodrzyński
12aa7fef04 [docs] Move sections from README to docs, add uf2families.json 2022-05-31 11:54:05 +02:00
Kuba Szczodrzyński
1ea6420bbc [core] Add Update library 2022-05-30 22:31:04 +02:00
Kuba Szczodrzyński
f3f1f36525 [core] Add uf2ota library source 2022-05-30 22:23:40 +02:00
Kuba Szczodrzyński
dee9a98cc3 [core] Update OTA API 2022-05-30 21:59:22 +02:00
Kuba Szczodrzyński
3345ce3fb9 [docs] Add missing documents to SUMMARY.md 2022-05-28 20:13:01 +02:00
Kuba Szczodrzyński
de70583838 [core] Put full UF2 Family ID in ChipFamily 2022-05-28 20:09:10 +02:00
Kuba Szczodrzyński
a43a737004 [realtek-ambz] Implement API class methods, fix CPU clock 2022-05-28 19:41:44 +02:00
Kuba Szczodrzyński
1f6899354f [core] Add LT class API methods 2022-05-28 19:39:43 +02:00
Kuba Szczodrzyński
9110a0c47e [docs] Document the UF2 OTA format 2022-05-28 14:48:07 +02:00
Kuba Szczodrzyński
b549790798 [tools] Implement UF2 binary patching, add dumping images 2022-05-28 14:47:45 +02:00
Kuba Szczodrzyński
5df430f3be [tools] Add UF2 OTA writer tool 2022-05-27 20:53:08 +02:00
Kuba Szczodrzyński
f3e8bcd74a [realtek-ambz] Add KVS partition, update boardgen 2022-05-27 15:29:46 +02:00
Kuba Szczodrzyński
4b050e11cf [core] Add dynamic FAL_PART_TABLE generation 2022-05-27 15:29:21 +02:00
Kuba Szczodrzyński
7ddbc09564 [core] Workaround LwIPmDNS compilation 2022-05-26 14:24:51 +02:00
Kuba Szczodrzyński
d3d62f80fd [realtek-ambz] Fix WiFi encryption type conversion 2022-05-26 14:24:29 +02:00
Kuba Szczodrzyński
bc7dbe6eec [api] Fix hexdump() default parameters 2022-05-25 12:27:52 +02:00
Kuba Szczodrzyński
e625f55353 [api] Add hexdump() utility 2022-05-24 17:55:12 +02:00
Kuba Szczodrzyński
aeebff9d5d [realtek-ambz] Fix WiFiEvents linking when not needed 2022-05-24 17:47:45 +02:00
Kuba Szczodrzyński
9a3c077ef1 [core] Add FlashDB KVS library 2022-05-24 17:43:30 +02:00
Kuba Szczodrzyński
91ae692058 [core] Fix gathering external library dependencies 2022-05-24 17:30:14 +02:00
71 changed files with 3945 additions and 374 deletions

View File

@@ -18,10 +18,10 @@
PlatformIO development platform for IoT modules manufactured by Tuya Inc.
The main goal of this project is to provide a usable build environment for IoT developers. While also providing vendor SDKs as PlatformIO cores,
the project focuses on developing working Arduino-compatible cores for supported platforms. The cores are inspired by Espressif's official core for ESP32,
which should make it easier to port/run existing ESP apps on Tuya IoT (and 3-rd party) platforms.
the project focuses on developing working Arduino-compatible cores for supported families. The cores are inspired by Espressif's official core for ESP32,
which should make it easier to port/run existing ESP apps on Tuya IoT (and 3-rd party) modules.
LibreTuya also provides a common interface for all platform implementations. The interface is based on ESP32 official libraries.
LibreTuya also provides a common interface for all family implementations. The interface is based on ESP32 official libraries.
**Note:** this project is work-in-progress.
@@ -36,7 +36,7 @@ LibreTuya also provides a common interface for all platform implementations. The
A (mostly) complete* list of Tuya wireless module boards.
  | Module Name | MCU | Flash | RAM | Pins** | Wi-Fi | BLE | Platform name
  | Module Name | MCU | Flash | RAM | Pins** | Wi-Fi | BLE | Family name
------------------------------|------------------------------------------------------------------------------------------------|-------------------------|-------|----------|-------------|-------|-----|---------------
❌ | [WB1S](https://developer.tuya.com/en/docs/iot/wb1s?id=K9duevbj3ol4x) | BK7231T @ 120 MHz | 2 MiB | 256 KiB | 18 (11 I/O) | ✔️ | ✔️ | -
❌ | [WB2L](https://developer.tuya.com/en/docs/iot/wb2l-datasheet?id=K9duegc9bualu) | BK7231T @ 120 MHz | 2 MiB | 256 KiB | 7 (5 I/O) | ✔️ | ✔️ | -
@@ -92,79 +92,6 @@ A (mostly) complete* list of Tuya wireless module boards.
** I/O count includes GPIOs, ADCs, PWM outputs and UART, but doesn't count CEN/RST and power pins.
## Project structure
```
arduino/
├─ <platform name>/ Arduino Core for specific SoC
│ ├─ cores/ Wiring core files
│ ├─ libraries/ Supported built-in platform libraries
├─ libretuya/
│ ├─ api/ Library interfaces
│ ├─ common/ Units common to all platforms
│ ├─ compat/ Fixes for compatibility with ESP32 framework
│ ├─ core/ LibreTuya API for Arduino cores
│ ├─ libraries/ Built-in platform-independent libraries
boards/
├─ <board name>/ Board-specific code
│ ├─ variant.cpp Arduino variant initialization
│ ├─ variant.h Arduino variant pin configs
├─ <board name>.json PlatformIO board description
builder/
├─ frameworks/ Framework builders for PlatformIO
│ ├─ <platform name>-sdk.py Vanilla SDK build system
│ ├─ <platform name>-arduino.py Arduino Core build system
├─ arduino-common.py Builder to provide ArduinoCore-API and LibreTuya APIs
├─ main.py Main PlatformIO builder
├─ utils.py SCons utils used during the build
docs/ Project documentation, guides, tips, etc.
platform/
├─ <platform name>/ Platform-specific configurations
│ ├─ bin/ Binary blobs (bootloaders, etc.)
│ ├─ fixups/ Code fix-ups to replace SDK parts
│ ├─ ld/ Linker scripts
│ ├─ openocd/ OpenOCD configuration files
tools/
├─ <tool name>/ Tools used during the build
platform.json PlatformIO manifest
platform.py Custom PlatformIO script
```
## Platforms
A list of platforms currently available in this project.
Platform name | Supported MCU(s) | Arduino Core | Source SDK (PIO framework)
---------------|------------------------------------------------------------------------|--------------|--------------------------------------------------------------------------
`realtek-ambz` | Realtek [AmebaZ](https://www.amebaiot.com/en/amebaz/) SoC (`RTL87xxB`) | ✔️ | `framework-realtek-amb1` ([amb1_sdk](https://github.com/ambiot/amb1_sdk))
### Realtek Ameba
The logic behind naming of Realtek chips and their series took me some time to figure out:
- RTL8xxxA - Ameba1/Ameba Series
- RTL8xxxB - AmebaZ Series
- RTL8xxxC - AmebaZ2/ZII Series
- RTL8xxxD - AmebaD Series
As such, there are numerous CPUs with the same numbers but different series, which makes them require different code and SDKs.
- [RTL8195AM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8195am)
- RTL8710AF (found in amb1_arduino)
- [RTL8711AM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8711am)
- [RTL8710BN](https://www.realtek.com/en/products/communications-network-ics/item/rtl8710bn)
- RTL8710BX (found in Tuya product pages)
- RTL8710B? (found in amb1_sdk)
- RTL8711B? (found in amb1_sdk)
- [RTL8710CM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8710cm)
- RTL8722CSM (found in ambd_arduino)
- RTL8720DN (found in ambd_arduino)
- [RTL8721DM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8721dm)
- RTL8722DM (found in ambd_arduino)
- and probably many more
Different Ameba series are not compatible with each other. Apparently, there isn't an official public SDK for AmebaZ that can support C++ properly.
## Arduino Core support status
Note: this list will probably change with each functionality update.
@@ -188,13 +115,12 @@ Wi-Fi Events | ✔️
IPv6 | ❌
HTTP Client (SSL) | ✔️ (✔️)
HTTP Server | ✔️
NVS / Preferences |
NVS / Preferences | ✔️
SPIFFS | ❌
BLE | -
HTTP | ❌
NTP | ❌
OTA |
MDNS |
OTA |
MDNS | ✔️
MQTT | ✅
SD | ❌

View File

@@ -1,10 +1,15 @@
* [Home](README.md)
* [Configuration](docs/config.md)
* Reference
* [💻 Family list](docs/families.md)
* [✔️ Implementation status](docs/implementation-status.md)
* [🔧 Configuration](docs/config.md)
* [📁 Project structure](docs/project-structure.md)
* 🔖 Code reference
* [LibreTuya API](docs/reference/lt-api.md)
* [Class reference](ltapi/class_libre_tuya.md)
* [Static functions](ltapi/_libre_tuya_a_p_i_8cpp.md)
* [Logger](ltapi/lt__logger_8h.md)
* [Chip types & UF2 families](ltapi/_chip_type_8h.md)
* [POSIX utilities](ltapi/lt__posix__api_8h.md)
* Common API
* [Flash](ltapi/class_i_flash_class.md)
* [FS](ltapi/classfs_1_1_f_s.md)
@@ -19,10 +24,12 @@
* [LibreTuya libraries](docs/libs-built-in.md)
* [base64](ltapi/classbase64.md)
* [HTTPClient](ltapi/class_h_t_t_p_client.md)
* [mDNS](ltapi/classm_d_n_s.md)
* NetUtils
* [ssl/MbedTLSClient](ltapi/class_mbed_t_l_s_client.md)
* [IPv6Address](ltapi/classarduino_1_1_i_pv6_address.md)
* [LwIPRxBuffer](ltapi/class_lw_i_p_rx_buffer.md)
* [Update](ltapi/class_update_class.md)
* [WebServer](ltapi/class_web_server.md)
* [WiFiMulti](ltapi/class_wi_fi_multi.md)
* [Third party libraries](docs/libs-3rd-party.md)
@@ -31,7 +38,12 @@
* [Functions](ltapi/functions.md)
* [Macros](ltapi/macros.md)
* [File list](ltapi/files.md)
* Platforms
* [✈️ OTA format](docs/ota/README.md)
* [uf2ota.py tool](docs/ota/uf2ota.md)
* [uf2ota.h library](docs/ota/library.md)
* [uf2ota.h reference](ltapi/uf2ota_8h.md)
* Families
* [Realtek - notes](docs/platform/realtek/README.md)
* Realtek AmebaZ Series
* Boards
* [WR3](boards/wr3/README.md)
@@ -40,3 +52,4 @@
* [Memory management](docs/platform/realtek-ambz/memory-management.md)
* [Debugging](docs/platform/realtek/debugging.md)
* [Exception decoder](docs/platform/realtek/exception-decoder.md)
* [🔗 Resources](docs/resources.md)

View File

@@ -0,0 +1,23 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-28. */
enum ChipFamily {
// used in UF2 Family ID
RTL8710A = 0x9FFFD543, // Realtek Ameba1
RTL8710B = 0x22E0D6FC, // Realtek AmebaZ (realtek-ambz)
RTL8720C = 0xE08F7564, // Realtek AmebaZ2
RTL8720D = 0x3379CFE2, // Realtek AmebaD
BK7231T = 0x675A40B0, // Beken 7231T
BK7231N = 0x7B3EF230, // Beken 7231N
BL602 = 0xDE1270B7, // Boufallo 602
XR809 = 0x51E903A8, // Xradiotech 809
};
enum ChipType {
// IDs copied from rtl8710b_efuse.h
RTL8710BL = ((RTL8710B >> 24) << 8) | 0xE0, // ???
RTL8710BN = ((RTL8710B >> 24) << 8) | 0xFF, // CHIPID_8710BN / QFN32
RTL8710BU = ((RTL8710B >> 24) << 8) | 0xFE, // CHIPID_8710BU / QFN48
RTL8710BX = ((RTL8710B >> 24) << 8) | 0xFB, // CHIPID_8710BN_L0 / QFN32
RTL8711BN = ((RTL8710B >> 24) << 8) | 0xFD, // CHIPID_8711BN / QFN48
RTL8711BU = ((RTL8710B >> 24) << 8) | 0xFC, // CHIPID_8711BG / QFN68
};

View File

@@ -8,7 +8,13 @@ String ipToString(const IPAddress &ip) {
return String(szRet);
}
static void lt_random_bytes(uint8_t *buf, size_t len) {
/**
* @brief Generate random bytes using rand().
*
* @param buf destination pointer
* @param len how many bytes to generate
*/
void lt_rand_bytes(uint8_t *buf, size_t len) {
int *data = (int *)buf;
size_t i;
for (i = 0; len >= sizeof(int); len -= sizeof(int)) {
@@ -20,3 +26,137 @@ static void lt_random_bytes(uint8_t *buf, size_t len) {
memcpy(buf + i * sizeof(int), pRem, len);
}
}
/**
* @brief Print data pointed to by buf in hexdump-like format (hex+ASCII).
*
* @param buf source pointer
* @param len how many bytes to print
* @param offset increment printed offset by this value
* @param width how many bytes on a line
*/
void hexdump(uint8_t *buf, size_t len, uint32_t offset, uint8_t width) {
uint16_t pos = 0;
while (pos < len) {
// print hex offset
printf("%06x ", offset + pos);
// calculate current line width
uint8_t lineWidth = min(width, len - pos);
// print hexadecimal representation
for (uint8_t i = 0; i < lineWidth; i++) {
if (i % 8 == 0) {
printf(" ");
}
printf("%02x ", buf[pos + i]);
}
// print ascii representation
printf(" |");
for (uint8_t i = 0; i < lineWidth; i++) {
char c = buf[pos + i];
printf("%c", isprint(c) ? c : '.');
}
printf("|\n");
pos += lineWidth;
}
}
/**
* @brief Get LibreTuya version string.
*/
const char *LibreTuya::getVersion() {
return LT_VERSION_STR;
}
/**
* @brief Get board name.
*/
const char *LibreTuya::getBoard() {
return LT_BOARD_STR;
}
/**
* @brief Get CPU family ID.
*/
ChipFamily LibreTuya::getChipFamily() {
return FAMILY;
}
/**
* @brief Get CPU family name as string.
*/
const char *LibreTuya::getChipFamilyName() {
return STRINGIFY_MACRO(FAMILY);
}
static char *deviceName = NULL;
/**
* @brief Get device friendly name in format "LT-<board>-<chip id>".
* Can be used as hostname.
*/
const char *LibreTuya::getDeviceName() {
if (deviceName)
return deviceName;
uint32_t chipId = getChipId();
uint8_t *id = (uint8_t *)&chipId;
const char *board = getBoard();
uint8_t boardLen = strlen(board);
deviceName = (char *)malloc(3 + boardLen + 1 + 6 + 1);
sprintf(deviceName, "LT-%s-%02x%02x%02x", board, id[0], id[1], id[2]);
return deviceName;
}
static uint8_t otaRunningIndex = 0;
/**
* @brief Get the currently running firmware OTA index.
*/
uint8_t LibreTuya::otaGetRunning() {
if (otaRunningIndex)
return otaRunningIndex;
// otaRunningIndex will be correct even after switchOta()
return otaRunningIndex = otaGetStoredIndex();
}
/**
* @brief Get the OTA index for updated firmware.
*
* Note: returns 1 for chips without dual-OTA.
*/
uint8_t LibreTuya::otaGetTarget() {
if (!otaSupportsDual())
return 1;
return otaGetRunning() ^ 0b11;
}
/**
* @brief Perform OTA rollback.
*
* @return false if no second image to run, writing failed or dual-OTA not supported
*/
bool LibreTuya::otaRollback() {
if (!otaCanRollback())
return false;
if (otaGetRunning() != otaGetStoredIndex())
// force switching back to current image
return otaSwitch(true);
return true;
}
/**
* @brief Check if OTA rollback is supported and available (there is another image to run).
* @return false if no second image to run or dual-OTA not supported
*/
bool LibreTuya::otaCanRollback() {
if (!otaSupportsDual())
return false;
if (otaGetRunning() == otaGetStoredIndex())
return true;
if (otaGetRunning() == 1 && otaHasImage1())
return true;
if (otaGetRunning() == 2 && otaHasImage2())
return true;
return false;
}

View File

@@ -43,33 +43,139 @@ extern "C" {
#define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
#define PGM_VOID_P const void *
// C functions
void lt_rand_bytes(uint8_t *buf, size_t len);
// C++ only functions
#ifdef __cplusplus
String ipToString(const IPAddress &ip);
void hexdump(uint8_t *buf, size_t len, uint32_t offset = 0, uint8_t width = 16);
#else
void hexdump(uint8_t *buf, size_t len, uint32_t offset, uint8_t width);
#endif
// Main class
#ifdef __cplusplus
#include <Flash.h> // for flash inline methods
#include <core/ChipType.h>
/**
* @brief Main LibreTuya API class.
*
* This class contains all functions common amongst all platforms.
* Implementations of these methods may vary between platforms.
* This class contains all functions common amongst all families.
* Implementations of these methods may vary between families.
*
* The class is accessible using the `LT` global object (defined by the platform).
* The class is accessible using the `LT` global object (defined by the family).
*/
class LibreTuya {
public: /* Common methods - note: these are documented in LibreTuyaAPI.cpp */
const char *getVersion();
const char *getBoard();
ChipFamily getChipFamily();
const char *getChipFamilyName();
const char *getDeviceName();
uint8_t otaGetRunning();
uint8_t otaGetTarget();
bool otaRollback();
bool otaCanRollback();
/* Common methods*/
public: /* Inline methods */
inline uint32_t getFlashChipSize() {
return Flash.getSize();
}
public:
/* Platform-defined methods */
// inline bool flashEraseSector(uint32_t sector) {}
// inline bool flashWrite(uint32_t offset, uint32_t *data, size_t size) {}
// inline bool flashRead(uint32_t offset, uint32_t *data, size_t size) {}
// inline bool partitionEraseRange(const esp_partition_t *partition, uint32_t offset, size_t size) {}
// inline bool partitionWrite(const esp_partition_t *partition, uint32_t offset, uint32_t *data, size_t size) {}
// inline bool partitionRead(const esp_partition_t *partition, uint32_t offset, uint32_t *data, size_t size) {}
public:
public: /* Family-defined methods */
/**
* @brief Reboot the CPU.
*/
void restart();
public: /* CPU-related */
/**
* @brief Get CPU model ID.
*/
ChipType getChipType();
/**
* @brief Get CPU model name as string.
*/
const char *getChipModel();
/**
* @brief Get CPU unique ID. This may be based on MAC, eFuse, etc.
*/
uint32_t getChipId();
/**
* @brief Get CPU core count.
*/
uint8_t getChipCores();
/**
* @brief Get CPU core type name as string.
*/
const char *getChipCoreType();
/**
* @brief Get CPU frequency in MHz.
*/
uint32_t getCpuFreqMHz();
/**
* @brief Get CPU cycle count.
*/
inline uint32_t getCycleCount() __attribute__((always_inline));
public: /* Memory management */
/**
* @brief Get total RAM size.
*/
uint32_t getRamSize();
/**
* @brief Get total heap size.
*/
uint32_t getHeapSize();
/**
* @brief Get free heap size.
*/
uint32_t getFreeHeap();
/**
* @brief Get lowest level of free heap memory.
*/
uint32_t getMinFreeHeap();
/**
* @brief Get largest block of heap that can be allocated at once.
*/
uint32_t getMaxAllocHeap();
public: /* OTA-related */
/**
* @brief Read the currently active OTA index, i.e. the one that will boot upon restart.
*/
uint8_t otaGetStoredIndex();
/**
* @brief Check if the chip supports dual-OTA.
*/
bool otaSupportsDual();
/**
* @brief Check if OTA1 image is valid.
*/
bool otaHasImage1();
/**
* @brief Check if OTA2 image is valid.
*/
bool otaHasImage2();
/**
* @brief Try to switch OTA index to the other image.
*
* Note: should return true for chips without dual-OTA. Should return false if one of two images is not valid.
*
* @param force switch even if other image already marked as active
* @return false if writing failed; true otherwise
*/
bool otaSwitch(bool force = false);
};
extern LibreTuya LT;
extern LibreTuya ESP;
#endif

View File

@@ -5,7 +5,7 @@
#include <api/WiFiClient.h>
#include <api/WiFiClientSecure.h>
#include <WiFiClient.h> // extend platform's WiFiClient impl
#include <WiFiClient.h> // extend family's WiFiClient impl
#ifdef __cplusplus
extern "C" {

View File

@@ -0,0 +1,199 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-29. */
#include "Update.h"
UpdateClass::UpdateClass() : ctx(NULL), info(NULL), buf(NULL) {
cleanup();
}
/**
* @brief Initialize the update process.
*
* @param size total UF2 file size
* @param command must be U_FLASH
* @return false if parameters are invalid or update is running, true otherwise
*/
bool UpdateClass::begin(size_t size, int command, int unused2, uint8_t unused3, const char *unused4) {
if (ctx)
return false;
cleanup();
ctx = uf2_ctx_init(LT.otaGetTarget(), FAMILY);
info = uf2_info_init();
if (!size)
return errorArd(UPDATE_ERROR_SIZE);
if (command != U_FLASH)
return errorArd(UPDATE_ERROR_BAD_ARGUMENT);
bytesTotal = size;
return true;
}
/**
* @brief Finalize the update process. Check for errors and update completion, then activate the new firmware image.
*
* @param evenIfRemaining no idea
* @return false in case of errors or no update running, true otherwise
*/
bool UpdateClass::end(bool evenIfRemaining) {
if (hasError() || !ctx)
// false if not running
return false;
if (!isFinished() && !evenIfRemaining) {
// abort if not finished
return errorArd(UPDATE_ERROR_ABORT);
}
// TODO what is evenIfRemaining for?
if (!LT.otaSwitch())
// try to activate the second OTA
return errorArd(UPDATE_ERROR_ACTIVATE);
cleanup();
return true;
}
/**
* @brief Write a chunk of data to the buffer or flash memory.
*
* It's advised to write in 512-byte chunks (or its multiples).
*
* @param data
* @param len
* @return size_t
*/
size_t UpdateClass::write(uint8_t *data, size_t len) {
size_t written = 0;
if (hasError() || !ctx)
// 0 if not running
return 0;
/* while (buf == bufPos && len >= UF2_BLOCK_SIZE) {
// buffer empty and entire block is in data
if (!tryWriteData(data, UF2_BLOCK_SIZE)) {
// returns 0 if data contains an invalid block
return written;
}
data += UF2_BLOCK_SIZE;
len -= UF2_BLOCK_SIZE;
written += UF2_BLOCK_SIZE;
} */
// write until buffer space is available
uint16_t toWrite;
while (len && (toWrite = min(len, bufLeft()))) {
tryWriteData(data, toWrite);
if (hasError())
// return on errors
return written;
data += toWrite;
len -= toWrite;
written += toWrite;
}
return written;
}
size_t UpdateClass::writeStream(Stream &data) {
size_t written = 0;
if (hasError() || !ctx)
// 0 if not running
return 0;
uint32_t lastData = millis();
// loop until the update is complete
while (remaining()) {
// check stream availability
int available = data.available();
if (available <= 0) {
if (millis() - lastData > UPDATE_TIMEOUT_MS) {
// waited for data too long; abort with error
errorArd(UPDATE_ERROR_STREAM);
return written;
}
continue;
}
// available > 0
lastData = millis();
// read data to fit in the remaining buffer space
bufAlloc();
uint16_t read = data.readBytes(bufPos, bufLeft());
bufPos += read;
written += read;
tryWriteData();
if (hasError())
// return on errors
return written;
}
}
/**
* @brief Try to use the buffer as a block to write. In case of UF2 errors,
* error codes are set, the update is aborted and 0 is returned
*
* @param data received data to copy to buffer or NULL if already in buffer
* @param len received data length - must be at most bufLeft()
* @return size_t "used" data size - 0 or 512
*/
size_t UpdateClass::tryWriteData(uint8_t *data, size_t len) {
uf2_block_t *block = NULL;
if (len == UF2_BLOCK_SIZE) {
// data has a complete block
block = (uf2_block_t *)data;
} else if (data && len) {
// data has a part of a block, copy it to buffer
bufAlloc();
memcpy(bufPos, data, len);
bufPos += len;
}
if (!block && bufSize() == UF2_BLOCK_SIZE) {
// use buffer as block (only if not found above)
block = (uf2_block_t *)buf;
}
// a complete block has been found
if (block) {
if (errorUf2(uf2_check_block(ctx, block)))
// block is invalid
return 0;
if (errUf2 == UF2_ERR_IGNORE)
// treat ignored blocks as valid
return UF2_BLOCK_SIZE;
if (!bytesWritten) {
// parse header block to allow retrieving firmware info
if (errorUf2(uf2_parse_header(ctx, block, info)))
// header is invalid
return 0;
if (bytesTotal == UPDATE_SIZE_UNKNOWN) {
// set total update size from block count info
bytesTotal = block->block_count * UF2_BLOCK_SIZE;
} else if (bytesTotal != block->block_count * UF2_BLOCK_SIZE) {
// given update size does not match the block count
return errorArd(UPDATE_ERROR_SIZE);
}
} else {
// write data blocks normally
if (errorUf2(uf2_write(ctx, block)))
// block writing failed
return 0;
}
// increment total writing progress
bytesWritten += UF2_BLOCK_SIZE;
// call progress callback
if (callback)
callback(bytesWritten, bytesTotal);
return UF2_BLOCK_SIZE;
}
return 0;
}
UpdateClass Update;

View File

@@ -0,0 +1,150 @@
#pragma once
#include <Arduino.h>
#include <functional>
#include "uf2ota/uf2ota.h"
// No Error
#define UPDATE_ERROR_OK (0)
// Flash Write Failed
#define UPDATE_ERROR_WRITE (1)
// Flash Erase Failed
#define UPDATE_ERROR_ERASE (2)
// Flash Read Failed
#define UPDATE_ERROR_READ (3)
// Not Enough Space
#define UPDATE_ERROR_SPACE (4)
// Bad Size Given
#define UPDATE_ERROR_SIZE (5)
// Stream Read Timeout
#define UPDATE_ERROR_STREAM (6)
// MD5 Check Failed
#define UPDATE_ERROR_MD5 (7)
// Wrong Magic Byte
#define UPDATE_ERROR_MAGIC_BYTE (8)
// Could Not Activate The Firmware
#define UPDATE_ERROR_ACTIVATE (9)
// Partition Could Not be Found
#define UPDATE_ERROR_NO_PARTITION (10)
// Bad Argument
#define UPDATE_ERROR_BAD_ARGUMENT (11)
// Aborted
#define UPDATE_ERROR_ABORT (12)
#define UPDATE_SIZE_UNKNOWN 0xFFFFFFFF
#define U_FLASH 0
#define U_SPIFFS 100
#define U_AUTH 200
#define ENCRYPTED_BLOCK_SIZE 16
#define UPDATE_TIMEOUT_MS 30 * 1000
class UpdateClass {
public:
typedef std::function<void(size_t, size_t)> THandlerFunction_Progress;
public: /* Update.cpp */
UpdateClass();
bool begin(
size_t size = UPDATE_SIZE_UNKNOWN,
int command = U_FLASH,
int unused2 = -1,
uint8_t unused3 = LOW,
const char *unused4 = NULL // this is for SPIFFS
);
bool end(bool evenIfRemaining = false);
size_t write(uint8_t *data, size_t len);
size_t writeStream(Stream &data);
bool canRollBack();
bool rollBack();
// bool setMD5(const char *expected_md5);
private: /* Update.cpp */
size_t tryWriteData(uint8_t *data = NULL, size_t len = 0);
public: /* UpdateUtil.cpp */
UpdateClass &onProgress(THandlerFunction_Progress callback);
void abort();
void printError(Print &out);
const char *errorString();
const char *getFirmwareName();
const char *getFirmwareVersion();
const char *getLibreTuyaVersion();
const char *getBoardName();
private: /* UpdateUtil.cpp */
void cleanup();
bool errorUf2(uf2_err_t err);
bool errorArd(uint8_t err);
void bufAlloc();
uint16_t bufLeft();
uint16_t bufSize();
private:
// uf2ota context
uf2_ota_t *ctx;
uf2_info_t *info;
// block buffer
uint8_t *buf;
uint8_t *bufPos;
// update progress - multiplies of 512 bytes
uint32_t bytesWritten;
uint32_t bytesTotal;
// errors
uf2_err_t errUf2;
uint8_t errArd;
// progress callback
THandlerFunction_Progress callback;
// String _target_md5;
// MD5Builder _md5;
public:
String md5String(void) {
// return _md5.toString();
}
void md5(uint8_t *result) {
// return _md5.getBytes(result);
}
uint8_t getError() {
return errArd;
}
uf2_err_t getUF2Error() {
return errUf2;
}
void clearError() {
errorUf2(UF2_ERR_OK);
}
bool hasError() {
return errArd != UPDATE_ERROR_OK;
}
bool isRunning() {
return ctx != NULL;
}
bool isFinished() {
return bytesWritten == bytesTotal;
}
size_t size() {
return bytesTotal;
}
size_t progress() {
return bytesWritten;
}
size_t remaining() {
return bytesTotal - bytesWritten;
}
};
extern UpdateClass Update;

View File

@@ -0,0 +1,162 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-30. */
#include "Update.h"
static const uint8_t errorMap[] = {
UPDATE_ERROR_OK, /* UF2_ERR_OK - no error */
UPDATE_ERROR_OK, /* UF2_ERR_IGNORE - block should be ignored */
UPDATE_ERROR_MAGIC_BYTE, /* UF2_ERR_MAGIC - wrong magic numbers */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_FAMILY - family ID mismatched */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_NOT_HEADER - block is not a header */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_OTA_VER - unknown/invalid OTA format version */
UPDATE_ERROR_MAGIC_BYTE, /* UF2_ERR_OTA_WRONG - no data for current OTA index */
UPDATE_ERROR_NO_PARTITION, /* UF2_ERR_PART_404 - no partition with that name */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_PART_ONE - only one partition tag in a block */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_PART_UNSET - attempted to write without target partition */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_DATA_TOO_LONG - data too long - tags won't fit */
UPDATE_ERROR_BAD_ARGUMENT, /* UF2_ERR_SEQ_MISMATCH - sequence number mismatched */
UPDATE_ERROR_ERASE, /* UF2_ERR_ERASE_FAILED - erasing flash failed */
UPDATE_ERROR_WRITE, /* UF2_ERR_WRITE_FAILED - writing to flash failed */
UPDATE_ERROR_WRITE /* UF2_ERR_WRITE_LENGTH - wrote fewer data than requested */
};
static char errorStr[14];
/**
* @brief Set the callback invoked after writing data to flash.
*/
UpdateClass &UpdateClass::onProgress(THandlerFunction_Progress callback) {
this->callback = callback;
return *this;
}
void UpdateClass::cleanup() {
free(ctx); // NULL in constructor
ctx = NULL;
uf2_info_free(info); // NULL in constructor
info = NULL;
free(buf); // NULL in constructor
buf = bufPos = NULL;
bytesWritten = 0;
bytesTotal = 0;
errUf2 = UF2_ERR_OK;
errArd = UPDATE_ERROR_OK;
}
/**
* @brief Check for UF2 errors. Set errArd and errUf2 in case of errors.
* Ignored blocks are not reported as errors.
* Abort the update.
* Use like: "if (errorUf2(...)) return false;"
* @return true if err is not OK, false otherwise
*/
bool UpdateClass::errorUf2(uf2_err_t err) {
if (err <= UF2_ERR_IGNORE)
return false;
cleanup();
errUf2 = err;
errArd = errorMap[err];
return true;
}
/**
* @brief Set errUf2 and errArd according to given Arduino error code.
* Abort the update.
* Use like: "return errorArd(...);"
* @return false - always
*/
bool UpdateClass::errorArd(uint8_t err) {
cleanup();
errUf2 = UF2_ERR_OK;
errArd = err;
return false;
}
/**
* @brief Abort the update with UPDATE_ERROR_ABORT reason.
*/
void UpdateClass::abort() {
errorArd(UPDATE_ERROR_ABORT);
}
void UpdateClass::bufAlloc() {
if (!buf)
buf = bufPos = (uint8_t *)malloc(UF2_BLOCK_SIZE);
}
uint16_t UpdateClass::bufLeft() {
return buf + UF2_BLOCK_SIZE - bufPos;
}
uint16_t UpdateClass::bufSize() {
return bufPos - buf;
}
/**
* @brief Print string error info to the stream.
*/
void UpdateClass::printError(Print &out) {
out.println(errorString());
}
/**
* @brief Get string representation of the error in format
* "ard=..,uf2=..". Returns "" if no error.
*/
const char *UpdateClass::errorString() {
if (!errArd)
return "";
sprintf(errorStr, "ard=%u,uf2=%u", errArd, errUf2);
return errorStr;
}
/**
* @brief Get firmware name from UF2 info.
*/
const char *UpdateClass::getFirmwareName() {
if (info)
return info->fw_name;
return NULL;
}
/**
* @brief Get firmware version from UF2 info.
*/
const char *UpdateClass::getFirmwareVersion() {
if (info)
return info->fw_version;
return NULL;
}
/**
* @brief Get LibreTuya version from UF2 info.
*/
const char *UpdateClass::getLibreTuyaVersion() {
if (info)
return info->lt_version;
return NULL;
}
/**
* @brief Get target board name from UF2 info.
*/
const char *UpdateClass::getBoardName() {
if (info)
return info->board;
return NULL;
}
/**
* @brief See LT.otaCanRollback() for more info.
*/
bool UpdateClass::canRollBack() {
return LT.otaCanRollback();
}
/**
* @brief See LT.otaRollback() for more info.
*/
bool UpdateClass::rollBack() {
return LT.otaRollback();
}

View File

@@ -0,0 +1,32 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-29. */
#include "uf2priv.h"
uf2_err_t uf2_binpatch(uint8_t *data, const uint8_t *binpatch, uint8_t binpatch_len) {
const uint8_t *binpatch_end = binpatch + binpatch_len;
// +2 to make sure opcode and length is present
while ((binpatch + 2) < binpatch_end) {
uf2_opcode_t opcode = binpatch[0];
uint8_t len = binpatch[1];
switch (opcode) {
case UF2_OPC_DIFF32:
uf2_binpatch_diff32(data, binpatch + 1);
break;
}
// advance by opcode + length + data
binpatch += len + 2;
}
return UF2_ERR_OK;
}
void uf2_binpatch_diff32(uint8_t *data, const uint8_t *patch) {
uint8_t num_offs = patch[0] - 4; // read offset count
uint32_t diff = *((uint32_t *)(patch + 1)); // read diff value
patch += 5; // skip num_offs and diff value
for (uint8_t i = 0; i < num_offs; i++) {
// patch the data
uint8_t offs = patch[i];
uint32_t *value = (uint32_t *)(data + offs);
*(value) += diff;
}
}

View File

@@ -0,0 +1,26 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-29. */
#pragma once
#include "uf2types.h"
/**
* @brief Apply binary patch to data.
*
* @param data input data
* @param data_len input data length
* @param binpatch binary patch data
* @param binpatch_len binary patch data length
* @return uf2_err_t error code
*/
uf2_err_t uf2_binpatch(uint8_t *data, const uint8_t *binpatch, uint8_t binpatch_len);
/**
* Apply DIFF32 binary patch.
*
* @param data input data
* @param len input data length
* @param patch patch data, incl. length byte
* @return uf2_err_t error code
*/
void uf2_binpatch_diff32(uint8_t *data, const uint8_t *patch);

View File

@@ -0,0 +1,100 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-29. */
#include "uf2priv.h"
uf2_ota_t *uf2_ctx_init(uint8_t ota_idx, uint32_t family_id) {
uf2_ota_t *ctx = (uf2_ota_t *)zalloc(sizeof(uf2_ota_t));
ctx->ota_idx = ota_idx;
ctx->family_id = family_id;
return ctx;
}
uf2_info_t *uf2_info_init() {
uf2_info_t *info = (uf2_info_t *)zalloc(sizeof(uf2_info_t));
return info;
}
void uf2_info_free(uf2_info_t *info) {
if (!info)
return;
free(info->fw_name);
free(info->fw_version);
free(info->lt_version);
free(info->board);
free(info);
}
uf2_err_t uf2_check_block(uf2_ota_t *ctx, uf2_block_t *block) {
if (block->magic1 != UF2_MAGIC_1)
return UF2_ERR_MAGIC;
if (block->magic2 != UF2_MAGIC_2)
return UF2_ERR_MAGIC;
if (block->magic3 != UF2_MAGIC_3)
return UF2_ERR_MAGIC;
if (block->file_container)
// ignore file containers, for now
return UF2_ERR_IGNORE;
if (!block->has_family_id || block->file_size != ctx->family_id)
// require family_id
return UF2_ERR_FAMILY;
return UF2_ERR_OK;
}
uf2_err_t uf2_parse_header(uf2_ota_t *ctx, uf2_block_t *block, uf2_info_t *info) {
if (!block->has_tags || block->file_container || block->len)
// header must have tags and no data
return UF2_ERR_NOT_HEADER;
uf2_err_t err = uf2_parse_block(ctx, block, info);
if (err)
return err;
if ((ctx->ota_idx == 1 && !ctx->has_ota1) || !ctx->has_ota2)
return UF2_ERR_OTA_WRONG;
return UF2_ERR_OK;
}
uf2_err_t uf2_write(uf2_ota_t *ctx, uf2_block_t *block) {
if (ctx->seq == 0)
return uf2_parse_header(ctx, block, NULL);
if (block->not_main_flash || !block->len)
// ignore blocks not meant for flashing
return UF2_ERR_IGNORE;
uf2_err_t err = uf2_parse_block(ctx, block, NULL);
if (err)
return err;
if (!ctx->part1 && !ctx->part2)
// no partitions set at all
return UF2_ERR_PART_UNSET;
fal_partition_t part = uf2_get_target_part(ctx);
if (!part)
// image is not for current OTA scheme
return UF2_ERR_IGNORE;
if (ctx->ota_idx == 2 && ctx->binpatch_len) {
// apply binpatch
err = uf2_binpatch(block->data, ctx->binpatch, ctx->binpatch_len);
if (err)
return err;
}
int ret;
// erase sectors if needed
if (!uf2_is_erased(ctx, block->addr, block->len)) {
ret = fal_partition_erase(part, block->addr, block->len);
if (ret < 0)
return UF2_ERR_ERASE_FAILED;
ctx->erased_offset = block->addr;
ctx->erased_length = ret;
}
// write data to flash
ret = fal_partition_write(part, block->addr, block->data, block->len);
if (ret < 0)
return UF2_ERR_WRITE_FAILED;
if (ret != block->len)
return UF2_ERR_WRITE_LENGTH;
return UF2_ERR_OK;
}

View File

@@ -0,0 +1,68 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-28. */
#pragma once
#ifdef __cplusplus
extern "C" {
#endif // __cplusplus
#include "uf2types.h"
/**
* @brief Create an UF2 OTA context.
*
* @param ota_idx target OTA index
* @param family_id expected family ID
* @return uf2_ota_t* heap-allocated structure
*/
uf2_ota_t *uf2_ctx_init(uint8_t ota_idx, uint32_t family_id);
/**
* @brief Create an UF2 Info structure.
*
* @return uf2_info_t* heap-allocated structure
*/
uf2_info_t *uf2_info_init();
/**
* @brief Free values in the info structure AND the structure itself.
*
* @param info structure to free; may be NULL
*/
void uf2_info_free(uf2_info_t *info);
/**
* @brief Check if block is valid.
*
* @param ctx context
* @param block block to check
* @return uf2_err_t error code; UF2_ERR_OK and UF2_ERR_IGNORE denote valid blocks
*/
uf2_err_t uf2_check_block(uf2_ota_t *ctx, uf2_block_t *block);
/**
* @brief Parse header block (LibreTuya UF2 first block).
*
* Note: caller should call uf2_check_block() first.
*
* @param ctx context
* @param block block to parse
* @param info structure to write firmware info, NULL if not used
* @return uf2_err_t error code
*/
uf2_err_t uf2_parse_header(uf2_ota_t *ctx, uf2_block_t *block, uf2_info_t *info);
/**
* @brief Write the block to flash memory.
*
* Note: caller should call uf2_check_block() first.
*
* @param ctx context
* @param block block to write
* @return uf2_err_t error code
*/
uf2_err_t uf2_write(uf2_ota_t *ctx, uf2_block_t *block);
#ifdef __cplusplus
} // extern "C"
#endif

View File

@@ -0,0 +1,146 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-29. */
#include "uf2priv.h"
uf2_err_t uf2_parse_block(uf2_ota_t *ctx, uf2_block_t *block, uf2_info_t *info) {
if (block->block_seq != ctx->seq)
// sequence number must match
return UF2_ERR_SEQ_MISMATCH;
ctx->seq++; // increment sequence number after checking it
if (!block->has_tags)
// no tags in this block, no further processing needed
return UF2_ERR_OK;
if (block->len > (476 - 4 - 4))
// at least one tag + last tag must fit
return UF2_ERR_DATA_TOO_LONG;
uint8_t *tags_start = block->data + block->len;
uint8_t tags_len = 476 - block->len;
uint8_t tags_pos = 0;
if (block->has_md5)
tags_len -= 24;
ctx->binpatch_len = 0; // binpatch applies to one block only
char *part1 = NULL;
char *part2 = NULL;
uf2_tag_type_t type;
while (tags_pos < tags_len) {
uint8_t len = uf2_read_tag(tags_start + tags_pos, &type);
if (!len)
break;
tags_pos += 4; // skip tag header
uint8_t *tag = tags_start + tags_pos;
char **str_dest = NULL; // char* to copy the tag into
switch (type) {
case UF2_TAG_OTA_VERSION:
if (tag[0] != 1)
return UF2_ERR_OTA_VER;
break;
case UF2_TAG_FIRMWARE:
if (info)
str_dest = &(info->fw_name);
break;
case UF2_TAG_VERSION:
if (info)
str_dest = &(info->fw_version);
break;
case UF2_TAG_LT_VERSION:
if (info)
str_dest = &(info->lt_version);
break;
case UF2_TAG_BOARD:
if (info)
str_dest = &(info->board);
break;
case UF2_TAG_HAS_OTA1:
ctx->has_ota1 = tag[0];
break;
case UF2_TAG_HAS_OTA2:
ctx->has_ota2 = tag[0];
break;
case UF2_TAG_PART_1:
str_dest = &(part1);
break;
case UF2_TAG_PART_2:
str_dest = &(part2);
break;
case UF2_TAG_BINPATCH:
ctx->binpatch = tag;
ctx->binpatch_len = len;
break;
default:
break;
}
if (str_dest) {
*str_dest = (char *)zalloc(len + 1);
memcpy(*str_dest, tag, len);
}
// align position to 4 bytes
tags_pos += (((len - 1) / 4) + 1) * 4;
}
if (part1 && part2) {
// update current target partition
uf2_err_t err = uf2_update_parts(ctx, part1, part2);
if (err)
return err;
} else if (part1 || part2) {
// only none or both partitions can be specified
return UF2_ERR_PART_ONE;
}
return UF2_ERR_OK;
}
uint8_t uf2_read_tag(const uint8_t *data, uf2_tag_type_t *type) {
uint8_t len = data[0];
if (!len)
return 0;
uint32_t tag_type = *((uint32_t *)data);
if (!tag_type)
return 0;
*type = tag_type >> 8; // remove tag length byte
return len - 4;
}
uf2_err_t uf2_update_parts(uf2_ota_t *ctx, char *part1, char *part2) {
// reset both target partitions
ctx->part1 = NULL;
ctx->part2 = NULL;
// reset offsets as they probably don't apply to this partition
ctx->erased_offset = 0;
ctx->erased_length = 0;
if (part1[0]) {
ctx->part1 = fal_partition_find(part1);
if (!ctx->part1)
return UF2_ERR_PART_404;
}
if (part2[0]) {
ctx->part2 = fal_partition_find(part2);
if (!ctx->part2)
return UF2_ERR_PART_404;
}
return UF2_ERR_OK;
}
fal_partition_t uf2_get_target_part(uf2_ota_t *ctx) {
if (ctx->ota_idx == 1)
return ctx->part1;
if (ctx->ota_idx == 2)
return ctx->part2;
return NULL;
}
bool uf2_is_erased(uf2_ota_t *ctx, uint32_t offset, uint32_t length) {
uint32_t erased_end = ctx->erased_offset + ctx->erased_length;
uint32_t end = offset + length;
return (offset >= ctx->erased_offset) && (end <= erased_end);
}

View File

@@ -0,0 +1,61 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-28. */
#pragma once
// include family stdlib APIs
#include <WVariant.h>
#include "uf2binpatch.h"
#include "uf2types.h"
/**
* @brief Parse a block and extract information from tags.
*
* @param ctx context
* @param block block to parse
* @param info structure to write firmware info, NULL if not used
* @return uf2_err_t error code
*/
uf2_err_t uf2_parse_block(uf2_ota_t *ctx, uf2_block_t *block, uf2_info_t *info);
/**
* @brief Parse a tag.
*
* @param data pointer to tag header beginning
* @param type [out] parsed tag type
* @return uint8_t parsed tag data length (excl. header); 0 if invalid/last tag
*/
uint8_t uf2_read_tag(const uint8_t *data, uf2_tag_type_t *type);
/**
* @brief Update destination partitions in context.
*
* Partition names cannot be NULL.
*
* Returns UF2_ERR_IGNORE if specified partitions don't match the
* current OTA index.
*
* @param ctx context
* @param part1 partition 1 name or empty string
* @param part2 partition 2 name or empty string
* @return uf2_err_t error code
*/
uf2_err_t uf2_update_parts(uf2_ota_t *ctx, char *part1, char *part2);
/**
* @brief Get target flashing partition, depending on OTA index.
*
* @param ctx context
* @return fal_partition_t target partition or NULL if not set
*/
fal_partition_t uf2_get_target_part(uf2_ota_t *ctx);
/**
* Check if specified flash memory region was already erased during update.
*
* @param ctx context
* @param offset offset to check
* @param length length to check
* @return bool true/false
*/
bool uf2_is_erased(uf2_ota_t *ctx, uint32_t offset, uint32_t length);

View File

@@ -0,0 +1,104 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-28. */
#pragma once
#include <stdbool.h>
#include <stdint.h>
#include <fal.h>
#define UF2_MAGIC_1 0x0A324655
#define UF2_MAGIC_2 0x9E5D5157
#define UF2_MAGIC_3 0x0AB16F30
#define UF2_BLOCK_SIZE sizeof(uf2_block_t)
typedef struct __attribute__((packed)) {
// 32 byte header
uint32_t magic1;
uint32_t magic2;
// flags split as bitfields
bool not_main_flash : 1;
uint16_t dummy1 : 11;
bool file_container : 1;
bool has_family_id : 1;
bool has_md5 : 1;
bool has_tags : 1;
uint16_t dummy2 : 16;
uint32_t addr;
uint32_t len;
uint32_t block_seq;
uint32_t block_count;
uint32_t file_size; // or familyID;
uint8_t data[476];
uint32_t magic3;
} uf2_block_t;
typedef struct {
uint32_t seq; // current block sequence number
uint8_t *binpatch; // current block's binpatch (if any) -> pointer inside block->data
uint8_t binpatch_len; // binpatch length
bool has_ota1; // image has any data for OTA1
bool has_ota2; // image has any data for OTA2
uint8_t ota_idx; // target OTA index
uint32_t family_id; // expected family ID
uint32_t erased_offset; // offset of region erased during update
uint32_t erased_length; // length of erased region
fal_partition_t part1; // OTA1 target partition
fal_partition_t part2; // OTA2 target partition
} uf2_ota_t;
typedef struct {
char *fw_name;
char *fw_version;
char *lt_version;
char *board;
} uf2_info_t;
typedef enum {
UF2_TAG_VERSION = 0x9FC7BC, // version of firmware file - UTF8 semver string
UF2_TAG_PAGE_SIZE = 0x0BE9F7, // page size of target device (32 bit unsigned number)
UF2_TAG_SHA2 = 0xB46DB0, // SHA-2 checksum of firmware (can be of various size)
UF2_TAG_DEVICE = 0x650D9D, // description of device (UTF8)
UF2_TAG_DEVICE_ID = 0xC8A729, // device type identifier
// LibreTuya custom, tags
UF2_TAG_OTA_VERSION = 0x5D57D0, // format version
UF2_TAG_BOARD = 0xCA25C8, // board name (lowercase code)
UF2_TAG_FIRMWARE = 0x00DE43, // firmware description / name
UF2_TAG_BUILD_DATE = 0x822F30, // build date/time as Unix timestamp
UF2_TAG_LT_VERSION = 0x59563D, // LT version (semver)
UF2_TAG_PART_1 = 0x805946, // OTA1 partition name
UF2_TAG_PART_2 = 0xA1E4D7, // OTA2 partition name
UF2_TAG_HAS_OTA1 = 0xBBD965, // image has any data for OTA1
UF2_TAG_HAS_OTA2 = 0x92280E, // image has any data for OTA2
UF2_TAG_BINPATCH = 0xB948DE, // binary patch to convert OTA1->OTA2
} uf2_tag_type_t;
typedef enum {
UF2_OPC_DIFF32 = 0xFE,
} uf2_opcode_t;
typedef enum {
UF2_ERR_OK = 0,
UF2_ERR_IGNORE, // block should be ignored
UF2_ERR_MAGIC, // wrong magic numbers
UF2_ERR_FAMILY, // family ID mismatched
UF2_ERR_NOT_HEADER, // block is not a header
UF2_ERR_OTA_VER, // unknown/invalid OTA format version
UF2_ERR_OTA_WRONG, // no data for current OTA index
UF2_ERR_PART_404, // no partition with that name
UF2_ERR_PART_ONE, // only one partition tag in a block
UF2_ERR_PART_UNSET, // image broken - attempted to write without target partition
UF2_ERR_DATA_TOO_LONG, // data too long - tags won't fit
UF2_ERR_SEQ_MISMATCH, // sequence number mismatched
UF2_ERR_ERASE_FAILED, // erasing flash failed
UF2_ERR_WRITE_FAILED, // writing to flash failed
UF2_ERR_WRITE_LENGTH, // wrote fewer data than requested
} uf2_err_t;

View File

@@ -9,7 +9,7 @@ extern "C" {
#include <lwip/netif.h>
}
extern u8_t mdns_netif_client_id;
static u8_t mdns_netif_client_id = 0; // TODO fix this
struct mdns_domain {
/* Encoded domain name */
@@ -60,6 +60,8 @@ bool mDNS::begin(const char *hostname) {
mdns_resp_init();
struct netif *netif = netif_list;
while (netif != NULL) {
// TODO: detect mdns_netif_client_id by checking netif_get_client_data()
// and finding the requested hostname in struct mdns_host
if (netif_is_up(netif) && mdns_resp_add_netif(netif, hostname, 255) != ERR_OK) {
return false;
}

View File

@@ -0,0 +1,25 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-24. */
#pragma once
// Flash device configuration
extern const struct fal_flash_dev flash0;
#define FAL_FLASH_DEV_NAME "flash0"
#define FAL_FLASH_DEV_TABLE \
{ &flash0, }
#define FAL_DEV_NAME_MAX 16 // no need for 24 chars (default)
// Partition table
#define FAL_PART_HAS_TABLE_CFG
#define FAL_PART_TABLE_ITEM(part_lower, part_upper) \
{ \
.magic_word = FAL_PART_MAGIC_WORD, /* magic word */ \
.name = #part_lower, /* lowercase name as string */ \
.flash_name = FAL_FLASH_DEV_NAME, /* flash device name */ \
.offset = FLASH_##part_upper##_OFFSET, /* partition offset macro as uppercase string */ \
.len = FLASH_##part_upper##_LENGTH, /* partition length macro as uppercase string */ \
},

View File

@@ -0,0 +1,45 @@
/*
* Copyright (c) 2020, Armink, <armink.ztl@gmail.com>
*
* SPDX-License-Identifier: Apache-2.0
*/
#ifndef _FDB_CFG_H_
#define _FDB_CFG_H_
/* using KVDB feature */
#define FDB_USING_KVDB
#ifdef FDB_USING_KVDB
/* Auto update KV to latest default when current KVDB version number is changed. @see fdb_kvdb.ver_num */
// #define FDB_KV_AUTO_UPDATE
#endif
/* using TSDB (Time series database) feature */
// #define FDB_USING_TSDB
/* Using FAL storage mode */
#define FDB_USING_FAL_MODE
#ifdef FDB_USING_FAL_MODE
/* the flash write granularity, unit: bit
* only support 1(nor flash)/ 8(stm32f2/f4)/ 32(stm32f1) */
#define FDB_WRITE_GRAN 8
#endif
/* Using file storage mode by LIBC file API, like fopen/fread/fwrte/fclose */
// #define FDB_USING_FILE_LIBC_MODE
/* Using file storage mode by POSIX file API, like open/read/write/close */
// #define FDB_USING_FILE_POSIX_MODE
/* MCU Endian Configuration, default is Little Endian Order. */
// #define FDB_BIG_ENDIAN
/* log print macro. default EF_PRINT macro is printf() */
#define FDB_PRINT(...)
/* print debug information */
// #define FDB_DEBUG_ENABLE
#endif /* _FDB_CFG_H_ */

View File

@@ -23,7 +23,7 @@ extern uint32_t SystemCoreClock;
#define interrupts() vPortClearInterruptMask(0)
#define noInterrupts() ulPortSetInterruptMask()
// Include platform-specific code
// Include family-specific code
#include "WVariant.h"
// Include board variant
#include "variant.h"

View File

@@ -0,0 +1,145 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-28. */
#include <LibreTuyaAPI.h>
extern "C" {
#include <flash_api.h>
#include <rtl8710b.h>
#include <sys_api.h>
}
void LibreTuya::restart() {
sys_reset();
}
/* CPU-related */
ChipType LibreTuya::getChipType() {
uint8_t chipId;
EFUSE_OneByteReadROM(9902, 0xF8, &chipId, L25EOUTVOLTAGE);
return (ChipType)(((RTL8710B >> 24) << 8) | chipId);
}
const char *LibreTuya::getChipModel() {
return STRINGIFY_MACRO(MCU);
}
uint32_t LibreTuya::getChipId() {
uint32_t chipId = 0;
uint8_t *id = (uint8_t *)&chipId;
// 9902 was extracted from ROM disassembly, probably not needed
EFUSE_OneByteReadROM(9902, 0x3B, id + 0, L25EOUTVOLTAGE);
EFUSE_OneByteReadROM(9902, 0x3C, id + 1, L25EOUTVOLTAGE);
EFUSE_OneByteReadROM(9902, 0x3D, id + 2, L25EOUTVOLTAGE);
return chipId;
}
uint8_t LibreTuya::getChipCores() {
return 1;
}
const char *LibreTuya::getChipCoreType() {
return "ARM Cortex-M4F";
}
uint32_t LibreTuya::getCpuFreqMHz() {
return CPU_ClkGet(false) / 1000000;
}
inline uint32_t LibreTuya::getCycleCount() {
return microsecondsToClockCycles(micros());
}
/* Memory management */
uint32_t LibreTuya::getRamSize() {
return 256 * 1024;
}
uint32_t LibreTuya::getHeapSize() {
return configTOTAL_HEAP_SIZE;
}
uint32_t LibreTuya::getFreeHeap() {
return xPortGetFreeHeapSize();
}
uint32_t LibreTuya::getMinFreeHeap() {
return xPortGetMinimumEverFreeHeapSize();
}
uint32_t LibreTuya::getMaxAllocHeap() {
return 0;
}
/* OTA-related */
uint8_t LibreTuya::otaGetStoredIndex() {
uint32_t *otaAddress = (uint32_t *)0x8009000;
if (*otaAddress == 0xFFFFFFFF)
return 1;
uint32_t otaCounter = *((uint32_t *)0x8009004);
// even count of zero-bits means OTA1, odd count means OTA2
// this allows to switch OTA images by simply clearing next bits,
// without needing to erase the flash
uint8_t count = 0;
for (uint8_t i = 0; i < 32; i++) {
if ((otaCounter & (1 << i)) == 0)
count++;
}
return 1 + (count % 2);
}
bool LibreTuya::otaSupportsDual() {
return true;
}
bool LibreTuya::otaHasImage1() {
uint8_t *ota1Addr = (uint8_t *)(SPI_FLASH_BASE + FLASH_OTA1_OFFSET);
return memcmp(ota1Addr, "81958711", 8) == 0;
}
bool LibreTuya::otaHasImage2() {
uint8_t *ota2Addr = (uint8_t *)(SPI_FLASH_BASE + FLASH_OTA2_OFFSET);
return memcmp(ota2Addr, "81958711", 8) == 0;
}
bool LibreTuya::otaSwitch(bool force) {
if (!force && otaGetRunning() != otaGetStoredIndex())
// OTA has already been switched
return true;
// this function does:
// - read OTA1 firmware magic from 0xB000
// - read OTA2 address from 0x9000
// - read OTA2 firmware magic from that address
// - read current OTA switch value from 0x9004
// - reset OTA switch to 0xFFFFFFFF if it's 0x0
// - check first non-zero bit of OTA switch
// - write OTA switch with first non-zero bit cleared
// sys_clear_ota_signature();
// ok, this function is broken (crashes with HardFault)
if (!otaHasImage1() || !otaHasImage2())
return false;
uint32_t value = HAL_READ32(SPI_FLASH_BASE, FLASH_SYSTEM_OFFSET + 4);
if (value == 0) {
// TODO does this work at all?
FLASH_EreaseDwordsXIP(FLASH_SYSTEM_OFFSET + 4, 1);
}
uint8_t i;
// find first non-zero bit
for (i = 0; i < 32; i++) {
if (value & (1 << i))
break;
}
// clear the bit
value &= ~(1 << i);
// write OTA switch to flash
flash_write_word(NULL, FLASH_SYSTEM_OFFSET + 4, value);
return true;
}
/* Global instance */
LibreTuya LT;

View File

@@ -68,7 +68,7 @@ uint32_t millis(void) {
uint32_t micros(void) {
uint32_t tick1, tick2;
uint32_t us;
uint32_t tick_per_us = 166666;
uint32_t tick_per_us = F_CPU / 1000;
if (__get_ipsr__() == 0) {
tick1 = xTaskGetTickCount();

View File

@@ -74,21 +74,20 @@ void WiFiClass::printDiag(Print &dest) {
}
WiFiAuthMode WiFiClass::securityTypeToAuthMode(uint8_t type) {
switch (wifi_setting.security_type) {
case RTW_SECURITY_OPEN:
// the value reported in rtw_scan_result is rtw_encryption_t, even though it's rtw_security_t in the header file
switch (type) {
case RTW_ENCRYPTION_OPEN:
return WIFI_AUTH_OPEN;
case RTW_SECURITY_WEP_SHARED:
case RTW_ENCRYPTION_WEP40:
case RTW_ENCRYPTION_WEP104:
return WIFI_AUTH_WEP;
case RTW_SECURITY_WPA_TKIP_PSK:
case RTW_ENCRYPTION_WPA_TKIP:
case RTW_ENCRYPTION_WPA_AES:
return WIFI_AUTH_WPA_PSK;
case RTW_SECURITY_WPA_AES_PSK:
return WIFI_AUTH_WPA;
case RTW_SECURITY_WPA2_TKIP_PSK:
case RTW_ENCRYPTION_WPA2_TKIP:
case RTW_ENCRYPTION_WPA2_AES:
case RTW_ENCRYPTION_WPA2_MIXED:
return WIFI_AUTH_WPA2_PSK;
case RTW_SECURITY_WPA2_AES_PSK:
return WIFI_AUTH_WPA2;
case RTW_SECURITY_WPA_WPA2_MIXED:
return WIFI_AUTH_WPA_WPA2_PSK;
}
return WIFI_AUTH_INVALID;
}

View File

@@ -10,6 +10,8 @@
static xQueueHandle wifiEventQueueHandle = NULL;
static xTaskHandle wifiEventTaskHandle = NULL;
WiFiClass *pWiFi = NULL;
// C code to support SDK-defined events (in wifi_conf.c)
extern "C" {
// SDK events
@@ -133,6 +135,9 @@ void WiFiClass::handleRtwEvent(uint16_t event, char *data, int len, int flags) {
handler(data, len, flags, event_callback_list[event][i].handler_user_data);
}
if (pWiFi == NULL)
return;
EventId eventId;
EventInfo eventInfo;
String ssid;
@@ -155,18 +160,18 @@ void WiFiClass::handleRtwEvent(uint16_t event, char *data, int len, int flags) {
case WIFI_EVENT_FOURWAY_HANDSHAKE_DONE:
eventId = ARDUINO_EVENT_WIFI_STA_CONNECTED;
ssid = WiFi.SSID();
ssid = pWiFi->SSID();
eventInfo.wifi_sta_connected.ssid_len = ssid.length();
eventInfo.wifi_sta_connected.channel = WiFi.channel();
eventInfo.wifi_sta_connected.authmode = WiFi.getEncryption();
eventInfo.wifi_sta_connected.channel = pWiFi->channel();
eventInfo.wifi_sta_connected.authmode = pWiFi->getEncryption();
memcpy(eventInfo.wifi_sta_connected.ssid, ssid.c_str(), eventInfo.wifi_sta_connected.ssid_len + 1);
memcpy(eventInfo.wifi_sta_connected.bssid, WiFi.BSSID(), 6);
memcpy(eventInfo.wifi_sta_connected.bssid, pWiFi->BSSID(), 6);
break;
case WIFI_EVENT_SCAN_DONE:
eventId = ARDUINO_EVENT_WIFI_SCAN_DONE;
eventInfo.wifi_scan_done.status = 0;
eventInfo.wifi_scan_done.number = WiFi._netCount;
eventInfo.wifi_scan_done.number = pWiFi->_netCount;
eventInfo.wifi_scan_done.scan_id = 0;
break;

View File

@@ -9,9 +9,13 @@ int32_t WiFiClass::channel() {
return channel;
}
extern WiFiClass *pWiFi;
extern void startWifiTask();
bool WiFiClass::mode(WiFiMode mode) {
// store a pointer to WiFi for WiFiEvents.cpp
pWiFi = this;
WiFiMode currentMode = getMode();
LT_D_WG("Mode changing %u -> %u", currentMode, mode);
if (mode == currentMode)

View File

@@ -0,0 +1,31 @@
/* Copyright (c) Kuba Szczodrzyński 2022-05-24. */
#include <fal.h>
#include <flash_api.h>
#define FLASH_ERASE_MIN_SIZE (4 * 1024)
static int read(long offset, uint8_t *buf, size_t size) {
return size * flash_stream_read(NULL, offset, size, buf);
}
static int write(long offset, const uint8_t *buf, size_t size) {
return size * flash_stream_write(NULL, offset, size, buf);
}
static int erase(long offset, size_t size) {
size = ((size - 1) / FLASH_ERASE_MIN_SIZE) + 1;
for (uint16_t i = 0; i < size; i++) {
flash_erase_sector(NULL, offset + i * FLASH_ERASE_MIN_SIZE);
}
return size * FLASH_ERASE_MIN_SIZE;
}
const struct fal_flash_dev flash0 = {
.name = FAL_FLASH_DEV_NAME,
.addr = 0x0,
.len = FLASH_LENGTH,
.blk_size = FLASH_ERASE_MIN_SIZE,
.ops = {NULL, read, write, erase},
.write_gran = 1,
};

View File

@@ -11,7 +11,8 @@
"calibration": "0x00A000+0x1000",
"ota1": "0x00B000+0xC5000",
"ota2": "0x0D0000+0xC5000",
"userdata": "0x195000+0x6A000",
"kvs": "0x195000+0x6000",
"userdata": "0x19B000+0x64000",
"rdp": "0x1FF000+0x1000"
},
"upload": {

View File

@@ -1,5 +1,6 @@
{
"build": {
"family": "RTL8710B",
"f_cpu": "125000000L",
"amb_flash_addr": "0x08000000"
},

View File

@@ -6,7 +6,6 @@
],
"build": {
"mcu": "rtl8710bn",
"family": "rtl8710",
"variant": "wr3"
},
"name": "WR3 Wi-Fi Module",

View File

@@ -38,8 +38,8 @@ D7 | PA05 | | | | PWM4 |
D8 | PA12 | | | | PWM3 |
D9 | PA18 | UART0_RX | I2C1_SCL | SPI0_SCK, SPI1_SCK | |
D10 | PA23 | UART0_TX | I2C1_SDA | SPI0_MOSI, SPI1_MOSI | PWM0 |
A0 | PA19, ADC1
A1 | ADC2
A0 | PA19, ADC1 | | | | |
A1 | ADC2 | | | | |
## Flash memory map
@@ -47,17 +47,18 @@ Flash size: 2 MiB / 2,097,152 B / 0x200000
Hex values are in bytes.
Name | Start | Length | End
------------|----------|-------------------|---------
Boot XIP | 0x000000 | 16 KiB / 0x4000 | 0x004000
Boot RAM | 0x004000 | 16 KiB / 0x4000 | 0x008000
(reserved) | 0x008000 | 4 KiB / 0x1000 | 0x009000
System Data | 0x009000 | 4 KiB / 0x1000 | 0x00A000
Calibration | 0x00A000 | 4 KiB / 0x1000 | 0x00B000
OTA1 Image | 0x00B000 | 788 KiB / 0xC5000 | 0x0D0000
OTA2 Image | 0x0D0000 | 788 KiB / 0xC5000 | 0x195000
User Data | 0x195000 | 424 KiB / 0x6A000 | 0x1FF000
RDP | 0x1FF000 | 4 KiB / 0x1000 | 0x200000
Name | Start | Length | End
----------------|----------|-------------------|---------
Boot XIP | 0x000000 | 16 KiB / 0x4000 | 0x004000
Boot RAM | 0x004000 | 16 KiB / 0x4000 | 0x008000
(reserved) | 0x008000 | 4 KiB / 0x1000 | 0x009000
System Data | 0x009000 | 4 KiB / 0x1000 | 0x00A000
Calibration | 0x00A000 | 4 KiB / 0x1000 | 0x00B000
OTA1 Image | 0x00B000 | 788 KiB / 0xC5000 | 0x0D0000
OTA2 Image | 0x0D0000 | 788 KiB / 0xC5000 | 0x195000
Key-Value Store | 0x195000 | 24 KiB / 0x6000 | 0x19B000
User Data | 0x19B000 | 400 KiB / 0x64000 | 0x1FF000
RDP | 0x1FF000 | 4 KiB / 0x1000 | 0x200000
RDP is most likely not used in Tuya firmwares, as the System Data partition contains an incorrect offset 0xFF000 for RDP, which is in the middle of OTA2 image.

View File

@@ -39,6 +39,7 @@ env.AddLibrary(
"+<common/*.c*>",
"+<core/*.c*>",
"+<libraries/**/*.c*>",
"+<port/**/*.c*>",
"+<posix/*.c>",
],
includes=[
@@ -46,9 +47,13 @@ env.AddLibrary(
"!<compat>",
"!<core>",
"!<libraries/*>",
"!<port/*>",
"!<posix>",
],
)
# Sources - external library ports
env.AddLibraryFlashDB(version="03500fa")
# Build all libraries
env.BuildLibraries(safe=False)

View File

@@ -45,7 +45,6 @@ env.Append(
"ARDUINO_AMEBA",
"ARDUINO_SDK",
"ARDUINO_ARCH_AMBZ",
"BOARD_${FAMILY}",
# the SDK declares bool if not defined before
# which conflicts with C++ built-in bool
# so it's either -fpermissive or this:
@@ -112,6 +111,18 @@ env.AddLibrary(
],
)
# Sources - external library ports
env.AddLibrary(
name="ambz_arduino_port",
base_dir="$ARDUINO_DIR",
srcs=[
"+<port/**/*.c*>",
],
includes=[
"+<port/*>",
],
)
# Libs & linker config
env.Append(
LIBS=[

View File

@@ -1,6 +1,5 @@
# Copyright (c) Kuba Szczodrzyński 2022-04-20.
import sys
from os.path import join
from SCons.Script import Builder, DefaultEnvironment
@@ -10,30 +9,6 @@ board = env.BoardConfig()
env.AddDefaults("realtek-ambz", "framework-realtek-amb1")
flash_addr = board.get("build.amb_flash_addr")
flash_ota1_offset = env.subst("$FLASH_OTA1_OFFSET")
flash_ota2_offset = env.subst("$FLASH_OTA2_OFFSET")
boot_all = board.get("build.amb_boot_all")
ota1_offset = hex(int(flash_addr, 16) + int(flash_ota1_offset, 16))
ota2_offset = hex(int(flash_addr, 16) + int(flash_ota2_offset, 16))
# Outputs
env.Replace(
IMG_FW="image2_all_ota1.bin",
IMG_OTA="ota_all.bin",
)
# Tools
# fmt: off
TOOL_DIR = join("$SDK_DIR", "component", "soc", "realtek", "8711b", "misc", "iar_utility", "common", "tools")
# fmt: on
env.Replace(
PICK=join(TOOL_DIR, "pick"),
PAD=join(TOOL_DIR, "pad"),
CHECKSUM=join(TOOL_DIR, "checksum"),
OTA=join(TOOL_DIR, "ota"),
)
# Flags
env.Append(
CFLAGS=[
@@ -57,7 +32,6 @@ env.Append(
CPPDEFINES=[
"M3",
"CONFIG_PLATFORM_8711B",
("F_CPU", "166000000L"),
# LwIP options
("LWIP_TIMEVAL_PRIVATE", "0"),
("LWIP_NETIF_HOSTNAME", "1"), # to support hostname changing
@@ -228,7 +202,7 @@ env.AddLibrary(
],
)
# Sources - platform fixups
# Sources - family fixups
env.AddLibrary(
name="ambz_fixups",
base_dir="$FIXUPS_DIR",
@@ -279,65 +253,6 @@ env.Replace(
SIZEPRINTCMD="$SIZETOOL -B -d $SOURCES",
)
# Image conversion
def pick_tool(target, source, env):
sections = [
"__ram_image2_text_start__",
"__ram_image2_text_end__",
"__xip_image2_start__",
]
addrs = [None] * len(sections)
with open(env.subst("${BUILD_DIR}/${PROGNAME}.nmap")) as f:
for line in f:
for i, section in enumerate(sections):
if section not in line:
continue
addrs[i] = line.split()[0]
files = [
join("$BUILD_DIR", "ram_2.r.bin"), # RAM image with padding
join("$BUILD_DIR", "ram_2.bin"), # RAM image, stripped
join("$BUILD_DIR", "ram_2.p.bin"), # RAM image, stripped, with header
join("$BUILD_DIR", "xip_image2.bin"), # raw firmware image
join("$BUILD_DIR", "xip_image2.p.bin"), # firmware with header
]
commands = [
f"$PICK 0x{addrs[0]} 0x{addrs[1]} {files[0]} {files[1]} raw",
f"$PICK 0x{addrs[0]} 0x{addrs[1]} {files[1]} {files[2]}",
f"$PICK 0x{addrs[2]} 0x{addrs[2]} {files[3]} {files[4]}",
]
for command in commands:
status = env.Execute("@" + command + " > " + join("$BUILD_DIR", "pick.txt"))
if status:
return status
def concat_xip_ram(target, source, env):
with open(env.subst("${BUILD_DIR}/xip_image2.p.bin"), "rb") as f:
xip = f.read()
with open(env.subst("${BUILD_DIR}/ram_2.p.bin"), "rb") as f:
ram = f.read()
with open(env.subst("${BUILD_DIR}/${IMG_FW}"), "wb") as f:
f.write(xip)
f.write(ram)
def checksum_img(target, source, env):
source = join("$BUILD_DIR", "$IMG_FW")
status = env.Execute(f"@$CHECKSUM {source}")
if status:
return status
def package_ota(target, source, env):
source = join("$BUILD_DIR", "$IMG_FW")
target = join("$BUILD_DIR", "$IMG_OTA")
status = env.Execute(
f"@$OTA {source} {ota1_offset} {source} {ota2_offset} 0x20170111 {target}"
)
if status:
return status
env.Append(
BUILDERS=dict(
BinToObj=Builder(
@@ -354,90 +269,9 @@ env.Append(
)
),
)
commands = [
(
"${PROGNAME}.nmap",
[
"$NM",
"$SOURCE",
"> $BIN",
],
),
(
"ram_2.r.bin",
[
"$OBJCOPY",
"-j .ram_image2.entry",
"-j .ram_image2.data",
"-j .ram_image2.bss",
"-j .ram_image2.skb.bss",
"-j .ram_heap.data",
"-O binary",
"$SOURCE",
"$BIN",
],
),
(
"xip_image2.bin",
[
"$OBJCOPY",
"-j .xip_image2.text",
"-O binary",
"$SOURCE",
"$BIN",
],
),
(
"rdp.bin",
[
"$OBJCOPY",
"-j .ram_rdp.text",
"-O binary",
"$SOURCE",
"$BIN",
],
),
]
actions = [
env.VerboseAction(
" ".join(command).replace("$BIN", join("$BUILD_DIR", target)),
f"Generating {target}",
)
for target, command in commands
]
actions.append(env.VerboseAction(pick_tool, "Wrapping binary images"))
actions.append(env.VerboseAction(concat_xip_ram, "Packaging firmware image - $IMG_FW"))
# actions.append(env.VerboseAction(checksum_img, "Generating checksum"))
actions.append(env.VerboseAction(package_ota, "Packaging OTA image - $IMG_OTA"))
actions.append(env.VerboseAction("true", f"- OTA1 flash offset: $FLASH_OTA1_OFFSET"))
actions.append(env.VerboseAction("true", f"- OTA2 flash offset: $FLASH_OTA2_OFFSET"))
# Uploader
upload_protocol = env.subst("$UPLOAD_PROTOCOL")
upload_source = ""
upload_actions = []
# from platform-espressif32/builder/main.py
if upload_protocol == "uart":
env.Replace(
UPLOADER=join("$TOOLS_DIR", "rtltool.py"),
UPLOADERFLAGS=[
"--port",
"$UPLOAD_PORT",
"--go", # run firmware after uploading
"wf", # Write a binary file to Flash data
],
UPLOADCMD='"$PYTHONEXE" "$UPLOADER" $UPLOADERFLAGS $FLASH_OTA1_OFFSET "$BUILD_DIR/$IMG_FW"',
)
upload_actions = [
env.VerboseAction(env.AutodetectUploadPort, "Looking for upload port..."),
env.VerboseAction("$UPLOADCMD", "Uploading $IMG_FW"),
]
elif upload_protocol == "custom":
upload_actions = [env.VerboseAction("$UPLOADCMD", "Uploading $IMG_FW")]
else:
sys.stderr.write("Warning! Unknown upload protocol %s\n" % upload_protocol)
# Bootloader library
boot_all = board.get("build.amb_boot_all")
target_boot = env.StaticLibrary(
join("$BUILD_DIR", "boot_all"),
env.BinToObj(
@@ -450,10 +284,19 @@ env.Prepend(LIBS=[target_boot])
# Build all libraries
env.BuildLibraries()
# Main firmware binary builder
env.Append(
BUILDERS=dict(
DumpFirmwareBinary=Builder(action=actions),
),
UPLOAD_ACTIONS=upload_actions,
# Main firmware outputs and actions
env.Replace(
# linker command (dual .bin outputs)
LINK="${LINK2BIN} AMBZ xip1 xip2",
# default output .bin name
IMG_FW="image_${FLASH_OTA1_OFFSET}.ota1.bin",
# UF2OTA input list
UF2OTA=[
(
"ota1",
"${BUILD_DIR}/image_${FLASH_OTA1_OFFSET}.ota1.bin",
"ota2",
"${BUILD_DIR}/image_${FLASH_OTA2_OFFSET}.ota2.bin",
),
],
)

29
builder/libs/flashdb.py Normal file
View File

@@ -0,0 +1,29 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-24.
from SCons.Script import DefaultEnvironment
env = DefaultEnvironment()
platform = env.PioPlatform()
def env_add_flashdb(
env,
version: str,
):
package_dir = platform.get_package_dir(f"library-flashdb@{version}")
env.AddLibrary(
name=f"flashdb{version}",
base_dir=package_dir,
srcs=[
"+<src/*.c>",
"+<port/fal/src/*.c>",
],
includes=[
"+<inc>",
"+<port/fal/inc>",
],
)
env.AddMethod(env_add_flashdb, "AddLibraryFlashDB")

View File

@@ -1,5 +1,7 @@
# Copyright (c) Kuba Szczodrzyński 2022-04-20.
import sys
from SCons.Script import Default, DefaultEnvironment
env = DefaultEnvironment()
@@ -7,8 +9,10 @@ board = env.BoardConfig()
# Utilities
env.SConscript("utils.py", exports="env")
env.SConscript("uf2.py", exports="env")
# Vendor-specific library ports
env.SConscript("libs/lwip.py", exports="env")
env.SConscript("libs/flashdb.py", exports="env")
# Firmware name
if env.get("PROGNAME", "program") == "program":
@@ -24,7 +28,6 @@ env.Replace(
GDB="arm-none-eabi-gdb",
NM="arm-none-eabi-gcc-nm",
LINK="arm-none-eabi-gcc",
LD="arm-none-eabi-gcc",
OBJCOPY="arm-none-eabi-objcopy",
OBJDUMP="arm-none-eabi-objdump",
# RANLIB="arm-none-eabi-gcc-ranlib",
@@ -35,34 +38,48 @@ env.Replace(
flash_layout: dict = board.get("flash")
if flash_layout:
defines = {}
flash_size = 0
fal_items = ""
for name, layout in flash_layout.items():
name = name.upper()
(offset, _, length) = layout.partition("+")
defines[f"FLASH_{name}_OFFSET"] = offset
defines[f"FLASH_{name}_LENGTH"] = length
fal_items += f"FAL_PART_TABLE_ITEM({name.lower()}, {name})"
flash_size = max(flash_size, int(offset, 16) + int(length, 16))
defines["FLASH_LENGTH"] = flash_size
defines["FAL_PART_TABLE"] = "{" + fal_items + "}"
env.Append(CPPDEFINES=defines.items())
env.Replace(**defines)
# Platform builders details:
# - call env.AddDefaults("platform name", "sdk name") to add dir paths
# Family builders details:
# - call env.AddDefaults("family name", "sdk name") to add dir paths
# - call env.AddLibrary("lib name", "base dir", [sources]) to add lib sources
# - output main firmware image binary as $IMG_FW
# - call env.BuildLibraries() to build lib targets with safe envs
# - configure LINK, UF2OTA and UPLOAD_ACTIONS
# - script code ordering:
# - global vars
# - # Outputs
# - # Tools
# - # Flags (C(XX)FLAGS / CPPDEFINES / LINKFLAGS)
# - sources (env.AddLibrary)
# - # Libs & linker config (LIBPATH / LIBS / LDSCRIPT_PATH)
# - # Misc options
# - # Image conversion (tools, functions, builders, actions, etc.)
# - # Uploader
# - # Library targets
# - # Bootloader library
# - env.BuildLibraries()
# - # Main firmware binary builder
# - # Main firmware outputs and actions
target_elf = env.BuildProgram()
target_fw = env.DumpFirmwareBinary("$IMG_FW", target_elf)
env.AddPlatformTarget("upload", target_fw, env["UPLOAD_ACTIONS"], "Upload")
Default(target_fw)
targets = [target_elf]
if "UF2OTA" in env:
target_uf2 = env.BuildUF2OTA(target_elf)
targets.append(target_uf2)
env.AddUF2Uploader(target_uf2)
elif "IMG_FW" in env:
target_fw = env.subst("$IMG_FW")
env.AddPlatformTarget("upload", target_fw, env["UPLOAD_ACTIONS"], "Upload")
else:
sys.stderr.write("Warning! Firmware outputs not specified.\n")
Default(targets)

85
builder/uf2.py Normal file
View File

@@ -0,0 +1,85 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
import sys
from datetime import datetime
from os.path import basename, join, normpath
from SCons.Script import Builder, DefaultEnvironment
env = DefaultEnvironment()
platform = env.PioPlatform()
def env_uf2ota(env, *args, **kwargs):
now = datetime.now()
project_dir = env.subst("$PROJECT_DIR")
project_name = basename(normpath(project_dir))
# TODO support specifying custom version
project_version = now.strftime("%y.%m.%d")
lt_version = platform.version
inputs = " ".join(f'"{";".join(input)}"' for input in env["UF2OTA"])
output = [
project_name,
project_version,
"${VARIANT}",
"${FAMILY}",
f"lt{lt_version}",
]
output = join("${BUILD_DIR}", "_".join(output)) + ".uf2"
env["UF2OUT"] = output
env["UF2OUT_BASE"] = basename(output)
cmd = [
"@${UF2OTA_PY}",
f'--output "{output}"',
"--family ${FAMILY}",
"--board ${VARIANT}",
f"--version {lt_version}",
f'--fw "{project_name}:{project_version}"',
f"--date {int(now.timestamp())}",
"write",
inputs,
]
print(f"|-- {basename(env.subst(output))}")
env.Execute(" ".join(cmd))
def env_uf2upload(env, target):
protocol = env.subst("${UPLOAD_PROTOCOL}")
actions = []
# from platform-espressif32/builder/main.py
if protocol == "uart":
# upload via UART
env["UPLOADERFLAGS"] = [
"${UF2OUT}",
"uart",
"${UPLOAD_PORT}",
]
actions = [
env.VerboseAction(env.AutodetectUploadPort, "Looking for upload port..."),
]
elif protocol == "custom":
actions = [
env.VerboseAction("${UPLOADCMD}", "Uploading firmware"),
]
else:
sys.stderr.write("Warning! Unknown upload protocol %s\n" % protocol)
return
# add main upload target
env.Replace(UPLOADER="${UF2UPLOAD_PY}", UPLOADCMD="${UPLOADER} ${UPLOADERFLAGS}")
actions.append(env.VerboseAction("${UPLOADCMD}", "Uploading ${UF2OUT_BASE}"))
env.AddPlatformTarget("upload", target, actions, "Upload")
env.Append(
BUILDERS=dict(
BuildUF2OTA=Builder(
action=[env.VerboseAction(env_uf2ota, "Building UF2 OTA image")]
)
)
)
env.AddMethod(env_uf2upload, "AddUF2Uploader")

View File

@@ -12,26 +12,30 @@ platform = env.PioPlatform()
board = env.BoardConfig()
def env_add_defaults(env, platform_name: str, sdk_name: str):
def env_add_defaults(env, family_name: str, sdk_name: str):
vars = dict(
SDK_DIR=platform.get_package_dir(sdk_name),
LT_DIR=platform.get_dir(),
# Root dirs
BOARD_DIR=join("${LT_DIR}", "boards", "${VARIANT}"),
ARDUINO_DIR=join("${LT_DIR}", "arduino", platform_name),
PLATFORM_DIR=join("${LT_DIR}", "platform", platform_name),
ARDUINO_DIR=join("${LT_DIR}", "arduino", family_name),
FAMILY_DIR=join("${LT_DIR}", "platform", family_name),
TOOLS_DIR=join("${LT_DIR}", "tools"),
# Platform-specific dirs
BIN_DIR=join("${PLATFORM_DIR}", "bin"),
FIXUPS_DIR=join("${PLATFORM_DIR}", "fixups"),
LD_DIR=join("${PLATFORM_DIR}", "ld"),
OPENOCD_DIR=join("${PLATFORM_DIR}", "openocd"),
# Family-specific dirs
BIN_DIR=join("${FAMILY_DIR}", "bin"),
FIXUPS_DIR=join("${FAMILY_DIR}", "fixups"),
LD_DIR=join("${FAMILY_DIR}", "ld"),
OPENOCD_DIR=join("${FAMILY_DIR}", "openocd"),
# Board config variables
MCU=board.get("build.mcu").upper(),
FAMILY=board.get("build.family").upper(),
FAMILY=board.get("build.family"),
VARIANT=board.get("build.variant"),
LDSCRIPT_SDK=board.get("build.ldscript_sdk"),
LDSCRIPT_ARDUINO=board.get("build.ldscript_arduino"),
# Link2Bin tool
LINK2BIN='"${PYTHONEXE}" "${LT_DIR}/tools/link2bin.py"',
UF2OTA_PY='"${PYTHONEXE}" "${LT_DIR}/tools/uf2ota/uf2ota.py"',
UF2UPLOAD_PY='"${PYTHONEXE}" "${LT_DIR}/tools/upload/uf2upload.py"',
)
env.Replace(**vars)
for k, v in vars.items():
@@ -49,6 +53,9 @@ def env_add_defaults(env, platform_name: str, sdk_name: str):
CPPDEFINES=[
("LT_VERSION", platform.version),
("LT_BOARD", board.get("build.variant")),
("F_CPU", board.get("build.f_cpu")),
("MCU", board.get("build.mcu").upper()),
("FAMILY", board.get("build.family")),
],
)

View File

@@ -1,6 +1,6 @@
# LibreTuya API Configuration
Note: see [LibreTuyaConfig.h](../arduino/libretuya/api/LibreTuyaConfig.h) for most options and their defaults.
Note: see [LibreTuyaConfig.h](../arduino/libretuya/core/LibreTuyaConfig.h) for most options and their defaults.
All options are configurable via C++ defines in PlatformIO project file. For example:
```ini
@@ -29,7 +29,7 @@ build_flags =
The following options enable library-specific debugging messages. They are only effective if `LT_LOGLEVEL` is set below INFO. All of them are disabled by default.
Platforms should generally call i.e. WiFiClient debugging for client-related code, even if the `WiFiClient.cpp` file is physically absent.
Families should generally call i.e. WiFiClient debugging for client-related code, even if the `WiFiClient.cpp` file is physically absent.
- LT_DEBUG_WIFI - `WiFi.cpp`
- LT_DEBUG_WIFI_CLIENT - `WiFiClient.cpp`
@@ -37,7 +37,7 @@ Platforms should generally call i.e. WiFiClient debugging for client-related cod
- LT_DEBUG_WIFI_STA - `WiFiSTA.cpp`
- LT_DEBUG_WIFI_AP - `WiFiAP.cpp`
## Platform options
## Family options
- LT_HAS_LWIP - whether platform SDK has LwIP. This causes `LwIPRxBuffer.cpp` to be compiled for platform libraries to use.
- LT_HAS_LWIP2 - whether platform has LwIP v2.0.0 or newer. This causes `LwIPmDNS.cpp` to be compiled.
- LT_HAS_LWIP - whether family SDK has LwIP. This causes `LwIPRxBuffer.cpp` to be compiled for family libraries to use.
- LT_HAS_LWIP2 - whether family has LwIP v2.0.0 or newer. This causes `LwIPmDNS.cpp` to be compiled.

18
docs/families.md Normal file
View File

@@ -0,0 +1,18 @@
# Families
A list of families currently available in this project.
**Note:** the term *family* was chosen over *platform*, in order to reduce possible confusion between LibreTuya supported "platforms" and PlatformIO's "platform", as an entire package. *Family* is also more compatible with the UF2 term.
The following list corresponds to UF2 OTA format family names, and is also [available as JSON](../uf2families.json). The IDs are also present in [ChipType.h](../arduino/libretuya/core/ChipType.h).
Full name | Code | Short name & ID | Supported MCU(s) | Arduino Core | Source SDK
-----------------------------------------------------------------------|--------|-------------------------|------------------|--------------|--------------------------------------------------------------------------
Realtek Ameba1 | `-` | `RTL8710A` (0x9FFFD543) | - | ❌ | -
[Realtek AmebaZ](https://www.amebaiot.com/en/amebaz/) (`realtek-ambz`) | `ambz` | `RTL8710B` (0x22E0D6FC) | RTL87xxB | ✔️ | `framework-realtek-amb1` ([amb1_sdk](https://github.com/ambiot/amb1_sdk))
Realtek AmebaZ2 | `-` | `RTL8720C` (0xE08F7564) | - | ❌ | -
Realtek AmebaD | `-` | `RTL8720D` (0x3379CFE2) | - | ❌ | -
Beken 7231T | `-` | `BK7231T` (0x675A40B0) | - | ❌ | -
Beken 7231N | `-` | `BK7231N` (0x7B3EF230) | - | ❌ | -
Boufallo 602 | `-` | `BL602` (0xDE1270B7) | - | ❌ | -
Xradiotech 809 | `-` | `XR809` (0x51E903A8) | - | ❌ | -

77
docs/families.py Normal file
View File

@@ -0,0 +1,77 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-31.
import json
from os.path import dirname, isdir, join
HEADER = """\
# Families
A list of families currently available in this project.
**Note:** the term *family* was chosen over *platform*, in order to reduce possible confusion between LibreTuya supported "platforms" and PlatformIO's "platform", as an entire package. *Family* is also more compatible with the UF2 term.
The following list corresponds to UF2 OTA format family names, and is also [available as JSON](../uf2families.json). The IDs are also present in [ChipType.h](../arduino/libretuya/core/ChipType.h).
"""
def format_row(row: list, lengths: list) -> str:
row = [col + " " * (lengths[i] - len(col)) for i, col in enumerate(row)]
return " | ".join(row).rstrip()
if __name__ == "__main__":
data = join(dirname(__file__), "..", "families.json")
out = join(dirname(__file__), "families.md")
with open(data, "r") as f:
data = json.load(f)
md = [HEADER]
lengths = [0, 0, 0, 0, 0, 0]
header = [
"Full name",
"Code",
"Short name & ID",
"Supported MCU(s)",
"Arduino Core",
"Source SDK",
]
rows = []
for family in data:
id = family["id"]
short_name = family["short_name"]
description = family["description"]
name = family.get("name", "")
code = family.get("code", "-")
url = family.get("url", "-")
sdk = family.get("sdk", "-")
framework = family.get("framework", "-")
mcus = family.get("mcus", "-")
sdk_name = sdk.rpartition("/")[2]
arduino = (
isdir(join(dirname(__file__), "..", "arduino", name)) if name else False
)
row = [
f"[{description}]({url}) (`{name}`)" if name else description,
f"`{code}`",
f"`{short_name}` ({id})",
", ".join(mcus),
"✔️" if arduino else "",
f"`{framework}` ([{sdk_name}]({sdk}))" if name else "-",
]
rows.append(row)
for row in [header] + rows:
for i, col in enumerate(row):
lengths[i] = max(lengths[i], len(col))
md.append(format_row(header, lengths))
md.append("-|-".join(length * "-" for length in lengths))
for row in rows:
md.append(format_row(row, lengths))
md.append("")
with open(out, "w", encoding="utf-8") as f:
f.write("\n".join(md))

View File

@@ -0,0 +1,7 @@
# Implementation status
{%
include-markdown "../README.md"
start="\n## Arduino Core support status\n"
end="\n## License\n"
%}

157
docs/ota/README.md Normal file
View File

@@ -0,0 +1,157 @@
# UF2-based OTA
LibreTuya's OTA updating is based on [Microsoft's UF2 specification](https://microsoft.github.io/uf2/). Some aspects of the process, such as OTA1/2 support and target partition selection, have been customized with extension tags.
Note: just like in UF2, all values in this format are little-endian.
## Firmware images
UF2 files may contain multiple firmware images that are to be flashed, i.e. main firmware + bootloader + some config partition.
Some CPUs support dual-OTA schemes: firmware runs from one image, while the other one is reserved for updated firmware. After applying the update, a reboot causes to run the other image instead.
Each firmware image may be either applicable:
1. only when flashing OTA1 (`part;file;;`)
2. only when flashing OTA2 (`;;part;file`)
3. for both schemes to a single partition (`part;file`)
4. for both schemes but different partitions (`part1;file;part2;file`)
5. for both schemes but with a different binary (`part;file1;part;file2`)
6. for both schemes, with different binaries and target partitions (`part1;file1;part2;file2`)
\* *`part` means partition here*
\*\* values in parentheses show the input format to use for [`uf2ota.py`](uf2ota.md)
For easier understanding, these update types will be referred to in this document using the numbers.
## Custom family IDs
Name | ID | Description
-----------|------------|----------------
`RTL8710A` | 0x9FFFD543 | Realtek Ameba1
`RTL8710B` | 0x22E0D6FC | Realtek AmebaZ
`RTL8720C` | 0xE08F7564 | Realtek AmebaZ2
`RTL8720D` | 0x3379CFE2 | Realtek AmebaD
`BK7231T` | 0x675A40B0 | Beken 7231T
`BK7231N` | 0x7B3EF230 | Beken 7231N
`BL602` | 0xDE1270B7 | Boufallo 602
`XR809` | 0x51E903A8 | Xradiotech 809
## Extension tags
Standard tags are used: `VERSION`, `DEVICE` and `DEVICE_ID`.
Additionally, custom tags are defined:
Name | ID | Type | Description
--------------|----------|------------|-------------------------------------------------
`OTA_VERSION` | 0x5D57D0 | int 8-bit | format version (for simple compatibility checks)
`BOARD` | 0xCA25C8 | string | board name / code (lowercase)
`FIRMWARE` | 0x00DE43 | string | firmware description / name
`BUILD_DATE` | 0x822F30 | int 32-bit | build date/time as Unix timestamp
`LT_VERSION` | 0x59563D | semver | LT version
`LT_PART_1` | 0x805946 | string | OTA1 partition name
`LT_PART_2` | 0xA1E4D7 | string | OTA2 partition name
`LT_HAS_OTA1` | 0xBBD965 | bool 8-bit | image has any data for OTA1
`LT_HAS_OTA2` | 0x92280E | bool 8-bit | image has any data for OTA2
`LT_BINPATCH` | 0xB948DE | bytes | binary patch to convert OTA1->OTA2
## Update types
### Single OTA scheme (1, 2)
Image is ignored if the OTA scheme does not match. UF2 has `LT_PART_1` or `LT_PART_2` set to target partition name. The other partition tag is present, but empty (0 bytes).
```
08 46 59 80 6f 74 61 31 | .FY.ota1 | LT_PART_1
04 d7 e4 a1 | .... | LT_PART_2
```
### Dual-OTA/single-file scheme (3, 4)
One image is used for both OTA schemes. UF2 has `LT_PART_1` and `LT_PART_2` tags set. For type `3` these two tags contain the same partition name.
```
08 46 59 80 6f 74 61 31 | .FY.ota1 | LT_PART_1
08 d7 e4 a1 6f 74 61 32 | ....ota2 | LT_PART_2
```
### Dual-OTA/dual-file scheme (5, 6)
Just like types `3` and `4`, UF2 has two partition tags set. For type `5` they have the same name.
The image stored in UF2 is meant for OTA1 scheme. There is an additional tag `LT_BINPATCH` present. In OTA1 flashing scheme, it is ignored.
## Binary patching
OTA2 images are not stored directly, as that would needlessly double the UF2 file size. Instead, binary patching instructions, embedded into the extension tags area, allow the CPU to convert the OTA1 image from UF2 into OTA2 image.
There can be at most one binpatch tag in a UF2 block. It has the following format:
- opcode (1 byte) - operation type:
- `DIFF32` (0xFE) - difference between 32-bit values
- length (1 byte) - data length
- data (`length` bytes)
- for `DIFF32`:
- difference value (signed int 32-bit)
- offset table (`length-4` bytes)
The presented structure can be repeated in a single binpatch tag.
### DIFF32
This method works by adding the difference value to a 32-bit integer. It allows to save the most space in OTA1/2 image scenarios, where the only different values are, for example, flash memory addresses. The offset table contains positions within the 256-byte block, to which the difference value should be mathematically added.
For a block like:
```
000 72 71 73 61 76 65 00 00 5f 66 72 65 65 72 74 6f |rqsave.._freerto|
010 73 5f 6d 75 74 65 78 5f 67 65 74 5f 74 69 6d 65 |s_mutex_get_time|
020 6f 75 74 00 5d a4 03 08 61 a4 03 08 85 a4 03 08 |out.]...a.......|
030 5d a4 03 08 61 a4 03 08 85 a4 03 08 81 a9 03 08 |]...a...........|
040 6d a9 03 08 7d a4 03 08 d9 a8 03 08 05 a7 03 08 |m...}...........|
050 bd a4 03 08 ad a8 03 08 59 a7 03 08 9d a8 03 08 |........Y.......|
060 01 a7 03 08 51 a8 03 08 21 aa 03 08 b9 a4 03 08 |....Q...!.......|
070 85 a3 03 08 89 a3 03 08 4d a4 03 08 a1 a8 03 08 |........M.......|
080 00 00 00 00 00 00 00 00 19 a8 03 08 c1 a4 03 08 |................|
090 8d a8 03 08 ed a6 03 08 dd a7 03 08 ad a4 03 08 |................|
0a0 9d a7 03 08 95 a4 03 08 81 a7 03 08 09 a7 03 08 |................|
0b0 31 a7 03 08 d1 a6 03 08 dd a5 03 08 61 aa 03 08 |1...........a...|
0c0 c5 a2 03 08 d5 a2 03 08 d9 a2 03 08 b1 a6 03 08 |................|
0d0 65 aa 03 08 ad a6 03 08 a9 a6 03 08 8d a6 03 08 |e...............|
0e0 e5 a2 03 08 e9 a2 03 08 1d a4 03 08 ed a3 03 08 |................|
0f0 35 a4 03 08 05 a4 03 08 bd a3 03 08 8d a3 03 08 |5...............|
```
a DIFF32 patch containing:
```
fe 39 00 50 0c 00 24 28 2c 30 34 38 3c 40 44 48 |.9.P..$(,048<@DH|
4c 50 54 58 5c 60 64 68 6c 70 74 78 7c 88 8c 90 |LPTX\`dhlptx|...|
94 98 9c a0 a4 a8 ac b0 b4 b8 bc c0 c4 c8 cc d0 |................|
d4 d8 dc e0 e4 e8 ec f0 f4 f8 fc |........... |
```
adds 0x000C5000 to 53 values, producing OTA2 output like this:
```
000 72 71 73 61 76 65 00 00 5f 66 72 65 65 72 74 6f |rqsave.._freerto|
010 73 5f 6d 75 74 65 78 5f 67 65 74 5f 74 69 6d 65 |s_mutex_get_time|
020 6f 75 74 00 5d f4 0f 08 61 f4 0f 08 85 f4 0f 08 |out.]...a.......|
030 5d f4 0f 08 61 f4 0f 08 85 f4 0f 08 81 f9 0f 08 |]...a...........|
040 6d f9 0f 08 7d f4 0f 08 d9 f8 0f 08 05 f7 0f 08 |m...}...........|
050 bd f4 0f 08 ad f8 0f 08 59 f7 0f 08 9d f8 0f 08 |........Y.......|
060 01 f7 0f 08 51 f8 0f 08 21 fa 0f 08 b9 f4 0f 08 |....Q...!.......|
070 85 f3 0f 08 89 f3 0f 08 4d f4 0f 08 a1 f8 0f 08 |........M.......|
080 00 00 00 00 00 00 00 00 19 f8 0f 08 c1 f4 0f 08 |................|
090 8d f8 0f 08 ed f6 0f 08 dd f7 0f 08 ad f4 0f 08 |................|
0a0 9d f7 0f 08 95 f4 0f 08 81 f7 0f 08 09 f7 0f 08 |................|
0b0 31 f7 0f 08 d1 f6 0f 08 dd f5 0f 08 61 fa 0f 08 |1...........a...|
0c0 c5 f2 0f 08 d5 f2 0f 08 d9 f2 0f 08 b1 f6 0f 08 |................|
0d0 65 fa 0f 08 ad f6 0f 08 a9 f6 0f 08 8d f6 0f 08 |e...............|
0e0 e5 f2 0f 08 e9 f2 0f 08 1d f4 0f 08 ed f3 0f 08 |................|
0f0 35 f4 0f 08 05 f4 0f 08 bd f3 0f 08 8d f3 0f 08 |5...............|
```

58
docs/ota/library.md Normal file
View File

@@ -0,0 +1,58 @@
# uf2ota library
uf2ota library allows to write a LibreTuya UF2 file to the flash, while parsing all the necessary tags. It manages the target partitions, compatibility checks, and works on top of the FAL provided by FlashDB.
## Usage example
```c
uint8_t target = 1; // target OTA scheme - 1 or 2
uint32_t family = RTL8710B; // chip's UF2 family ID
uf2_ota_t *ctx = uf2_ctx_init(target, family);
uf2_info_t *info = uf2_info_init(); // optional, for getting firmware info
uf2_block_t *block = (uf2_block_t *)malloc(UF2_BLOCK_SIZE);
uf2_err_t err;
// ... // read the first header block (512 bytes) into *block
// check the block for validity
err = uf2_check_block(ctx, block);
if (err > UF2_ERR_IGNORE)
// handle the error
return;
// parse the header block
// note: if you don't need info, you can skip this step and call uf2_write() directly
err = uf2_parse_header(ctx, block, info);
if (err)
// handle the error
return;
while (/* have input data */) {
// ... // read the next block into *block
// check the block for validity
err = uf2_check_block(ctx, block);
if (err == UF2_ERR_IGNORE)
// skip this block
continue;
if (err)
// handle the error
return;
// write the block to flash
err = uf2_write(ctx, block);
if (err > UF2_ERR_IGNORE)
// handle the error
return;
}
// finish the update process
// ... // activate your new OTA partition
// cleanup
free(ctx);
free(block);
uf2_info_free(info);
```

73
docs/ota/uf2ota.md Normal file
View File

@@ -0,0 +1,73 @@
# uf2ota.py
This is a tool for converting LibreTuya firmware images to UF2 format for OTA updates.
```bash
$ python uf2ota.py
usage: uf2ota [-h] [--output OUTPUT] [--family FAMILY] [--board BOARD] [--version VERSION] [--fw FW] {info,dump,write} inputs [inputs ...]
uf2ota: error: the following arguments are required: action, inputs
```
# write
Generate a UF2 file from a firmware image or several images.
```bash
$ python uf2ota.py write --family RTL8710B --board wr3 --version 0.4.0 --fw esphome:2022.6.0-dev "ota1;xip1.bin;ota2;xip2.bin"
$ ls -l out.uf2
-rw-r--r-- 1 Kuba None 605696 May 28 14:35 out.uf2
```
## inputs format
Format for `inputs` parameter is `part;file[;part;file]` (square brackets mean optional). First two (colon separated) values correspond to flashing OTA1 region, second two to OTA2.
Partition name can be suffixed by `+offset`, which causes writing the image file to the partition after some byte offset. Both files and/or partition names can be equal. Values can be empty (like `part;file;;` or `;;part;file`) if OTA1/2 images are not present in this file.
When using two different firmware binaries, they need to have the same `offset` and be of the same size.
`inputs` parameter can be repeated in order to embed multiple files in the UF2. For example:
```bash
"bootloader;boot.bin" "ota1;xip1.bin;ota2;xip2.bin" "config;config1.bin;config;config2.bin"
```
will:
- flash the bootloader in both OTA schemes
- flash `xip1.bin` or `xip2.bin` to `ota1` or `ota2` partitions
- flash `config1.bin` or `config2.bin` to `config` partition
# info
This command shows some basic parameters of a UF2 image.
```bash
$ python uf2ota.py info out.uf2
Family: RTL8710B
Tags:
- BOARD: wr3
- DEVICE_ID: 312d5ec5
- LT_VERSION: 0.4.0
- FIRMWARE: esphome
- VERSION: 2022.6.0-dev
- OTA_VERSION: 01
- DEVICE: LibreTuya
- LT_HAS_OTA1: 01
- LT_HAS_OTA2: 01
- LT_PART_1: ota1
- LT_PART_2: ota2
- LT_BINPATCH: fe0900500c009094989ca0
Data chunks: 1182
Total binary size: 302448
```
# dump
Dump UF2 file (only LibreTuya format) into separate firmware binaries.
```bash
$ python uf2ota.py dump out.uf2
$ ls -1 out.uf2_dump/
esphome_2022.6.0-dev_lt0.4.0_wr3_1_ota1_0x0.bin
esphome_2022.6.0-dev_lt0.4.0_wr3_2_ota2_0x0.bin
```

View File

@@ -0,0 +1,26 @@
# Realtek Ameba - notes
The logic behind naming of Realtek chips and their series took me some time to figure out:
- RTL8xxxA - Ameba1/Ameba Series
- RTL8xxxB - AmebaZ Series
- RTL8xxxC - AmebaZ2/ZII Series
- RTL8xxxD - AmebaD Series
As such, there are numerous CPUs with the same numbers but different series, which makes them require different code and SDKs.
- [RTL8195AM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8195am)
- RTL8710AF (found in amb1_arduino)
- [RTL8711AM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8711am)
- [RTL8710BN](https://www.realtek.com/en/products/communications-network-ics/item/rtl8710bn)
- RTL8710BX (found in Tuya product pages)
- RTL8710B? (found in amb1_sdk)
- RTL8711B? (found in amb1_sdk)
- [RTL8710CM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8710cm)
- RTL8722CSM (found in ambd_arduino)
- RTL8720DN (found in ambd_arduino)
- [RTL8721DM](https://www.realtek.com/en/products/communications-network-ics/item/rtl8721dm)
- RTL8722DM (found in ambd_arduino)
- and probably many more
Different Ameba series are not compatible with each other. Apparently, there isn't an official public SDK for AmebaZ that can support C++ properly.

38
docs/project-structure.md Normal file
View File

@@ -0,0 +1,38 @@
# Project structure
```
arduino/
├─ <family name>/ Arduino Core for specific SoC
│ ├─ cores/ Wiring core files
│ ├─ libraries/ Supported built-in family libraries
├─ libretuya/
│ ├─ api/ Library interfaces
│ ├─ common/ Units common to all families
│ ├─ compat/ Fixes for compatibility with ESP32 framework
│ ├─ core/ LibreTuya API for Arduino cores
│ ├─ libraries/ Built-in family-independent libraries
boards/
├─ <board name>/ Board-specific code
│ ├─ variant.cpp Arduino variant initialization
│ ├─ variant.h Arduino variant pin configs
├─ <board name>.json PlatformIO board description
builder/
├─ frameworks/ Framework builders for PlatformIO
│ ├─ <family name>-sdk.py Vanilla SDK build system
│ ├─ <family name>-arduino.py Arduino Core build system
├─ arduino-common.py Builder to provide ArduinoCore-API and LibreTuya APIs
├─ main.py Main PlatformIO builder
├─ utils.py SCons utils used during the build
docs/ Project documentation, guides, tips, etc.
platform/
├─ <family name>/ Family-specific configurations
│ ├─ bin/ Binary blobs (bootloaders, etc.)
│ ├─ fixups/ Code fix-ups to replace SDK parts
│ ├─ ld/ Linker scripts
│ ├─ openocd/ OpenOCD configuration files
tools/
├─ <tool name>/ Tools used during the build
families.json List of supported device families
platform.json PlatformIO manifest
platform.py Custom PlatformIO script
```

54
docs/resources.md Normal file
View File

@@ -0,0 +1,54 @@
# Resources
## Realtek
Code | Name
-------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
&nbsp; | From **amb1_sdk**
AN0004 | [Realtek low power wi-fi mp user guide](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0004%20Realtek%20low%20power%20wi-fi%20mp%20user%20guide.pdf)
AN0011 | [Realtek wlan simple configuration](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0011%20Realtek%20wlan%20simple%20configuration.pdf)
AN0012 | [Realtek secure socket layer(ssl)](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0012%20Realtek%20secure%20socket%20layer(ssl).pdf)
AN0025 | [Realtek at command](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0025%20Realtek%20at%20command.pdf)
AN0033 | [Realtek Ameba-1 over the air firmware update](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0033%20Realtek%20Ameba-1%20over%20the%20air%20firmware%20update.pdf)
AN0045 | [Realtek Ameba-1 power modes](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0045%20Realtek%20Ameba-1%20power%20modes.pdf)
AN0046 | [Realtek Ameba uart adapter](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0046%20Realtek%20Ameba%20uart%20adapter.pdf)
AN0060 | [Realtek UART update user manual](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0060%20Realtek%20UART%20update%20user%20manual.pdf)
AN0075 | [Realtek Ameba-all at command v2.0](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0075%20Realtek%20Ameba-all%20at%20command%20v2.0.pdf)
AN0096 | [Realtek xmodem UART update user manual](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0096%20Realtek%20xmodem%20UART%20update%20user%20manual.pdf)
AN0110 | [Realtek Ameba-Z over the air firmware update](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0110%20Realtek%20Ameba-Z%20over%20the%20air%20firmware%20update.pdf)
AN0111 | [Realtek Ameba-Z FreeRTOS tickless](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/AN0111%20Realtek%20Ameba-Z%20FreeRTOS%20tickless.pdf)
UM0006 | [Realtek wificonf application programming interface](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0006%20Realtek%20wificonf%20application%20programming%20interface.pdf)
UM0014 | [Realtek web server user guide](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0014%20Realtek%20web%20server%20user%20guide.pdf)
UM0023 | [Realtek Ameba-1 build environment setup - iar](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0023%20Realtek%20Ameba-1%20build%20environment%20setup%20-%20iar.pdf)
UM0027 | [Realtek Ameba-1 crypto engine](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0027%20Realtek%20Ameba-1%20crypto%20engine.pdf)
UM0034 | [Realtek Ameba-1 memory layout](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0034%20Realtek%20Ameba-1%20memory%20layout.pdf)
UM0039 | [Realtek Ameba-1 SDK quick start](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0039%20Realtek%20Ameba-1%20SDK%20quick%20start.pdf)
UM0048 | [Realtek Ameba1 DEV 1v0 User Manual_1v8_20160328](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0048%20Realtek%20Ameba1%20DEV%201v0%20User%20Manual_1v8_20160328.pdf)
UM0060 | [Realtek Ameba-1 mqtt user guide](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0060%20Realtek%20Ameba-1%20mqtt%20user%20guide.pdf)
UM0096 | [Realtek Ameba build environment setup - gcc](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0096%20Realtek%20Ameba%20build%20environment%20setup%20-%20gcc.pdf)
UM0096 | [Realtek Ameba-1 build environment setup - gcc](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0096%20Realtek%20Ameba-1%20build%20environment%20setup%20-%20gcc.pdf)
UM0101 | [Realtek Ameba-1 peripheral developerment user manual](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0101%20Realtek%20Ameba-1%20peripheral%20developerment%20user%20manual.pdf)
UM0110 | [Realtek Ameba-Z build environment setup - iar](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0110%20Realtek%20Ameba-Z%20build%20environment%20setup%20-%20iar.pdf)
UM0111 | [Realtek Ameba-Z memory layout](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0111%20Realtek%20Ameba-Z%20memory%20layout.pdf)
UM0112 | [Realtek Ameba-Z SDK quick start](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0112%20Realtek%20Ameba-Z%20SDK%20quick%20start.pdf)
UM0113 | [Realtek Ameba-Z DEV 1v0 User Manual](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0113%20Realtek%20Ameba-Z%20DEV%201v0%20User%20Manual.pdf)
UM0115 | [Realtek Ameba-Z Introduction](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0115%20Realtek%20Ameba-Z%20Introduction.pdf)
UM0116 | [Realtek Ameba-Z SDK change](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0116%20Realtek%20Ameba-Z%20SDK%20change.pdf)
UM0120 | [Realtek Ameba-Z User Configuration](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0120%20Realtek%20Ameba-Z%20User%20Configuration.pdf)
UM0121 | [Realtek Ameba-Z suspend resume api](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0121%20Realtek%20Ameba-Z%20suspend%20resume%20api.pdf)
UM0123 | [Realtek Ameba-Z power modes](https://raw.githubusercontent.com/ambiot/amb1_sdk/0c8da639b097f01c60e419405aecfafab1d08e43/doc/UM0123%20Realtek%20Ameba-Z%20power%20modes.pdf)
&nbsp; | From **ambd_sdk**
AN0004 | [Realtek low power wi-fi mp user guide](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0004%20Realtek%20low%20power%20wi-fi%20mp%20user%20guide.pdf)
AN0011 | [Realtek wlan simple configuration](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0011%20Realtek%20wlan%20simple%20configuration.pdf)
AN0012 | [Realtek secure socket layer(ssl)](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0012%20Realtek%20secure%20socket%20layer(ssl).pdf)
AN0025 | [Realtek at command](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0025%20Realtek%20at%20command.pdf)
AN0075 | [Realtek Ameba-all at command v2.0](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0075%20Realtek%20Ameba-all%20at%20command%20v2.0.pdf)
AN0096 | [Realtek Ameba-all xmodem uart update firmware](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0096%20Realtek%20Ameba-all%20xmodem%20uart%20update%20firmware.pdf)
AN0400 | [Ameba-D Application Note](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/AN0400%20Ameba-D%20Application%20Note.pdf)
UM0150 | [Realtek Ameba CoAP User Guide](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/UM0150%20Realtek%20Ameba%20CoAP%20User%20Guide.pdf)
UM0201 | [Ameba Common BT Application User Manual EN](https://raw.githubusercontent.com/ambiot/ambd_sdk/12dab4363fd0087eb4874461f8d3f6094110595f/doc/UM0201%20Ameba%20Common%20BT%20Application%20User%20Manual%20EN.pdf)
&nbsp; | Found elsewhere
AN0400 | [Ameba-D Application Note_v3_watermark](https://files.seeedstudio.com/products/102110419/Basic%20documents/AN0400%20Ameba-D%20Application%20Note_v3_watermark.pdf)
AN0500 | [Realtek Ameba-ZII application note](https://www.e-paper-display.com/99IOT/00015797-AN0500-Realtek-Ameba-ZII-application-note.en_233850.pdf)
UM0114 | [Realtek Ameba-Z datasheet v3.4](https://adelectronicsru.files.wordpress.com/2018/10/um0114-realtek-ameba-z-data-sheet-v3-4.pdf)
&nbsp; | [Product pages / realtek.com](https://www.realtek.com/en/products/communications-network-ics/category/802-11b-g-n)

50
families.json Normal file
View File

@@ -0,0 +1,50 @@
[
{
"id": "0x9FFFD543",
"short_name": "RTL8710A",
"description": "Realtek Ameba1"
},
{
"id": "0x22E0D6FC",
"short_name": "RTL8710B",
"description": "Realtek AmebaZ",
"name": "realtek-ambz",
"code": "ambz",
"url": "https://www.amebaiot.com/en/amebaz/",
"sdk": "https://github.com/ambiot/amb1_sdk",
"framework": "framework-realtek-amb1",
"mcus": [
"RTL87xxB"
]
},
{
"id": "0xE08F7564",
"short_name": "RTL8720C",
"description": "Realtek AmebaZ2"
},
{
"id": "0x3379CFE2",
"short_name": "RTL8720D",
"description": "Realtek AmebaD"
},
{
"id": "0x675A40B0",
"short_name": "BK7231T",
"description": "Beken 7231T"
},
{
"id": "0x7B3EF230",
"short_name": "BK7231N",
"description": "Beken 7231N"
},
{
"id": "0xDE1270B7",
"short_name": "BL602",
"description": "Boufallo 602"
},
{
"id": "0x51E903A8",
"short_name": "XR809",
"description": "Xradiotech 809"
}
]

View File

@@ -6,7 +6,7 @@
"type": "git",
"url": "https://github.com/kuba2k2/platformio-libretuya"
},
"version": "0.4.0",
"version": "0.5.0",
"frameworks": {
"arduino": {
"title": "Generic Arduino framework",
@@ -43,6 +43,11 @@
"version": "https://github.com/arduino/ArduinoCore-API",
"manifest": {
"description": "Hardware independent layer of the Arduino cores"
},
"libraries": {
"flashdb": [
"03500fa"
]
}
},
"library-lwip": {
@@ -53,6 +58,14 @@
"description": "lwIP - A Lightweight TCPIP stack"
}
},
"library-flashdb": {
"type": "framework",
"optional": true,
"base_url": "https://github.com/armink/FlashDB",
"manifest": {
"description": "An ultra-lightweight database that supports key-value and time series data"
}
},
"toolchain-gccarmnoneeabi": {
"type": "toolchain",
"optionalVersions": [

View File

@@ -39,15 +39,16 @@ def load_manifest(self, src):
# find additional manifest info
manifest = manifest.get("manifest", manifest_default)
# extract tag version
if "#" in spec.url:
manifest["version"] = spec.url.rpartition("#")[2].lstrip("v")
url = getattr(spec, "url", None) or getattr(spec, "uri", None) or ""
if "#" in url:
manifest["version"] = url.rpartition("#")[2].lstrip("v")
# put info from spec
manifest.update(
{
"name": spec.name,
"repository": {
"type": "git",
"url": spec.url,
"url": url,
},
}
)
@@ -95,22 +96,37 @@ class LibretuyaPlatform(PlatformBase):
framework = next(fw for fw in frameworks if framework in fw)
options.get("pioframework")[0] = framework
framework_obj = self.frameworks[framework]
# set specific compiler versions
if framework.startswith("realtek-ambz"):
self.packages["toolchain-gccarmnoneeabi"]["version"] = "~1.50401.0"
# make ArduinoCore-API required
if "arduino" in framework:
self.packages["framework-arduino-api"]["optional"] = False
# mark framework SDK as required
self.packages[framework_obj["package"]]["optional"] = False
# gather library dependencies
libraries = framework_obj["libraries"] if "libraries" in framework_obj else {}
for name, package in self.packages.items():
if "optional" in package and package["optional"]:
continue
if "libraries" not in package:
continue
libraries.update(package["libraries"])
# use appropriate vendor library versions
sdk_package_name = self.frameworks[framework]["package"]
sdk_package = self.packages[sdk_package_name]
sdk_libraries = sdk_package["libraries"] if "libraries" in sdk_package else {}
packages_new = {}
for name, package in self.packages.items():
if not name.startswith("library-"):
continue
name = name[8:] # strip "library-"
if name not in sdk_libraries:
if name not in libraries:
continue
lib_version = sdk_libraries[name][-1] # get latest version tag
lib_version = libraries[name][-1] # get latest version tag
package = dict(**package) # clone the base package
package["version"] = (
package["base_url"] + "#" + lib_version
@@ -121,10 +137,6 @@ class LibretuyaPlatform(PlatformBase):
packages_new[name] = package # put the package under a new name
self.packages.update(packages_new)
# make ArduinoCore-API required
if "arduino" in framework:
self.packages["framework-arduino-api"]["optional"] = False
# save platform packages for later
global libretuya_packages
libretuya_packages = self.packages
@@ -189,7 +201,7 @@ class LibretuyaPlatform(PlatformBase):
args.extend(
[
"-f",
"$LTPATH/platform/$LTPLATFORM/openocd/%s"
"$LTPATH/platform/$LTFAMILY/openocd/%s"
% debug.get("openocd_config"),
]
)
@@ -214,7 +226,7 @@ class LibretuyaPlatform(PlatformBase):
opts = debug_config.env_options
server = debug_config.server
lt_path = dirname(__file__)
lt_platform = opts["framework"][0].rpartition("-")[0]
lt_family = opts["framework"][0].rpartition("-")[0]
if not server:
debug_tool = opts.get("debug_tool", "custom")
board = opts.get("board", "<unknown>")
@@ -232,8 +244,8 @@ class LibretuyaPlatform(PlatformBase):
"-f",
"interface/%s.cfg" % opts.get("openocd_interface"),
] + server["arguments"]
# replace $LTPLATFORM with actual name
# replace $LTFAMILY with actual name
server["arguments"] = [
arg.replace("$LTPLATFORM", lt_platform).replace("$LTPATH", lt_path)
arg.replace("$LTFAMILY", lt_family).replace("$LTPATH", lt_path)
for arg in server["arguments"]
]

Submodule tools/boardgen updated: ae96c74bbc...0ea8d42303

View File

@@ -1,24 +1,17 @@
# Copyright 2022-04-24 kuba2k2
import json
import sys
from os.path import dirname, join
sys.path.append(join(dirname(__file__), ".."))
from argparse import ArgumentParser, FileType
from binascii import crc32
from os import makedirs
from os.path import basename, dirname, isfile, join
def crc16(data):
# https://gist.github.com/pintoXD/a90e398bba5a1b6c121de4e1265d9a29
crc = 0x0000
for b in data:
crc ^= b
for j in range(0, 8):
if (crc & 0x0001) > 0:
crc = (crc >> 1) ^ 0xA001
else:
crc = crc >> 1
return crc
from os.path import basename, dirname, join
from tools.util.crypto import crc16
from tools.util.platform import get_board_manifest
if __name__ == "__main__":
parser = ArgumentParser("dumptool", description="Convert flash dump images")
@@ -30,21 +23,12 @@ if __name__ == "__main__":
parser.add_argument("--no-checksum", "-c", help="Don't append checksum to filename")
args = parser.parse_args()
if isfile(args.board):
board = args.board
else:
board = join(dirname(__file__), "..", "boards", f"{args.board}.json")
if not isfile(board):
print("Board not found")
exit()
board = get_board_manifest(args.board)
with open(board, "r") as f:
data = json.load(f)
if "flash" not in data:
if "flash" not in board:
print("Flash layout not defined")
exit()
flash_layout = data["flash"]
flash_layout = board["flash"]
output = join(dirname(args.file.name), basename(args.file.name) + ".split")
output = args.output or output

227
tools/link2bin.py Normal file
View File

@@ -0,0 +1,227 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-31.
from argparse import ArgumentParser
from enum import Enum
from os import stat, unlink
from os.path import basename, dirname, getmtime, isfile, join
from shutil import copyfile
from subprocess import PIPE, Popen
from typing import IO, Dict, List, Tuple
class SocType(Enum):
UNSET = ()
AMBZ = (1, "arm-none-eabi-", True)
def cmd(self, cmd: str) -> IO[bytes]:
process = Popen(self.value[1] + cmd, stdout=PIPE)
return process.stdout
@property
def dual_ota(self):
return self.value[2]
soc: "SocType" = SocType.UNSET
# _ _ _ _ _ _ _ _
# | | | | | (_) (_) | (_)
# | | | | |_ _| |_| |_ _ ___ ___
# | | | | __| | | | __| |/ _ \/ __|
# | |__| | |_| | | | |_| | __/\__ \
# \____/ \__|_|_|_|\__|_|\___||___/
def chname(path: str, name: str) -> str:
return join(dirname(path), name)
def chext(path: str, ext: str) -> str:
return path.rpartition(".")[0] + "." + ext
def isnewer(what: str, than: str) -> bool:
if not isfile(than):
return True
if not isfile(what):
return False
return getmtime(what) > getmtime(than)
def readbin(file: str) -> bytes:
with open(file, "rb") as f:
data = f.read()
return data
def checkfile(path: str):
if not isfile(path) or stat(path).st_size == 0:
exit(1)
# ____ _ _ _ _
# | _ \(_) | | (_) |
# | |_) |_ _ __ _ _| |_ _| |___
# | _ <| | '_ \| | | | __| | / __|
# | |_) | | | | | |_| | |_| | \__ \
# |____/|_|_| |_|\__,_|\__|_|_|___/
def nm(input: str) -> Dict[str, int]:
out = {}
stdout = soc.cmd(f"gcc-nm {input}")
for line in stdout.readlines():
line = line.decode().strip().split(" ")
if len(line) != 3:
continue
out[line[2]] = int(line[0], 16)
return out
def objcopy(
input: str,
output: str,
sections: List[str],
fmt: str = "binary",
) -> str:
# print graph element
print(f"| | |-- {basename(output)}")
if isnewer(input, output):
sections = " ".join(f"-j {section}" for section in sections)
soc.cmd(f"objcopy {sections} -O {fmt} {input} {output}").read()
return output
# ______ _ ______ _ ____ _____ _ _
# | ____| | | ____| | | | _ \_ _| \ | |
# | |__ | | | |__ | |_ ___ | |_) || | | \| |
# | __| | | | __| | __/ _ \ | _ < | | | . ` |
# | |____| |____| | | || (_) | | |_) || |_| |\ |
# |______|______|_| \__\___/ |____/_____|_| \_|
def elf2bin_ambz(input: str, ota_idx: int = 1) -> Tuple[int, str]:
def write_header(f: IO[bytes], start: int, end: int):
f.write(b"81958711")
f.write((end - start).to_bytes(length=4, byteorder="little"))
f.write(start.to_bytes(length=4, byteorder="little"))
f.write(b"\xff" * 16)
sections_ram = [
".ram_image2.entry",
".ram_image2.data",
".ram_image2.bss",
".ram_image2.skb.bss",
".ram_heap.data",
]
sections_xip = [".xip_image2.text"]
sections_rdp = [".ram_rdp.text"]
nmap = nm(input)
ram_start = nmap["__ram_image2_text_start__"]
ram_end = nmap["__ram_image2_text_end__"]
xip_start = nmap["__flash_text_start__"] - 0x8000020
# build output name
output = chname(input, f"image_0x{xip_start:06X}.ota{ota_idx}.bin")
out_ram = chname(input, f"ota{ota_idx}.ram_2.r.bin")
out_xip = chname(input, f"ota{ota_idx}.xip_image2.bin")
out_rdp = chname(input, f"ota{ota_idx}.rdp.bin")
# print graph element
print(f"| |-- {basename(output)}")
# objcopy required images
ram = objcopy(input, out_ram, sections_ram)
xip = objcopy(input, out_xip, sections_xip)
objcopy(input, out_rdp, sections_rdp)
# return if images are up to date
if not isnewer(ram, output) and not isnewer(xip, output):
return (xip_start, output)
# read and trim RAM image
ram = readbin(ram).rstrip(b"\x00")
# read XIP image
xip = readbin(xip)
# align images to 4 bytes
ram += b"\x00" * (((((len(ram) - 1) // 4) + 1) * 4) - len(ram))
xip += b"\x00" * (((((len(xip) - 1) // 4) + 1) * 4) - len(xip))
# write output file
with open(output, "wb") as f:
# write XIP header
write_header(f, 0, len(xip))
# write XIP image
f.write(xip)
# write RAM header
write_header(f, ram_start, ram_end)
# write RAM image
f.write(ram)
return (xip_start, output)
def elf2bin(input: str, ota_idx: int = 1) -> Tuple[int, str]:
checkfile(input)
if soc == SocType.AMBZ:
return elf2bin_ambz(input, ota_idx)
raise NotImplementedError(f"SoC ELF->BIN not implemented: {soc}")
def ldargs_parse(
args: List[str],
ld_ota1: str,
ld_ota2: str,
) -> List[Tuple[str, List[str]]]:
args1 = list(args)
args2 = list(args)
elf1 = elf2 = None
for i, arg in enumerate(args):
arg = arg.strip('"').strip("'")
if ".elf" in arg:
if not ld_ota1:
# single-OTA chip, return the output name
return [(arg, args)]
# append OTA index in filename
elf1 = chext(arg, "ota1.elf")
elf2 = chext(arg, "ota2.elf")
args1[i] = '"' + elf1 + '"'
args2[i] = '"' + elf2 + '"'
if arg.endswith(".ld") and ld_ota1:
# use OTA2 linker script
args2[i] = arg.replace(ld_ota1, ld_ota2)
return [(elf1, args1), (elf2, args2)]
def link2bin(
args: List[str],
ld_ota1: str = None,
ld_ota2: str = None,
) -> List[str]:
elfs = []
if soc.dual_ota:
# process linker arguments for dual-OTA chips
elfs = ldargs_parse(args, ld_ota1, ld_ota2)
else:
# just get .elf output name for single-OTA chips
elfs = ldargs_parse(args, None, None)
ota_idx = 1
for elf, ldargs in elfs:
# print graph element
print(f"|-- Image {ota_idx}: {basename(elf)}")
if isfile(elf):
unlink(elf)
ldargs = " ".join(ldargs)
soc.cmd(f"gcc {ldargs}").read()
checkfile(elf)
elf2bin(elf, ota_idx)
ota_idx += 1
if soc.dual_ota:
# copy OTA1 file as firmware.elf to make PIO understand it
elf, _ = ldargs_parse(args, None, None)[0]
copyfile(elfs[0][0], elf)
if __name__ == "__main__":
parser = ArgumentParser(
prog="link2bin",
description="Link to BIN format",
prefix_chars="@",
)
parser.add_argument("soc", type=str, help="SoC name/family short name")
parser.add_argument("ota1", type=str, help=".LD file OTA1 pattern")
parser.add_argument("ota2", type=str, help=".LD file OTA2 pattern")
parser.add_argument("args", type=str, nargs="*", help="Linker arguments")
args = parser.parse_args()
soc = next(soc for soc in SocType if soc.name == args.soc)
link2bin(args.args, args.ota1, args.ota2)

2
tools/uf2ota/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
*.uf2
*.bin

100
tools/uf2ota/dump.py Normal file
View File

@@ -0,0 +1,100 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-28.
from io import BytesIO, FileIO
from os import makedirs
from os.path import join
from typing import Dict, Tuple
from models import Opcode, Tag
from uf2 import UF2
from tools.util.intbin import inttole32, letoint, letosint
fs: Dict[str, Tuple[int, FileIO]] = {}
output_dir = ""
output_basename = ""
part1 = ""
part2 = ""
def write(part: str, offs: int, data: bytes):
global fs
if part not in fs or fs[part][0] != offs:
path = join(output_dir, output_basename + part + f"_0x{offs:x}.bin")
f = open(path, "wb")
if part in fs:
fs[part][1].close()
else:
f = fs[part][1]
fs[part] = (offs + f.write(data), f)
def update_parts(tags: Dict[Tag, bytes]):
global part1, part2
if Tag.LT_PART_1 in tags:
part1 = tags[Tag.LT_PART_1].decode()
part1 = ("1_" + part1) if part1 else None
if Tag.LT_PART_2 in tags:
part2 = tags[Tag.LT_PART_2].decode()
part2 = ("2_" + part2) if part2 else None
def uf2_dump(uf2: UF2, outdir: str):
global output_dir, output_basename
makedirs(outdir, exist_ok=True)
if Tag.LT_VERSION not in uf2.tags:
raise RuntimeError("Can only dump LibreTuya firmware images")
output_dir = outdir
output_basename = "_".join(
filter(
None,
[
uf2.tags.get(Tag.FIRMWARE, b"").decode(),
uf2.tags.get(Tag.VERSION, b"").decode(),
"lt" + uf2.tags[Tag.LT_VERSION].decode(),
uf2.tags.get(Tag.BOARD, b"").decode(),
],
)
)
output_basename += "_"
update_parts(uf2.tags)
for block in uf2.data:
# update target partition info
update_parts(block.tags)
# skip empty blocks
if not block.length:
continue
data1 = block.data if part1 else None
data2 = block.data if part2 else None
if Tag.LT_BINPATCH in block.tags:
# type 5, 6
data2 = bytearray(data2)
tag = block.tags[Tag.LT_BINPATCH]
binpatch = BytesIO(tag)
while binpatch.tell() < len(tag):
opcode = Opcode(binpatch.read(1)[0])
length = binpatch.read(1)[0]
data = binpatch.read(length)
if opcode == Opcode.DIFF32:
value = letosint(data[0:4])
for offs in data[4:]:
chunk = data2[offs : offs + 4]
chunk = letoint(chunk)
chunk += value
chunk = inttole32(chunk)
data2[offs : offs + 4] = chunk
data2 = bytes(data2)
if data1:
# types 1, 3, 4
write(part1, block.address, data1)
if data2:
# types 2, 3, 4
write(part2, block.address, data2)

137
tools/uf2ota/models.py Normal file
View File

@@ -0,0 +1,137 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-27.
from enum import IntEnum
class Tag(IntEnum):
VERSION = 0x9FC7BC # version of firmware file - UTF8 semver string
PAGE_SIZE = 0x0BE9F7 # page size of target device (32 bit unsigned number)
SHA2 = 0xB46DB0 # SHA-2 checksum of firmware (can be of various size)
DEVICE = 0x650D9D # description of device (UTF8)
DEVICE_ID = 0xC8A729 # device type identifier
# LibreTuya custom tags
OTA_VERSION = 0x5D57D0 # format version
BOARD = 0xCA25C8 # board name (lowercase code)
FIRMWARE = 0x00DE43 # firmware description / name
BUILD_DATE = 0x822F30 # build date/time as Unix timestamp
LT_VERSION = 0x59563D # LT version (semver)
LT_PART_1 = 0x805946 # OTA1 partition name
LT_PART_2 = 0xA1E4D7 # OTA2 partition name
LT_HAS_OTA1 = 0xBBD965 # image has any data for OTA1
LT_HAS_OTA2 = 0x92280E # image has any data for OTA2
LT_BINPATCH = 0xB948DE # binary patch to convert OTA1->OTA2
class Opcode(IntEnum):
DIFF32 = 0xFE # difference between 32-bit values
class Flags:
not_main_flash: bool = False
file_container: bool = False
has_family_id: bool = False
has_md5: bool = False
has_tags: bool = False
def encode(self) -> int:
val = 0
if self.not_main_flash:
val |= 0x00000001
if self.file_container:
val |= 0x00001000
if self.has_family_id:
val |= 0x00002000
if self.has_md5:
val |= 0x00004000
if self.has_tags:
val |= 0x00008000
return val
def decode(self, data: int):
self.not_main_flash = (data & 0x00000001) != 0
self.file_container = (data & 0x00001000) != 0
self.has_family_id = (data & 0x00002000) != 0
self.has_md5 = (data & 0x00004000) != 0
self.has_tags = (data & 0x00008000) != 0
def __str__(self) -> str:
flags = []
if self.not_main_flash:
flags.append("NMF")
if self.file_container:
flags.append("FC")
if self.has_family_id:
flags.append("FID")
if self.has_md5:
flags.append("MD5")
if self.has_tags:
flags.append("TAG")
return ",".join(flags)
class Input:
ota1_part: str = None
ota1_offs: int = 0
ota1_file: str = None
ota2_part: str = None
ota2_offs: int = 0
ota2_file: str = None
def __init__(self, input: str) -> None:
input = input.split(";")
n = len(input)
if n not in [2, 4]:
raise ValueError(
"Incorrect input format - should be part+offs;file[;part+offs;file]"
)
# just spread the same image twice for single-OTA scheme
if n == 2:
input += input
if input[0] and input[1]:
if "+" in input[0]:
(self.ota1_part, self.ota1_offs) = input[0].split("+")
self.ota1_offs = int(self.ota1_offs, 16)
else:
self.ota1_part = input[0]
self.ota1_file = input[1]
if input[2] and input[3]:
if "+" in input[2]:
(self.ota2_part, self.ota2_offs) = input[2].split("+")
self.ota2_offs = int(self.ota2_offs, 16)
else:
self.ota2_part = input[2]
self.ota2_file = input[3]
if self.ota1_offs != self.ota2_offs:
# currently, offsets cannot differ when storing images
# (this would require to actually store it twice)
raise ValueError(f"Offsets cannot differ ({self.ota1_file})")
@property
def is_single(self) -> bool:
return self.ota1_part == self.ota2_part and self.ota1_file == self.ota2_file
@property
def single_part(self) -> str:
return self.ota1_part or self.ota2_part
@property
def single_offs(self) -> int:
return self.ota1_offs or self.ota2_offs
@property
def single_file(self) -> str:
return self.ota1_file or self.ota2_file
@property
def has_ota1(self) -> bool:
return not not (self.ota1_part and self.ota1_file)
@property
def has_ota2(self) -> bool:
return not not (self.ota2_part and self.ota2_file)
@property
def is_simple(self) -> bool:
return self.ota1_file == self.ota2_file or not (self.has_ota1 and self.has_ota2)

View File

@@ -0,0 +1,17 @@
[tool.poetry]
name = "uf2ota"
version = "0.1.0"
description = "UF2 OTA update format"
authors = ["Kuba Szczodrzyński <kuba@szczodrzynski.pl>"]
license = "MIT"
[tool.poetry.dependencies]
python = "^3.7"
[tool.poetry.dev-dependencies]
black = "^22.3.0"
isort = "^5.10.1"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

152
tools/uf2ota/uf2.py Normal file
View File

@@ -0,0 +1,152 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-27.
from io import BytesIO, FileIO
from typing import Dict, List
from models import Tag
from uf2_block import Block
from tools.util.intbin import align_down, align_up, intto8, inttole16, inttole32
from tools.util.models import Family
class UF2:
f: FileIO
seq: int = 0
family: Family = None
tags: Dict[Tag, bytes] = {}
data: List[Block] = []
def __init__(self, f: FileIO) -> None:
self.f = f
def store(
self,
address: int,
data: bytes,
tags: Dict[Tag, bytes] = {},
block_size: int = 256,
):
if len(data) <= block_size:
block = Block(self.family)
block.tags = tags
block.address = address
block.data = data
block.length = len(data)
self.data.append(block)
return
for offs in range(0, len(data), block_size):
block = Block(self.family)
block.tags = tags
data_part = data[offs : offs + block_size]
block.address = address + offs
block.data = data_part
block.length = len(data_part)
self.data.append(block)
tags = {}
def put_str(self, tag: Tag, value: str):
self.tags[tag] = value.encode("utf-8")
def put_int32le(self, tag: Tag, value: int):
self.tags[tag] = inttole32(value)
def put_int16le(self, tag: Tag, value: int):
self.tags[tag] = inttole16(value)
def put_int8(self, tag: Tag, value: int):
self.tags[tag] = intto8(value)
def read(self, block_tags: bool = True) -> bool:
while True:
data = self.f.read(512)
if len(data) not in [0, 512]:
print(f"Block size invalid ({len(data)})")
return False
if not len(data):
break
block = Block()
if not block.decode(data):
return False
if self.family and self.family != block.family:
print(f"Mismatched family ({self.family} != {block.family})")
return False
self.family = block.family
if block.block_seq != self.seq:
print(f"Mismatched sequence number ({self.seq} != {block.block_seq}")
return False
self.seq += 1
if block_tags or not block.length:
self.tags.update(block.tags)
if block.length and not block.flags.not_main_flash:
self.data.append(block)
return True
def dump(self):
print(f"Family: {self.family.short_name} / {self.family.description}")
print(f"Tags:")
for k, v in self.tags.items():
if "\\x" not in str(v):
v = v.decode()
else:
v = v.hex()
print(f" - {k.name}: {v}")
print(f"Data chunks: {len(self.data)}")
print(f"Total binary size: {sum(bl.length for bl in self.data)}")
@property
def block_count(self) -> int:
cnt = len(self.data)
if self.tags:
cnt += 1
return cnt
def write_header(self):
comment = "Hi! Please visit https://kuba2k2.github.io/libretuya/ to read specifications of this file format."
bl = Block(self.family)
bl.flags.has_tags = True
bl.flags.not_main_flash = True
bl.block_seq = 0
bl.block_count = self.block_count
bl.tags = self.tags
data = bl.encode()
# add comment in the unused space
tags_len = align_up(Block.get_tags_length(bl.tags), 16)
comment_len = len(comment)
if 476 - 16 >= tags_len + comment_len:
space = 476 - 16 - tags_len
start = (space - comment_len) / 2
start = align_down(start, 16)
padding1 = b"\x00" * start
padding2 = b"\x00" * (476 - tags_len - comment_len - start)
data = (
data[0 : 32 + tags_len]
+ padding1
+ comment.encode()
+ padding2
+ data[-4:]
)
self.f.write(data)
def write(self):
if self.tags and self.seq == 0:
self.write_header()
self.seq += 1
bio = BytesIO()
for bl in self.data:
bl.block_count = self.block_count
bl.block_seq = self.seq
bio.write(bl.encode())
if self.seq % 128 == 0:
# write the buffer every 64 KiB
self.f.write(bio.getvalue())
bio = BytesIO()
self.seq += 1
self.f.write(bio.getvalue())

139
tools/uf2ota/uf2_block.py Normal file
View File

@@ -0,0 +1,139 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-27.
from math import ceil
from typing import Dict
from models import Flags, Tag
from tools.util.intbin import align_up, intto8, inttole24, inttole32, letoint
from tools.util.models import Family
from tools.util.platform import get_family
class Block:
flags: Flags
address: int = 0
length: int = 0
block_seq: int = 0
block_count: int = 0
file_size: int = 0
family: Family
data: bytes = None
md5_data: bytes = None
tags: Dict[Tag, bytes] = {}
def __init__(self, family: Family = None) -> None:
self.flags = Flags()
self.family = family
self.flags.has_family_id = not not self.family
def encode(self) -> bytes:
self.flags.has_tags = not not self.tags
# UF2 magic 1 and 2
data = b"\x55\x46\x32\x0A\x57\x51\x5D\x9E"
# encode integer variables
data += inttole32(self.flags.encode())
data += inttole32(self.address)
data += inttole32(self.length)
data += inttole32(self.block_seq)
data += inttole32(self.block_count)
if self.flags.file_container:
data += inttole32(self.file_size)
elif self.flags.has_family_id:
data += inttole32(self.family.id)
else:
data += b"\x00\x00\x00\x00"
if not self.data:
self.data = b""
# append tags
tags = b""
if self.flags.has_tags:
for k, v in self.tags.items():
tag_size = 4 + len(v)
tags += intto8(tag_size)
tags += inttole24(k.value)
tags += v
tag_size %= 4
if tag_size:
tags += b"\x00" * (4 - tag_size)
# append block data with padding
data += self.data
data += tags
data += b"\x00" * (476 - len(self.data) - len(tags))
data += b"\x30\x6F\xB1\x0A" # magic 3
return data
def decode(self, data: bytes) -> bool:
# check block size
if len(data) != 512:
print(f"Invalid block size ({len(data)})")
return False
# check Magic 1
if letoint(data[0:4]) != 0x0A324655:
print(f"Invalid Magic 1 ({data[0:4]})")
return False
# check Magic 2
if letoint(data[4:8]) != 0x9E5D5157:
print(f"Invalid Magic 2 ({data[4:8]})")
return False
# check Magic 3
if letoint(data[508:512]) != 0x0AB16F30:
print(f"Invalid Magic 13({data[508:512]})")
return False
self.flags.decode(letoint(data[8:12]))
self.address = letoint(data[12:16])
self.length = letoint(data[16:20])
self.block_seq = letoint(data[20:24])
self.block_count = letoint(data[24:28])
if self.flags.file_container:
self.file_size = letoint(data[28:32])
if self.flags.has_family_id:
self.family = get_family(id=letoint(data[28:32]))
if self.flags.has_md5:
self.md5_data = data[484:508] # last 24 bytes of data[]
# decode tags
self.tags = {}
if self.flags.has_tags:
tags = data[32 + self.length :]
i = 0
while i < len(tags):
length = tags[i]
if not length:
break
tag_type = letoint(tags[i + 1 : i + 4])
tag_data = tags[i + 4 : i + length]
self.tags[Tag(tag_type)] = tag_data
i += length
i = int(ceil(i / 4) * 4)
self.data = data[32 : 32 + self.length]
return True
@staticmethod
def get_tags_length(tags: Dict[Tag, bytes]) -> int:
out = 0
# add tag headers
out += 4 * len(tags)
# add all tag lengths, padded to 4 bytes
out += sum(align_up(l, 4) for l in map(len, tags.values()))
# add final 0x00 tag
out += 4
return out
def __str__(self) -> str:
flags = self.flags
address = hex(self.address)
length = hex(self.length)
block_seq = self.block_seq
block_count = self.block_count
file_size = self.file_size
family = self.family.short_name
tags = [(k.name, v) for k, v in self.tags.items()]
return f"Block[{block_seq}/{block_count}](flags={flags}, address={address}, length={length}, file_size={file_size}, family={family}, tags={tags})"

145
tools/uf2ota/uf2ota.py Normal file
View File

@@ -0,0 +1,145 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-27.
import sys
from os.path import dirname, join
sys.path.append(join(dirname(__file__), "..", ".."))
from argparse import ArgumentParser
from datetime import datetime
from zlib import crc32
from dump import uf2_dump
from models import Input, Tag
from uf2 import UF2
from uf2_block import Block
from utils import binpatch32
from tools.util.platform import get_family
BLOCK_SIZE = 256
def cli():
parser = ArgumentParser("uf2ota", description="UF2 OTA update format")
parser.add_argument("action", choices=["info", "dump", "write"])
parser.add_argument("inputs", nargs="+", type=str)
parser.add_argument("--output", help="Output .uf2 binary", type=str)
parser.add_argument("--family", help="Family name", type=str)
parser.add_argument("--board", help="Board name/code", type=str)
parser.add_argument("--version", help="LibreTuya core version", type=str)
parser.add_argument("--fw", help="Firmware name:version", type=str)
parser.add_argument("--date", help="Build date (Unix, default now)", type=int)
args = parser.parse_args()
if args.action == "info":
with open(args.inputs[0], "rb") as f:
uf2 = UF2(f)
if not uf2.read():
raise RuntimeError("Reading UF2 failed")
uf2.dump()
return
if args.action == "dump":
input = args.inputs[0]
outdir = input + "_dump"
with open(input, "rb") as f:
uf2 = UF2(f)
if not uf2.read(block_tags=False):
raise RuntimeError("Reading UF2 failed")
uf2_dump(uf2, outdir)
return
out = args.output or "out.uf2"
with open(out, "wb") as f:
uf2 = UF2(f)
uf2.family = get_family(args.family)
# store global tags (for entire file)
if args.board:
uf2.put_str(Tag.BOARD, args.board.lower())
key = f"LibreTuya {args.board.lower()}"
uf2.put_int32le(Tag.DEVICE_ID, crc32(key.encode()))
if args.version:
uf2.put_str(Tag.LT_VERSION, args.version)
if args.fw:
if ":" in args.fw:
(fw_name, fw_ver) = args.fw.split(":")
uf2.put_str(Tag.FIRMWARE, fw_name)
uf2.put_str(Tag.VERSION, fw_ver)
else:
uf2.put_str(Tag.FIRMWARE, args.fw)
uf2.put_int8(Tag.OTA_VERSION, 1)
uf2.put_str(Tag.DEVICE, "LibreTuya")
uf2.put_int32le(Tag.BUILD_DATE, args.date or int(datetime.now().timestamp()))
any_ota1 = False
any_ota2 = False
for input in args.inputs:
input = Input(input)
any_ota1 = any_ota1 or input.has_ota1
any_ota2 = any_ota2 or input.has_ota2
# store local tags (for this image only)
tags = {
Tag.LT_PART_1: input.ota1_part.encode() if input.has_ota1 else b"",
Tag.LT_PART_2: input.ota2_part.encode() if input.has_ota2 else b"",
}
if input.is_simple:
# single input image:
# - same image and partition (2 args)
# - same image but different partitions (4 args)
# - only OTA1 image
# - only OTA2 image
with open(input.single_file, "rb") as f:
data = f.read()
uf2.store(input.single_offs, data, tags, block_size=BLOCK_SIZE)
continue
# different images and partitions for both OTA schemes
with open(input.ota1_file, "rb") as f:
data1 = f.read()
with open(input.ota2_file, "rb") as f:
data2 = f.read()
if len(data1) != len(data2):
raise RuntimeError(
f"Images must have same lengths ({len(data1)} vs {len(data2)})"
)
for i in range(0, len(data1), 256):
block1 = data1[i : i + 256]
block2 = data2[i : i + 256]
if block1 == block2:
# blocks are identical, simply store them
uf2.store(
input.single_offs + i, block1, tags, block_size=BLOCK_SIZE
)
tags = {}
continue
# calculate max binpatch length (incl. existing tags and binpatch tag header)
max_length = 476 - BLOCK_SIZE - Block.get_tags_length(tags) - 4
# try 32-bit binpatch for best space optimization
binpatch = binpatch32(block1, block2, bladdr=i)
if len(binpatch) > max_length:
raise RuntimeError(
f"Binary patch too long - {len(binpatch)} > {max_length}"
)
tags[Tag.LT_BINPATCH] = binpatch
uf2.store(input.single_offs + i, block1, tags, block_size=BLOCK_SIZE)
tags = {}
uf2.put_int8(Tag.LT_HAS_OTA1, any_ota1 * 1)
uf2.put_int8(Tag.LT_HAS_OTA2, any_ota2 * 1)
uf2.write()
if __name__ == "__main__":
cli()

72
tools/uf2ota/utils.py Normal file
View File

@@ -0,0 +1,72 @@
# Copyright (c) Kuba Szczodrzyński 2022-05-27.
from typing import Dict, List, Tuple
from models import Opcode
from tools.util.intbin import intto8, letoint, sinttole32
def bindiff(
data1: bytes, data2: bytes, width: int = 1, single: bool = False
) -> Dict[int, Tuple[bytes, bytes]]:
out: Dict[int, Tuple[bytes, bytes]] = {}
offs = -1
diff1 = b""
diff2 = b""
for i in range(0, len(data1), width):
block1 = data1[i : i + width]
block2 = data2[i : i + width]
if block1 == block2:
# blocks are equal again
if offs != -1:
# store and reset current difference
out[offs] = (diff1, diff2)
offs = -1
diff1 = b""
diff2 = b""
continue
# blocks still differ
if single:
# single block per difference, so just store it
out[i] = (block1, block2)
else:
if offs == -1:
# difference starts here
offs = i
diff1 += block1
diff2 += block2
return out
def binpatch32(block1: bytes, block2: bytes, bladdr: int = 0) -> bytes:
# compare blocks:
# - in 4 byte (32 bit) chunks
# - report a single chunk in each difference
diffs = bindiff(block1, block2, width=4, single=True)
binpatch: Dict[int, List[int]] = {}
# gather all repeating differences (i.e. memory offsets for OTA1/OTA2)
for offs, diff in diffs.items():
(diff1, diff2) = diff
diff1 = letoint(diff1)
diff2 = letoint(diff2)
diff = diff2 - diff1
if diff in binpatch:
# difference already in this binpatch, add the offset
binpatch[diff].append(offs)
else:
# a new difference value
binpatch[diff] = [offs]
# print(f"Block at 0x{bladdr:x}+{offs:02x} -> {diff1:08x} - {diff2:08x} = {diff2-diff1:x}")
# print(f"Block at 0x{bladdr:x}: {len(binpatch)} difference(s) at {sum(len(v) for v in binpatch.values())} offsets")
# write binary patches
out = b""
for diff, offs in binpatch.items():
out += intto8(Opcode.DIFF32.value)
out += intto8(len(offs) + 4)
out += sinttole32(diff)
out += bytes(offs)
return out

26
tools/upload/binpatch.py Normal file
View File

@@ -0,0 +1,26 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
from io import BytesIO
from tools.uf2ota.models import Opcode
from tools.util.intbin import inttole32, letoint, letosint
def binpatch_diff32(data: bytearray, patch: bytes) -> bytearray:
diff = letosint(patch[0:4])
for offs in patch[4:]:
value = letoint(data[offs : offs + 4])
value += diff
data[offs : offs + 4] = inttole32(value)
return data
def binpatch_apply(data: bytearray, binpatch: bytes) -> bytearray:
io = BytesIO(binpatch)
while io.tell() < len(binpatch):
opcode = io.read(1)[0]
length = io.read(1)[0]
bpdata = io.read(length)
if opcode == Opcode.DIFF32:
data = binpatch_diff32(data, bpdata)
return data

157
tools/upload/ctx.py Normal file
View File

@@ -0,0 +1,157 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
from datetime import datetime
from io import BytesIO
from typing import Dict, Tuple
from tools.uf2ota.models import Tag
from tools.uf2ota.uf2 import UF2
from tools.upload.binpatch import binpatch_apply
from tools.util.intbin import letoint
from tools.util.obj import get
from tools.util.platform import get_board_manifest
class UploadContext:
uf2: UF2
seq: int = 0
part1: str = None
part2: str = None
has_ota1: bool
has_ota2: bool
board_manifest: dict = None
def __init__(self, uf2: UF2) -> None:
self.uf2 = uf2
self.has_ota1 = uf2.tags.get(Tag.LT_HAS_OTA1, None) == b"\x01"
self.has_ota2 = uf2.tags.get(Tag.LT_HAS_OTA2, None) == b"\x01"
@property
def fw_name(self) -> str:
return self.uf2.tags.get(Tag.FIRMWARE, b"").decode()
@property
def fw_version(self) -> str:
return self.uf2.tags.get(Tag.VERSION, b"").decode()
@property
def lt_version(self) -> str:
return self.uf2.tags.get(Tag.LT_VERSION, b"").decode()
@property
def board(self) -> str:
return self.uf2.tags.get(Tag.BOARD, b"").decode()
@property
def build_date(self) -> datetime:
if Tag.BUILD_DATE not in self.uf2.tags:
return None
return datetime.fromtimestamp(letoint(self.uf2.tags[Tag.BUILD_DATE]))
def get_offset(self, part: str, offs: int) -> int:
if not self.board_manifest:
self.board_manifest = get_board_manifest(self.board)
part = get(self.board_manifest, f"flash.{part}")
(offset, length) = map(lambda x: int(x, 16), part.split("+"))
if offs >= length:
return None
return offset + offs
def read(self, ota_idx: int = 1) -> Tuple[str, int, bytes]:
"""Read next available data block for the specified OTA scheme.
Returns:
Tuple[str, int, bytes]: target partition, relative offset, data block
"""
if ota_idx not in [1, 2]:
print(f"Invalid OTA index - {ota_idx}")
return None
if ota_idx == 1 and not self.has_ota1:
print(f"No data for OTA index - {ota_idx}")
return None
if ota_idx == 2 and not self.has_ota2:
print(f"No data for OTA index - {ota_idx}")
return None
for _ in range(self.seq, len(self.uf2.data)):
block = self.uf2.data[self.seq]
self.seq += 1
part1 = block.tags.get(Tag.LT_PART_1, None)
part2 = block.tags.get(Tag.LT_PART_2, None)
if part1 and part2:
self.part1 = part1.decode()
self.part2 = part2.decode()
elif part1 or part2:
print(f"Only one target partition specified - {part1} / {part2}")
return None
if not block.data:
continue
part = None
if ota_idx == 1:
part = self.part1
elif ota_idx == 2:
part = self.part2
if not part:
continue
# got data and target partition
offs = block.address
data = block.data
if ota_idx == 2 and Tag.LT_BINPATCH in block.tags:
binpatch = block.tags[Tag.LT_BINPATCH]
data = bytearray(data)
data = binpatch_apply(data, binpatch)
data = bytes(data)
return (part, offs, data)
return (None, 0, None)
def collect(self, ota_idx: int = 1) -> Dict[int, BytesIO]:
"""Read all UF2 blocks. Gather continuous data parts into sections
and their flashing offsets.
Returns:
Dict[int, BytesIO]: map of flash offsets to streams with data
"""
out: Dict[int, BytesIO] = {}
while True:
ret = self.read(ota_idx)
if not ret:
return False
(part, offs, data) = ret
if not data:
break
offs = self.get_offset(part, offs)
if offs is None:
return False
# find BytesIO in the dict
for io_offs, io_data in out.items():
if io_offs + len(io_data.getvalue()) == offs:
io_data.write(data)
offs = 0
break
if offs == 0:
continue
# create BytesIO at specified offset
io = BytesIO()
io.write(data)
out[offs] = io
# rewind BytesIO back to start
for io in out.values():
io.seek(0)
return out

View File

@@ -0,0 +1,67 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
from io import BytesIO
from tools.upload.ctx import UploadContext
from tools.upload.rtltool import RTLXMD
from tools.util.intbin import letoint
def upload_uart(
ctx: UploadContext,
port: str,
baud: int = None,
**kwargs,
) -> bool:
prefix = "| |--"
rtl = RTLXMD(port=port)
print(prefix, f"Connecting to {port}...")
if not rtl.connect():
print(prefix, f"Failed to connect on port {port}")
return False
# read system data to get active OTA index
io = BytesIO()
if not rtl.ReadBlockFlash(io, offset=0x9000, size=256):
print(prefix, "Failed to read from 0x9000")
return False
# get as bytes
system = io.getvalue()
if len(system) != 256:
print(prefix, f"Length invalid while reading from 0x9000 - {len(system)}")
return False
# read OTA switch value
ota_switch = bin(letoint(system[4:8]))[2:]
# count 0-bits
ota_idx = 1 + (ota_switch.count("0") % 2)
# validate OTA2 address in system data
if ota_idx == 2:
ota2_addr = letoint(system[0:4]) & 0xFFFFFF
part_addr = ctx.get_offset("ota2", 0)
if ota2_addr != part_addr:
print(
prefix,
f"Invalid OTA2 address on chip - found {ota2_addr}, expected {part_addr}",
)
return False
print(prefix, f"Flashing image to OTA {ota_idx}...")
# collect continuous blocks of data
parts = ctx.collect(ota_idx=ota_idx)
# write blocks to flash
for offs, data in parts.items():
offs |= 0x8000000
length = len(data.getvalue())
data.seek(0)
print(prefix, f"Writing {length} bytes to 0x{offs:06x}")
if not rtl.WriteBlockFlash(data, offs, length):
print(prefix, f"Writing failed at 0x{offs:x}")
return False
return True
def upload(ctx: UploadContext, protocol: str, **kwargs) -> bool:
if protocol == "uart":
return upload_uart(ctx, **kwargs)
print(f"Unknown upload protocol - {protocol}")
return False

53
tools/upload/uf2upload.py Normal file
View File

@@ -0,0 +1,53 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
import sys
from os.path import dirname, join
from time import time
sys.path.append(join(dirname(__file__), "..", ".."))
sys.path.append(join(dirname(__file__), "..", "uf2ota"))
from argparse import ArgumentParser, FileType
from tools.uf2ota.uf2 import UF2
from tools.upload.ctx import UploadContext
# TODO document this tool
if __name__ == "__main__":
parser = ArgumentParser("uf2upload", description="UF2 uploader")
parser.add_argument("file", type=FileType("rb"), help=".uf2 file")
subp = parser.add_subparsers(dest="protocol", help="Upload protocol", required=True)
parser_uart = subp.add_parser("uart", help="UART uploader")
parser_uart.add_argument("port", type=str, help="Serial port device")
parser_uart.add_argument("-b", "--baud", type=int, help="Serial baudrate")
args = parser.parse_args()
uf2 = UF2(args.file)
if not uf2.read(block_tags=False):
exit(1)
ctx = UploadContext(uf2)
print(
f"|-- {ctx.fw_name} {ctx.fw_version} @ {ctx.build_date} -> {ctx.board} via {args.protocol}"
)
start = time()
args = dict(args._get_kwargs())
if uf2.family.code == "ambz":
from uf2_rtltool import upload
if not upload(ctx, **args):
exit(1)
else:
print(f"Unsupported upload family - {uf2.family.name}")
exit(1)
duration = time() - start
print(f"|-- Finished in {duration:.3f} s")
exit(0)

14
tools/util/crypto.py Normal file
View File

@@ -0,0 +1,14 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
def crc16(data):
# https://gist.github.com/pintoXD/a90e398bba5a1b6c121de4e1265d9a29
crc = 0x0000
for b in data:
crc ^= b
for j in range(0, 8):
if (crc & 0x0001) > 0:
crc = (crc >> 1) ^ 0xA001
else:
crc = crc >> 1
return crc

61
tools/util/intbin.py Normal file
View File

@@ -0,0 +1,61 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
def bswap(data: bytes) -> bytes:
return bytes(reversed(data))
def betoint(data: bytes) -> int:
return int.from_bytes(data, byteorder="big")
def letoint(data: bytes) -> int:
return int.from_bytes(data, byteorder="little")
def betosint(data: bytes) -> int:
return int.from_bytes(data, byteorder="big", signed=True)
def letosint(data: bytes) -> int:
return int.from_bytes(data, byteorder="little", signed=True)
def inttole32(data: int) -> bytes:
return data.to_bytes(length=4, byteorder="little")
def inttole24(data: int) -> bytes:
return data.to_bytes(length=3, byteorder="little")
def inttole16(data: int) -> bytes:
return data.to_bytes(length=2, byteorder="little")
def intto8(data: int) -> bytes:
return data.to_bytes(length=1, byteorder="big")
def sinttole32(data: int) -> bytes:
return data.to_bytes(length=4, byteorder="little", signed=True)
def sinttole24(data: int) -> bytes:
return data.to_bytes(length=3, byteorder="little", signed=True)
def sinttole16(data: int) -> bytes:
return data.to_bytes(length=2, byteorder="little", signed=True)
def sintto8(data: int) -> bytes:
return data.to_bytes(length=1, byteorder="little", signed=True)
def align_up(x: int, n: int) -> int:
return int((x - 1) // n + 1) * n
def align_down(x: int, n: int) -> int:
return int(x // n) * n

25
tools/util/models.py Normal file
View File

@@ -0,0 +1,25 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
from typing import List
class Family:
id: int
short_name: str
description: str
name: str = None
code: str = None
url: str = None
sdk: str = None
framework: str = None
mcus: List[str] = []
def __init__(self, data: dict):
for key, value in data.items():
if key == "id":
self.id = int(value, 16)
else:
setattr(self, key, value)
def __eq__(self, __o: object) -> bool:
return isinstance(__o, Family) and self.id == __o.id

29
tools/util/obj.py Normal file
View File

@@ -0,0 +1,29 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
import json
from typing import Union
def merge_dicts(d1, d2, path=None):
if path is None:
path = []
for key in d2:
if key in d1 and isinstance(d1[key], dict) and isinstance(d2[key], dict):
merge_dicts(d1[key], d2[key], path + [str(key)])
else:
d1[key] = d2[key]
return d1
def load_json(file: str) -> Union[dict, list]:
with open(file, "r", encoding="utf-8") as f:
return json.load(f)
def get(data: dict, path: str):
if not isinstance(data, dict) or not path:
return None
if "." not in path:
return data.get(path, None)
key, _, path = path.partition(".")
return get(data.get(key, None), path)

70
tools/util/platform.py Normal file
View File

@@ -0,0 +1,70 @@
# Copyright (c) Kuba Szczodrzyński 2022-06-02.
from os.path import dirname, isfile, join
from typing import Dict, List, Union
from tools.util.models import Family
from tools.util.obj import load_json, merge_dicts
boards_base: Dict[str, dict] = {}
families: List[Family] = []
def get_board_manifest(board: Union[str, dict]) -> dict:
boards_dir = join(dirname(__file__), "..", "..", "boards")
if not isinstance(board, dict):
if not isfile(board):
board = join(boards_dir, f"{board}.json")
board = load_json(board)
if "_base" in board:
base = board["_base"]
if not isinstance(base, list):
base = [base]
result = None
for base_name in base:
if base_name not in boards_base:
file = join(boards_dir, "_base", f"{base_name}.json")
boards_base[base_name] = load_json(file)
if not result:
result = boards_base[base_name]
else:
merge_dicts(result, boards_base[base_name])
merge_dicts(result, board)
board = result
return board
def get_families() -> List[Family]:
global families
if families:
return families
file = join(dirname(__file__), "..", "..", "families.json")
families = [Family(f) for f in load_json(file)]
return families
def get_family(
any: str = None,
id: Union[str, int] = None,
short_name: str = None,
name: str = None,
code: str = None,
) -> Family:
if any:
id = any
short_name = any
name = any
code = any
if id and isinstance(id, str) and id.startswith("0x"):
id = int(id, 16)
for family in get_families():
if id and family.id == id:
return family
if short_name and family.short_name == short_name:
return family
if name and family.name == name:
return family
if code and family.code == code:
return family
text = ", ".join(filter(None, [id, short_name, name, code]))
raise ValueError(f"Family not found - {text}")