PlatformIO STM32 Build Fails on Windows Chinese Version: TMP File Encoding Issue

[yehjf]

Describe the bug
When I use PlatformIO to compile an STM32 program, I encounter a linking failure error. After debugging, I found that when the command becomes too long, the command arguments are written into a temporary file using the TempFileMunge method. However, since I'm using the Chinese version of Windows, the default encoding is cp936. If the temporary file is written using UTF-8 encoding, it causes errors when gcc tries to read it.
On the other hand, if the temporary file is written using GBK encoding, everything works fine.

os.write(fd, bytearray(join_char.join(args) + "\n", 'gbk'))

Therefore, the correct encoding should be selected based on the system's default locale when writing the temporary file.

Required information

• Link to SCons Users thread discussing your issue.
• Version of SCons
  scons-local-4.8.1
• Version of Python
  Python 3.11.7 (tags/v3.11.7:fa7a6f2, Dec 4 2023, 19:24:49) [MSC v.1937 64 bit (AMD64)] on win32
• Which python distribution if applicable (python.org, cygwin, anaconda, macports, brew,etc)
• How you installed SCons
• What Platform are you on? (Linux/Windows and which version)
  Windows 10 Pro 22H2 19045.6093 (Chinese Version)
• How to reproduce your issue? Please include a small self contained reproducer. Likely a SConstruct should do for most issues.
• How you invoke scons (The command line you're using "scons --flags some_arguments")

[bdbaddog]

You should file your issue with PlatformIO!
Also 4.8.1 is not the newest version of SCons.
Please try with 4.9.1

If 4.9.1 doesn't fix it, let us know and we'll take a look.

[yehjf]

I've tried it already. PlatformIO uses the package tool-scons@4.40801.0, which references scons@4.8.1, so I can't simply upgrade it separately.
However, I looked at the relevant code in version 4.9.1, and the temporary file is still written using UTF-8 encoding. Therefore, I suspect that compatibility issues still exist.

SCons/Platform/init.py:282

os.write(fd, bytearray(join_char.join(args) + "\n", encoding="utf-8"))

[bdbaddog]

@yehjf what does the following python code output on your system

import sys
print(sys.getdefaultencoding())

[yehjf]

Display utf-8

After my debugging, I found that after writing the temporary file (which contains Chinese characters), the command is eventually executed via the spawnve method:󠁪

C:\WINDOWS\System32\cmd.exe /C arm-none-eabi-g++ @D:\\Desktop\\测试\\firmware\\.pio\\build\\bluepill_f103c6\\tmpagpc__cg.tmp

The key issue is that the g++ program reads this temporary file using GBK encoding. I suspect that g++ uses the default encoding from the Windows locale settings when opening the tmp file.

I tried running the command chcp 65001 before execution to switch to UTF-8 mode, but it still didn't work.
However, when I changed the encoding to GBK when writing the tmp file, the command was executed successfully.

By default, the result of chcp on my system is 936, which corresponds to GBK encoding.

[mwichmann]

Yeah, these days Python itself defaults to utf-8 (for the most part), and we're forcing it in that case, that's not necessarily the case for an external command.

[bdbaddog]

Can you share the output of "set" in a cmd shell on your computer?

…

On Wed, Jul 16, 2025 at 5:44 AM Mats Wichmann ***@***.***> wrote: *mwichmann* left a comment (SCons/scons#4746) <#4746 (comment)> Yeah, these days Python itself defaults to utf-8 (for the most part), and we're forcing it in that case, that's not necessarily the case for an external command. — Reply to this email directly, view it on GitHub <#4746 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAUCNHPJSTSEB3KZR76K7D3IZCLRAVCNFSM6AAAAACBQV6MROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTANZYGQYTQMRZHE> . You are receiving this because you commented.Message ID: ***@***.***>

[yehjf]

ALLUSERSPROFILE=C:\ProgramData
ANDROID_SDK_ROOT=D:\android-sdk
APPDATA=C:\Users\Hello\AppData\Roaming
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=DESKTOP-P357RS7
ComSpec=C:\WINDOWS\system32\cmd.exe
DriverData=C:\Windows\System32\Drivers\DriverData
GHIDRA_INSTALL_DIR=D:\scoop\apps\ghidra\current
GIT_INSTALL_ROOT=D:\scoop\apps\git\current
GRADLE_USER_HOME=D:\scoop\apps\gradle\current\.gradle
HADOOP_HOME=D:\hadoop-3.3.5
HOMEDRIVE=C:
HOMEPATH=\Users\Hello
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_181
LOCALAPPDATA=C:\Users\Hello\AppData\Local
LOGONSERVER=\\DESKTOP-P357RS7
NUMBER_OF_PROCESSORS=6
OneDrive=C:\Users\Hello\OneDrive
OneDriveConsumer=C:\Users\Hello\OneDrive
OS=Windows_NT
Path=D:\scoop\apps\shims;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64;C:\Program Files (x86)\ZeroTier\One\;C:\Program Files\dotnet\;E:\metasploit-framework\bin\;C:\Program Files\Docker\Docker\resources\bin;D:\hadoop-3.3.5\bin;D:\scoop\apps\python36\current\Scripts;D:\scoop\apps\python36\current;D:\scoop\apps\nodejs\current\bin;D:\scoop\apps\nodejs\current;D:\scoop\apps\vscode\current\bin;D:\scoop\apps\python39\current\Scripts;D:\scoop\apps\python39\current;D:\scoop\apps\openjdk\current\bin;D:\scoop\apps\python310\current\Scripts;D:\scoop\apps\python310\current;D:\scoop\apps\python\current\Scripts;D:\scoop\apps\python\current;C:\Users\Hello\go\bin;D:\scoop\shims;C:\Users\Hello\AppData\Local\Microsoft\WindowsApps;C:\Users\Hello\.dotnet\tools
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=9e0d
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PSModulePath=C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
PUBLIC=C:\Users\Public
PYSPARK_DRIVER_PYTHON=D:\scoop\apps\python\current\python.exe
PYSPARK_PYTHON=D:\scoop\apps\python\current\python.exe
SBT_HOME=D:\scoop\apps\sbt\current
SCALA_HOME=D:\scoop\apps\scala2\current
SESSIONNAME=Console
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=C:\Users\Hello\AppData\Local\Temp
TMP=C:\Users\Hello\AppData\Local\Temp
USERDOMAIN=DESKTOP-P357RS7
USERDOMAIN_ROAMINGPROFILE=DESKTOP-P357RS7
USERNAME=Hello
USERPROFILE=C:\Users\Hello
windir=C:\WINDOWS
WIRESHARK_CONFIG_DIR=D:\scoop\apps\wireshark\current\Data
WIRESHARK_DATA_DIR=D:\scoop\apps\wireshark\current\Data
ZES_ENABLE_SYSMAN=1

[bdbaddog]

How about try this:

import locale

default_encoding = locale.getpreferredencoding(False)
print(f"The Windows default encoding for Python is: {default_encoding}")

[yehjf]

The Windows default encoding for Python is: cp936

[mwichmann]

To some of the above: sys.getdefaultencoding() is not useful, it's defined to return utf-8 these days. That's for Python. locale.getencoding() is far more interesting.

sys.getdefaultencoding()
Return 'utf-8'. This is the name of the default string encoding, used in methods like str.encode.

I wonder if the answer is to just omit the encoding, letting it default? Hard to test for me, I only have English Windows, and on Linux it's hard to force things into non-UTF-8.

[jcbrill]

I only have English Windows

With a VMWare or VirtualBox Windows VM, it would be possible to change the Windows language/locale for testing.

I did that for some testing a while back using German. I think the language and keyboard was changed I don't remember exactly. Not knowing the language made using the keyboard a challenge to launch the tests though.

[mwichmann]

I've done that too, but it needs to be a different enough locale to trigger the problem.