Ensuring Robust Unicode Handling by Explicitly Setting UTF-8 Encoding in File I/O
from: gh-156810: Write the profiler's collapsed-stack export as UTF-8
The Concept
File encoding specifies how text is represented as bytes when reading from or writing to files. Explicitly setting UTF-8 encoding in file operations ensures consistent handling of Unicode characters across different environments and locales, preventing errors like UnicodeEncodeError when non-ASCII characters are present.
How This PR Does It
In this PR, the `CollapsedStackCollector.export` method was changed to open its output file with `encoding='utf-8'` and `errors='surrogatepass'`. Previously, it used the default encoding by calling `open(filename, 'w')`, which caused crashes when writing non-ASCII frame names or surrogate-escaped filenames. By aligning the encoding with other exporters and allowing surrogate escapes, the code safely writes frame names verbatim without raising encoding errors, and the added test verifies this fix by failing without it.
Why It Matters
Understanding and explicitly specifying file encodings prevents unexpected crashes and data corruption when handling Unicode text, especially in diverse environments with varying locale settings. This leads to more reliable and portable code when dealing with profiling data or any text output.
Try It Yourself
Review the other exporters in `stack_collector.py` that write to files. How do they handle encoding and error strategies? Modify one of them to use a different error handling mode (e.g., `errors='replace'`) and observe how the output changes when encountering invalid Unicode sequences. What trade-offs does this introduce compared to `surrogatepass` used in this PR?