The Open-Source project: Emscripten

I have been working on a WebAssembly project porting some desktop code to run in a web browser and we have been using Emscripten as our tool chain. Emscripten is an open-source compiler toolchain that allows you to compile C/C++ code to WebAssembly, enabling it to run in web browsers. But it is more than a just compiler, Emscripten is a full SDK as it also includes the runtime library allowing interaction with the web browser with JavaScript.

The bug, a race condition?

My system has an Emscripten compiled C program running in a Web Worker that writes data that is read by another Web Worker. A SharedArrayBuffer is used to connect the two Web Workers together, basically as a queue. When the queue was big enough the program behaved exactly as expected, however when the queue filled up I would observe out of order and corrupt data on the read side.

For a simple example, if the C program did fprintf(f, "abcdefghijklmnopqrstuvwxyz\n“) the receiving Web Worker would receive a random character replaced with a newline, such as “abcdefg\nijklmnopqrstuvwxyz\n". I eventually determined that the corruption only occurred if the queue went from full to not full in the middle of processing a single write.

Determining this was difficult because any debugging I did or logging I added to my program changed the behaviour sufficiently to remove or at least significantly relocate when I saw the bug. This is why I concluded that it was a race condition.

Understanding the problem was related to the queue being full allowed me to configure my test case with a full buffer that I could control when it transitioned from full to not full (by careful placement of breakpoint on the receiving side) allowing me to step through a reproducible test case.

Once I was able to step through the code finding the bug was very easy, a simple loop termination condition was missing in the implementation of the doWritev function that underpins much of the stdio output handling.

Submitting my first issue

The first step in contributing anything non-trivial to an open-source project is to create a bug report. This allows you to discuss with the maintainers and the rest of the project’s ecosystem about the validity of your findings.

I summarized all my findings, along with my probable fix as a bug report in Emscripten’s GitHub issue tracker, Issue #22258.

The Emscripten maintainers were very responsive to my bug report which allowed me to have confidence that my assessment was probably correct, and that the community would be receptive to me doing the work to submit a Pull Request to resolve the issue.

Creating a Pull Request – have a good test case!

I could have created a PR with the one-line fix and called it a day as I already had a fix applied in my code base. The problem was possibly well enough understood to be merged with only that. However, to demonstrate exactly what the fix meant and to ensure it did not regress in the future writing a new test case was something I felt was important.

I had a working test as part of the test suite of my internal work, but it relied on my components that were not suitable to contribute to Emscripten at this time. My tests also relied on running the test in a tight loop to increase the chances of hitting the race condition. Therefore, I needed a more suitable test case.

Emscripten has an extensive test suite. The first step was to make sure I could run some tests and find a similar test I could copy as a starting point. Running some tests was straightforward with the test/runner program. To identify a test that may cover the same code I ran git blame on doWritev and identified a recent fix with associated test case that I used as my basis for my test, test_fs_writev.

I was able to write a simple test case that demonstrated the problem. The process of writing the test case also allowed me to understand the problem better, as well as learn a lot about the Emscripten code base. I captured my better understanding of the original problem in a new comment on my issue, and I submitted a Pull Request with my fix and test case, PR #22261.

The Review Process

Submitting the pull request was just the beginning. The next step was the review process. Emscripten maintainers reviewed my changes, provided feedback, and suggested improvements. This process was incredibly valuable as it helped me learn more and gave me the opportunity to dive into other related parts of the code base, such as the new WasmFS component to verify it didn’t suffer from the same issue. I was also able to submit a few additional smaller issues and PRs that were triggered by working on this bug fix.

After addressing the feedback, my pull request was merged into main. From observing the bug in my project to getting the PR merged was approximately one week and I am pleased that my bug fix will be part of an upcoming release of Emscripten.

Encouragement for New Contributors

I have been an open-source maintainer for many years, including being a project lead on various Eclipse open-source projects. This was my first contribution in a while on a project I had no prior relationship with. It was really useful to be in the position of a new contributor again as I plan to adapt some of my experiences to improving the other open-source projects that I am part of.

If you’re considering contributing to open-source projects, I highly encourage you to take the plunge. The Emscripten is a healthy project with attentive maintainers who engage well with potential contributors. I hope I can help provide contributors to Eclipse projects have the same sense of pride at contributing as I got from being a first-time contributor to Emscripten.