ConPTY Performance in Eclipse Terminal

Background

For many years the Eclipse IDE has provided an integrated terminal (called Eclipse TM Terminal) and now maintained by the Eclipse CDT team. On Windows the terminal uses the amazing WinPTY library to provide a PTY as Windows did not come with one. For the last number of years, Windows 10 has a native version called Windows Pseudo Console (ConPTY) which programs such as VSCode and Eclipse Theia have converted to using, in part because of the fundamental bugs that can’t be fixed in WinPTY. The WinPTY version in Eclipse is also quite out of date, and hard to develop as it is interfaced to by JNI.

For Eclipse 2021-06 the Eclipse CDT will be releasing a preview version of the terminal that will use ConPTY. For interfacing to ConPTY we are using JNA which is much easier to develop because all the interfacing is in the Java code.

One of the open questions I had was whether there would be a performance issue because of the change to ConPTY. In particular, while JNA is slower for some things the ease of use of JNA normally far outweighs the performance hit. But I wanted to make sure our use case wasn’t a problem and that there wasn’t anything else getting in the way of the terminal’s performance.

Shell to Eclipse Terminal Performance

I have analyzed the performance of a process running in the shell writing to stdout as fast as possible to compare various different terminal options on my Windows machine. The Java program creates a byte[] of configurable size and writes that all to System.out.write() in one go, with some simple wall clock timing around it. See the SpeedTest attachment for the source.

I used 5 terminal programs to test the performance:

  • Windows Command – the classic terminal when you run cmd.exe for example
  • Eclipse with WinPTY
  • Eclipse with ConPTY
  • Windows Terminal (from the people who wrote conpty)
  • VSCode’s integrated terminal using ConPTY

And in each of them I ran the same Java program in 3 different shells:

  • cmd.exe
  • WSL2 running Ubuntu bash
  • git bash

Short summary is that WinPTY and Windows Command are much faster than the rest. ConPTY is quite a bit slower, whether used in Eclipse or Windows Terminal. VSCode is dramatically slower than the rest.

cmd.exeWSL2git bash
Windows Command8.33.54.2
Eclipse with WinPTY12.51.67.7
Eclipse with ConPTY1.81.72.0
Windows Terminal2.22.12.4
VSCode0.80.80.8
Full table of results based on a 10MiB write, reported in MiB/second, rounded to nearest 0.1 MiB/s:

As a comparison, on the same machine dual-booted into Xubuntu 18.04 I ran the following 5 terminals:

  • Eclipse – 23.1 MiB/s
  • VSCode – 3.0 MiB/s
  • xterm – 6.3 MiB/s
  • xfce4-terminal – 10.7 MiB/s
  • gnome-terminal – 10.2 MiB/s

The above shows that the raw speed of Eclipse Terminal is very good, it simply requires the best possible PTY layer to achieve the best speeds.

Eclipse Terminal to Shell Performance

I was going to run an Eclipse -> Shell test to make sure writes to the terminal hadn’t regressed. However the terminal has an artificial throttle in this path that limits performance to around 0.01 MiB/s, plenty fast to type, but much slower than a performant system could be. The code could probably be revisited because presumably the new ConPTY does not suffer from these buffering issues, and the throttling probably should not be there for non-Windows at all.

Conclusion

I am pleased that the performance of ConPTY with JNA is close to the new dedicated Microsoft Terminal and much faster than VSCode. Therefore I plan to focus my time on other areas of the terminal, like WSL integration and bug fixes with larger impacts. I am grateful to the community’s contributions and I will happily support/test/integrate any improvements, such as the upcoming Ctrl-Click functionality that was contributed by Fabrizio Iannetti and will be available in Eclipse IDE 2021-06.

Because much of the performance slowdown is because of ConPTY itself, which is actively being developed at Microsoft I hope that Eclipse will benefit from those performance improvements over time. There is no plan to remove the WinPTY implementation anytime soon, so if there is a user who feels impacted by the slowdown I encourage them to reach out to the community (cdt-dev mailing list, tweet me, comment on this bug or create a bug report).

Join us at EclipseCon France 2016

eclipsecon-728x90

Kichwa Coders are looking forward to EclipseCon France this June in lovely Toulouse.

We’ve teamed up with Christian Pontesegger to deliver the first ever EASE Python Scripting workshop. We’re looking forward to helping folks use scripting to really make the most out of their IDE. There are  also lots of other great workshops at the conference: Tracy Miranda, as part of the program committee,  has written a summary of them here.

Jonah Graham will be speaking at the conference on “CDT: Latest & Greatest Tooling for C/C++“. Join us to keep up to date about all things CDT.

And finally, we’ll be sticking around for the Unconference (on the 10th) with the Science Working Group. We’re currently making plans for a code sprint (or two). Everybody’s welcome to join!

Lean Design Critique: Function Fitting Tool

The DAWN Science function fitting tool was at that stage where all the functionality was in place but it needed some attention from a usability point of view. To drive the user design forward it was run through a lean design critique  – a quick way of running a design critique to get the most worthwhile  optimizations in a short space of time.The lean design critique is described below.

The original function fitting tool

1. Meet the requirements:

The most critical requirement is to have someone who understands who the different types of users are, what they use the tool for and how they use it. The design will be largely based on that information supplied so needs to be as accurate as possible. Then you need someone to apply some design principles. To keep it lean, ideally this will be 2, max 3 people (If it was just 1 person it would just happen, no need to formalise at all).

2. Identify the 90% task

This is the job or set of jobs that will be carried out again and again (so 90% of the time) in this user interface. In this case, while it seems easy to say that that task is ‘function fitting’, it is important to go further and break that down to the key steps. As important is to identify the tasks that are infrequent, in this case things like importing and exporting of fit date to files is done occasionally. A good question to ask for each UI element is ‘How often is that used?‘ – in this case it highlighted that while users can change the accuracy of the fitting routine, in practice they rarely do.

3. Do a Cognitive Walkthrough

In this step we breakdown the 90% task to its constituent steps and perform the tasks as a user would. Because we are keeping it lean we focus on the main steps and disregard the occasionally done steps (for example deleting or duplicating a function).

In this case a typical user scenario involves the following:

  1. Click ‘Auto ReFit’
  2. Select the desired function e.g. gaussian
  3. Set the parameters accordingly (mostly value)
  4. Look at plot to determine by eye if fit looks close enough
  5. If not, tweak parameters until fit is good
  6. When fit looks good, click update all to accept fit data
  7. Repeat tweaking until happy with fit, also measured by the goodness of fit value

This is the key step and worth taking the time to get as accurate data as possible.

4. Analyse Eye Travel, Mouse Travel & Clicks

Using the steps identified, apply these to the UI with a big red marker to get an idea of the path the user has to follow with their eye or mouse. You can also count clicks to see how many it takes to perform different tasks.
Analysing mouse travel to perform 90% task
Carrying out this step for this tool quickly highlighted the following areas:
– Step 5, repeatedly tweaking the parameters requires to and fro movement between two tables. It also highlights the same data (parameters) shown in two different tables which may not be obvious to the user that one is for editing and one for just viewing.
– Step 2 – an early step for the task is all the way down and in the middle of the UI – a non-obvious place for a user, particularly a new one to intuit what to do next.
– Step 6 – a key step is hidden in a toolbar alongside other anonymous buttons

In this case, mouse clicks were not used as a significant measure, particularly due to the iterative nature of tweaking the parameters.

5. Analyse Screen Space Usage

The principle here is that UI elements should be somewhat proportional to their importance such that they draw attention to themselves. So anything used for the 90% task needs to be big and obvious, while less important tasks should occupy less of ‘prime real estate’ screen space.

Running through this on the UI highlights to key areas that are prominent and well positioned but very infrequently used.

Optimizing screen real estate

6. Explore Existing Designs

Before you optimize, a useful thing to do is a field survey/research of existing UI designs that follow similar paradigms. A proven design which already has a lot of users can often be a good starting point.

In this case, inspiration was drawn from the Eclipse expressions view. It features a table with hierarchy (treeviewer) that allows adding and editing of items in the same place – ideal to replace the duplicate tables of the function fitting tool. It also has a nice obvious way of adding new elements to the table and a sash window to a text form for additional information.

Eclipse expressions view

7. Optimize

Based on the analysis feedback combined with UI research you can then go ahead and optimize the design to suit. In this step it is useful to use a usability heuristic checklist to cover common usability areas.
Here is the optimized design for the function fitting tool:
New design for function fitting tool

Efficiency of use – the two tables are combined into one table with hierarchy to allow creating and editing of functions all in one place.

Aesthetic and minimalist design – the non-essential UI elements are removed and put in preference pages accessed by the toolbar menu. This reduces the clutter and keeps the user focused on the main tasks at hand.

Visibility of system status – the value for normalised goodness of fit is given a more prominent position. Also the previously obscured update button is given more prominence and screen real estate.

8. Rerun Analysis

For good measure we run the analysis steps on the new design to ensure it fits the bill.
Analyse mouse travel

9. Conclusion

Further analysis and optimization can of course be done to ensure other tasks are not too overbearing for the user, but when design time is limited, doing a lean design critique is most efficient for a good working design optimized for the main user task.

Function Fitting Tool:Before & After

Continue reading “Lean Design Critique: Function Fitting Tool”