ThingMonk is an annual London conference that brings together the people building and shaping the Internet of Things. This year I spoke at the conference on ‘The Future of Developer Tools for IoT’. This talk looks at emerging and future trends in the developer tools space. Check out the slides and feedback from the audience, as well as reference links at the end. Plus thanks to Marcel Bruch & Codetrails for input on AI tools. Be sure to share your thoughts on how you see developer tools shaping up to scale for building the Internet of Things.
While looking to the future of debugger tooling, it is still important to consider the prior art and the solutions that have stood the test of time. For embedded development, gdb is high on that list, so it is worth considering if gdb’s interface could be the basis of a debug protocol.
If you’ve used gdb to debug C/C++ code then you are probably aware of MI, the machine interface layer used to communicate between the debugger backend and the IDE front end. MI is not only used by gdb but also adopted by lldb (the defacto debugger for Swift) and more recently by clrdbg (.NET Core). MI defines a rich set of functionality from standard debug run control and breakpoints up to advanced features for multi-process debug, reverse debugging and dynamic printf. With MI being pretty pervasive and supporting such rich functionality, it is tempting to think it might make the basis of a good debug protocol. However in practice it lacks some of the qualities of a good protocol:
1. A Specification
We once had the opportunity to work on a project where the brief was to integrate into Eclipse IDE/CDT a custom debugger that ‘implemented the MI spec’. We can tell you we learnt the hard way that MI has plenty of useful documentation but no spec to speak of. This matters when you get into the nitty gritty of implementation details for example: what syntax should be used to notify when a bad condition has been created on a breakpoint?
The documentation does not necessarily reflect what the code does, some command or command variants have inconsistencies with the source code or don’t reflect platform dependent issues. For example, the -exec-step-instruction in practice takes an argument (e.g -exec-step-instruction 1) even though this is not documented.
The main message here is documentation, even good documentation as in the case of gdb, is not the same as a protocol specification, so one can’t blindly implement to the docs (and if you think it’s just a case of looking at the code… well, which version?- see #4 below).
2. Clean Interfaces with no Idiosyncrasies
This piece of code from Visual Studio’s MIEngine demonstrates how rife MI is with idiosyncrasies. The code launches a debugger which will use MI to communicate i.e. to gdb, lldb or clrdbg. There are special cases for each tool that an IDE just shouldn’t need to know about:
- Different ways of specifying a working directory depending on the tool
- Environment variables are set differently: before launch for gdb/lldb after for clrdbg
- Details of which Operating System the debugger is being run on
And this is even before you launch MI. In Eclipse CDT just after launching MI, the IDE has to know about and issue commands about all sorts of things e.g. ‘set print sevenbit-strings on’ c’mon, really, seriously? Tom sums it up nicely:
“It is an oddity that currently an MI consumer must check gdb’s host charset in order to know how to decode its output.“
Once you get into actual debugging there’s a fair amount of ‘need-to-know’ for special cases & exceptions. A protocol needs to steer-clear of implementation details, but in the case of MI these have all too often leaked in.
3. Fit for Purpose
As MI was not specifically designed to be a protocol, not suprisingly there are a few behaviour specific things that make it not fit to be a protocol. For example:
- If your program prints to stdout, then that can corrupt the output stream of MI, breaking the instructions.
- In some cases GDB responds twice from a single command. In such cases, for example Eclipse CDT has a special MIAsyncErrorProcessor class just to manage such cases.
A good protocol has defined versions that clients and subscribers can adapt to. With each new version of GDB, MI has subtle differences that make client implementation long-winded and difficult to maintain. For example, in Eclipse CDT’s gdb debugger implementation (DSF) separate classes are created to manage differences in MI in different versions of gdb. There are 5 different breakpoint classes, 7 different run control classes, etc And this is just gdb versions, let alone lldb or clrdb – imagine trying to implement wide-scale support for all those in a new IDE!
While feature-rich and ubiquitous, gdb’s MI is a reasonable syntax, but not a good debug protocol. A good protocol needs much more than that – clean interfaces, fit for purpose, a spec & versioning – if it is really going to make common debugger implementations easier.
‘From then on, when anything went wrong with a computer, we said it had bugs in it.’ — Grace Hopper
As developers, we all know languages and frameworks are emerging and changing at breakneck speed. And the tools just can’t keep up. On top of that, there’s the move to tools in the cloud, which promise the ultimate in developer convenience. While this will be inevitable, the current tools still have a way to go in terms of functionality offered.
As a result, momentum is building around solutions that work for multi-language support in multiple environments. The language server protocol (LSP) has emerged as the chosen way for various IDEs and editors to keep pace with all the different language changes. For new cloud IDEs like Eclipse Che, it is a vital part of the roadmap, providing an effective way to deal with the sheer scale of the problem. The LSP solves this problem by having a server for each language, with a common protocol that all front ends can use to communicate with it. One of the key ideas of the LSP is that the IDE knows as little as it can, delegating down to the language server to do the specifics.
Even for established IDEs such as Eclipse, LSP has a lot to offer. When Eclipse came on the scene more than 15 years ago it was a massive step forward when it came to sharing common UI parts between multiple languages. But that framework is far too cumbersome for today’s rate of change. The LSP takes things to the next level and makes the split between UI and backend even more absolute. Lots of progress has been made on the LSP4E & LSP4J projects.
LSP is not just for Eclipse projects. Originating from Microsoft, it also forms the basis for language support in VS Code. Just last month there were 27 protocol implementations, today there are 35 and counting. Truly impressive growth.
With language support taken care of, it’s time to ask the question about debugging. Again, when Eclipse first emerged it offered a state-of-the-art debugger framework. A common debug interface that separated UI and backend and allowed for implementations in multiple languages: Java, C/C++, Python, etc. But with little investment in recent years it is difficult to see this as being up to the job of being a general framework for the next generation of tools.
Similarly to LSP, Microsoft provides a Debug Protocol with a number of adapters written, many already being used in VS Code. So far there has not yet been a rush to adopt it in the same way as LSP, but is this the next logical step for tool developers? Especially for nascent platforms such as Eclipse Che, where the debug support is at minimal viable demo levels, and will likely rise up the priority list soon.
Yet the whole area of debugging is pretty substantial. In order to provide rich debug features, tools need to look at supporting a whole range of functionality, for example:
- Processes, threads, etc
- Stack Traces
- Run Control (step, continue, run to, etc)
- Breakpoints, watchpoints
- Source Code Lookup
- I/O, Console support
- Expressions, etc
So all of those features. For every language we use please. In every IDE we might want to use, thanks. Oh, and don’t forget to make it all work asynchronously.
But hey, do developers still even use richly-featured debuggers? Is it even feasible to have a single protocol or framework cover these for a wide range of languages? The answers to all these may not be so clear at the moment. However as likely as there are still bugs to be found in software, we will still need good quality debuggers and a solution to make them quickly available for multiple languages in multiple tools.
The concept of marginal gains, made famous by Team Sky, has revolutionized some sports. The principle is that if you make 1% improvements in a number of areas, in the long run the cumulative gains will be hugely significant. And in that vein, a 1% decline here-and-there will lead to significant problems further down the line.
So how could we apply that principle to the user experience (UX) of Eclipse C/C++ Development (CDT) tools? What would happen if we continuously improved lots of small things in Eclipse CDT? Such as the build console speed? Or a really annoying message in the debugger source window? It is still too soon to analyse the impact of these changes but we believe even the smallest positive change will be worth it. Plus it is a great way to get new folks involved with the project. Here’s a guest post from Pierre Sachot, a computer science student at IUT Blagnac who is currently doing open-source work experience with Kichwa Coders. Pierre has written an experience report on fixing his very first CDT UX issue.
This week I worked with Yannick on fixing the CDT CSourceNotFoundEditor problem – the unwanted error message that Eclipse CDT shows when users are running the debugger and jumping into a function which is in another project file. When Eclipse CDT users were running the debugger on the C Project, a window was opening on screen. This window was both alarming in appearance and obtrusive. In addition, the message itself was unclear. For example, it could display “No source available for 0x02547”, which is irrelevent to the user because he/she does not have an access to this memory address. Several users had complained about it and expressed a desire to disable the window (see: stack overflow: “Eclipse often opens editors for hex numbers (addresses?) then fails to load anything”). In this post I will show you how we replaced CSourceUserNot FoundEditor with a better user experience display.
This is a guest post from Yannick Mayeur, a computer science student at IUT Blagnac who is currently doing open-source work experience with Kichwa Coders. It was originally one of his weekly write-ups which can be found here.
You are probaly familiar with the pull request system of GitHub that programmers use to contribute to an open-source project. Gerrit (named after its designer Gerrit Rietveld) is basically an improved version of this system. Gerrit allows the committer to give more precise feedback on each line of code edited, and allows other members of the team to review those changes. Gerrit is used by the Eclipse CDT community. In this blog post I will show you how to efficiently get started with it.
The required tools & knowledge
Having Git is basically all you need to clone the sources, and push them. If you want to edit them in a good environment use the Eclipse JAVA IDE. Knowing the basics of Git is also required, though I think you could pick up Git as you go along with a bit of trial and error.
How to get the sources of CDT
Cloning the sources to your computer is an easy but essential task.
The link of the repository is: git://git.eclipse.org/gitroot/cdt/org.eclipse.cdt
To clone use the following command:
git clone git://git.eclipse.org/gitroot/cdt/org.eclipse.cdt
Once you have the files, go to Bugzilla and find a bug you want to fix.
Now comes the tricky part. In order for you to be able to push your change a few things have to be respected. Continue reading “Getting Started With Gerrit on Eclipse CDT”