The Open Source Advent Calendar

oscalendar

It’s that time of year again. But I don’t really like chocolate. And there’s only so many times you can do a Lego advent calendar. However, I do really love open source software. So for a change, it’s time to spread some open source software magic.

The Open Source Advent calendar works like a normal advent calendar, but instead of the usual countdown, in this case each day has a specific ‘act of open source goodness’. Whether you are an existing open source committer, a contributor or even a lurker looking to get involved, the open source advent calendar will get you right into the spirit of things. It’s like the Kindness Calendar but for open source software.

We’ll be working as a team at Kichwa Coders to step through the calendar, focusing on the projects we love.  We’re using the hashtag #OpenSourceAdventCalendar to share our acts of open source. We invite you to join us, create an open source ripple effect and make a difference to open source projects everywhere this December!

Download the Open Source Advent Calendar here.

Pushing the Limits of Xtext for C/C++ Linker Scripts

simplemem

Xtext is the popular Eclipse language development framework for domain specific languages. Its sweet spot is JVM-languages and it is excellent for languages where you can define the grammar yourself. But how well can Xtext cope with a non-JVM language that has undergone decades of evolution?

In our case, we want to see if we can take advantage of Xtext to create an editor for C/C++ linker scripts in CDT. Linker scripts are used to specify the memory sections, layouts and how code relates to these sections. Linker scripts consist of the ld command language, and this is what a simple typical script might look like:

MEMORY {
 RAM : ORIGIN = 0x0, LENGTH = 0x2000
 ROM : ORIGIN = 0x80000, LENGTH = 0x10000
}

SECTIONS {
 .text : { *(.text) *(.text.*) } > ROM
 .rodata : { *(.rodata) *(.rodata.*) } > ROM
 .data : { *(.data) *(.data.*) } > RAM
 .bss : { _bss = .; *(.bss) *(.bss.*) *(COMMON) _ebss = .; _end = .; } > RAM
}

Alternatives to Xtext

Besides using Xtext, its worth considering some of the other options there are for this task:

  • Roll-your-own – the existing C/C++ Editor in CDT does this, gives full control, best error-recovery and supports bidirectionality, recreating source from abstract syntax tree (AST), but it is a last resort as it would be an incredible amount of work that would take a long time to get right.
  • Antlr – write your own antlr grammar, but since antlr is already used in Xtext, may as well use Xtext and get benefits of Eclipse editor integration
  • Reuse linker’s bison grammar – would give perfect parsing, but it is a no-go because i) it’s GPL ii) it generates C code not Java & iii) requirements for editing are much more strenuous than for linking and this for example, would not support bidirectionality (i.e you can’t recreate the linker file from the AST).

Benefits of Xtext

The Xtext framework additionally provides these nice features we are interested in:

  • Parsing, lexing & AST generation
    • serialisation support is particularly important to support bidirectionality and preserve users comments, whitespace etc.
  • Rich Editor Features
    • syntax highlighting
    • content assist
    • validation & error markers
    • code folding & bracket matching
  • Integrated Outline editor
  • Ecore model generation which can be used for integration with UI frameworks such as EMF Forms, Sirius, etc.

Linker Script Parsing Challenges

When we talk about the ld command language being a non-JVM language, here are some specific challenges related to what that means.

  1. Crazy Identifiers! The following are valid identifiers in linker scripts:
    • .text
    • *
    • hello*.o
    • “spaces are ok, just quote the identifier”
    • this+is-another*crazy[example]
  2. Identifier or Number? Things that appear to be identifiers may actually be numbers:
    • a123 – identifier
    • a123x – number
    • 123y – identifier
    • 123h -number
  3. Identifier or Expression?

In the grammar 2+3, for example, depending on context, can either be an identifier or an expression:

SECTIONS {
 .out_name : {
  file*.o(.text.*)
  2+3(*)
  symbol = 2+3;
 }
}

The first 2+3 is a filename, so almost anything that can be a filename is allowed there. The second 2+3 is an expression to be assigned to symbol.

Resolutions

Here’s what we did to support the linker language as far as we could:

  1. Custom Xtext grammar – as extending the XType grammar does not make sense, the main job is to craft the grammar to understand all the linker script identifier and expressions specifics. This involves iterating as we add in more and more language feature support, here’s the work in progress.
  2. Limited Identifier Support – in some cases we opted to not support certain identifiers unless they are escaped (double-quoted). While linker scripts theoretically support such identifiers (e.g. 1234abcd) we have not found a single case yet of an identifier that would actually need escaping. If one did crop up, the user could adjust it to work with the editor (e.g. “1234abcd”).
  3. Context Based Lexing – knowing the difference between an identifier or expression would require context based lexing rules. However this will not work with the antlr lexer. We have the option to replace it with a custom or external lexer. This is an option that can be considered in the future if desirable.

Conclusion

Xtext is a great language development framework. While Xtext may not be able to support every theoretical case of the long-lived linker script command language, it can be used to provide a very high level of support for the common features. Support for context based lexing in the future would enable a higher level of language support. Xtext can be used to provide a rich language editor with syntax colouring, command completion, integrated outline view & more in a relatively short space of time. A powerful linker script editor is another great feature for C/C++ developers that use CDT, the reference C/C++ IDE in the industry.

The Sound of the Universe @ EclipseCon Europe 2016

At EclipseCon this year I heard the sound of the universe. And it was awesome and breathtaking. To be precise, it was the sound of two black holes colliding over a billion years ago,  part of the enthralling final keynote from Dr Benno Wilke on detecting gravitational waves. It was a fitting way to end a conference that had kicked off with another amazing keynote:  Stephen Carver delivering a powerful and emotional story of the people and tech behind the space shuttle disasters, framed in profound lessons on real communication and avoiding silo thinking.

For the very first time at EclipseCon Europe we held a CDT summit. Over 10 years ago I had the honour of being the first developer from Europe involved in CDT, so to bring the summit to Europe was a particularly special moment for me, especially with our renowned project co-lead Doug Schaefer in attendance. The summit was a success, particularly welcoming contributors from the wider community into the fold, and will definitely something we will be doing again next year.

As this year’s focus there was also a big community focus on diversity and raising awareness on this topic. The activity included my talk on ‘7 Habits of Highly Diverse Communities‘, addressing the board on the topic and a diversity BOF session. The discussions were great, lots of good energy, practical suggestions and I am so proud to see the community work together to ensure we can be as open and inclusive as possible.

The Science Working Group had good reason to celebrate at the conference: we have just completed our very first simultaneous release of five projects. A significant milestone for this nascent group, and was terrific to talk about the projects to the rest of the community.

There was an incredible amount on at the conference this year, the best way to get a quick taste was hearing what people enjoyed: language servers, Xtext, Sirius, scripting, IoT & testing were topics that kept coming up. On a personal level, it was my most intense EclipseCon yet with three talks, a BOF and a summit to organize. On the whole it was the busiest conference yet with a record attendance of 619. The most important thing is always the people: lots of new and old friends to talk to and exchange energy. At EclipseCon this year I heard the sound of the Eclipse universe. And it was awesome and breathtaking.

(A version of this article was first published on jaxenter.com: https://jaxenter.com/eclipsecon-europe-at-a-glance-129883.html)

Why You Should Send Flowers To Open Source Developers

IMG_1182

You are probably thinking flowers are dumb: they die and they don’t do anything; they are for dates, weddings and funerals. Besides, open source developers do not need flowers, they need money. But nobody wants to pay for software, let alone open source software. Even though today open source software is as vital as roads and bridges, nobody wants to pay to maintain it.

So by all means let’s figure out a way to get more money for open source developers. But in the meantime, here’s why you should still send flowers.

  1. Flowers show appreciation

Taskwarrior share some great lessons learnt for open source maintainers. This includes cautionary tales like: “People will pick a fight with you about all your incidental choices. Your issue tracker, your branching strategy, your version numbers, the text editor you use, and so on.” Open source developers are a tough bunch, but are still likely to suffer from burnout. Sending flowers is the ultimate gesture of positivity and understanding. Plus there is all sorts of research out there that suggests flowers & plants improve mood and promote creativity in the workplace.

  1. People think you are more capable & emotionally intelligent

If altruistic reasons are not enough, then consider this. The act of giving flowers has benefits for the giver. The act of giving makes people happy. There is also research that says people who send flower are perceived as having higher emotional intelligence, capable and courageous.

  1. You will be a better leader

Small things can make all the difference. This idea is partly inspired by my favourite blogger, who wrote about the effect receiving flowers from Marc Benioff, CEO of Salesforce had on her. Basically really good leaders are always thinking of thoughtful ways to get the best out of everybody.

OK so now I have a confession to make. Maybe I am telling you this because Kichwa Coders have recently joined the realm of open source maintainers, heading up and contributing to the January project  (numpy & datastructures for Java). And we are gearing up, together with the rest of the Eclipse Science Working Group, for our first major science release. It is an amazing project that will provide tool infrastructure for countless scientific projects. So it will be awesome, but there is still some trepidation when you cross the threshold from open source user to open source maintainer. It is a little bit like how when you have your own kids, you finally really appreciate your parents and wonder how they did it. Right now I am in awe of open source leads and maintainers everywhere. So just do it. Send flowers to open source developers. And while you’re at it, send some to your parents too.

Is Eclipse Che Really The Next Generation Eclipse IDE?

jet-437741_1920

Ever since the keynote at EclipseCon, Eclipse Che has taken center-stage, and is being touted as ‘Eclipse Che, the next-generation Eclipse IDE’.  This has, to say the least, annoyed some in the Eclipse community with the piggy-backing on the established Eclipse brand.

You could liken it a bit to how we have London Heathrow Airport: major, international, well-known and serves millions of people. Then along came London Luton Airport, a newer airport that anyone living in the UK knows very well is really not in London. However, some foreigners may not be so clear on that distinction and when they land might feel perplexed as they figure out how to make it to Buckingham Palace… But hey, London Luton is modern, has shorter queues and is investing a ton in transport links, new terminals, supporting new destinations etc. Meanwhile at Heathrow even getting a third runway built seems like an interminable venture. So who’s to say where things will stand in the future, but we can be sure there will always be someone pointing out that Luton is not in London…

OK, so airports aren’t IDEs and this analogy quickly gets to the end of its runway (badum-tish!) but the point is people are travelling to London rather than going elsewhere. So for the Eclipse community, isn’t it great to have renewed interest and a new avenue of investment that will bring new users and contributors into the community?

So like it or not, Eclipse Che, the next generation Eclipse IDE is here to stay, and our job is to help our community navigate the changes. When my clients ask about Che (which they do, often referring to it as ‘Eclipse Cloud’ or ‘Cloud IDE’), here are three things I tell them:

  1. Eclipse Che is the open source project with the most compelling cloud-IDE story yet. It has a major focus on ease-of-use by use of shareable workspaces that wrap up projects & all their dependencies including runtimes. As such it is a different beast to Eclipse IDE, with no established migration path (yet?).
  2. Che is on something like a 10+ year arc towards maturity, so all features mentioned are likely in a preliminary state and far from the rich support available in Eclipse IDE or RCP-based products today.
  3. There is major investment into Che from companies looking to build their cloud platforms. Adopters so far include Microsoft, SAP, IBM and Redhat. Also recently Samsung adopted it for its IoT IDE Artik.

Next generation Eclipse IDE or not, time will tell, but the reality is that it doesn’t really matter. There is no doubt for us that this is a good thing for the Eclipse community as a whole. Certainly for the Eclipse I know that is synonymous with commercially-friendly, innovative-tech & open-source goodness.