Creating a new stack in Eclipse Che from an existing Docker image

title_che_custom_whalesayIn Eclipse Che, stacks are runtime configurations for workspaces.  Eclipse Che 5.0 provides the ability to quickly create a new stack based on an existing Docker image. There are a few requirements, with the main one being that the image must include a bash shell (fortunately most already do). Additionally, Docker images used with Che must have a non-terminating CMD command so that the container doesn’t shut down immediately after loading. However, we can do this in Che without having to modify the Docker image.

This article outlines how you can create a new stack based on the docker/whalesay image, the Docker image many folks would have come across when going through the Docker tutorial.

Step 1: Create a new Runtime Stack

  1. Install and launch Eclipse Che. (This article is based on 5.3.1)
  2. In the left-hand menu, click on ‘Stacks’ then ‘Add Stack’stacks1
  3. In the new stack page, fill in the ‘Name’ field e.g. Eclipse Che Whalesay stacks2
  4. For good measure, in the Tags section, delete the Java 1.8 tag and you can add in any preferred tags (press Enter after each tag to turn it blue).

From now your stack is available on the dashboard or stack list, but it doesn’t yet do anything impressive.

Step 2: Reference the Docker Image

  1. To reference the Docker image, in the stack editor, first scroll down to the ‘Raw Configuration’ section and click ‘Show’. Find where it says ‘image’ and replace the existing entry with the docker/whalesay one. Click ‘Save’.stacks3
  2. As the whalesay image normally shuts down after running, we need to make it so it keeps running. We can do this using Docker Compose syntax that is now supported in the stack editor. To do this we add the following command to the content.
    command: [tail, -f, /dev/null]

    stacks5.png
    Whitespace matters so check the recipe preview in the ‘Runtimes’ section to ensure it looks like correctly formatted Docker Compose syntax.  Although this is a bit hacky, it is great because it means we can reuse the docker image completely unchanged. stacks7.png

Step 3: Add a Custom Command

Next we will create a command to allow us easily run the whalesay image within the browser IDE.

  1. In the Commands section, click Add. Fill in the name and command fields. In this case we’ll simply make the command just say ‘boo’.
    • Name: cowsay
    • Command: cowsay boostacks8

Step 4: Test the Stack

  1. That’s it, now we can test it straightaway by clicking on the ‘Test’ button. A dialog will pop-up, click ‘Test Workspace’, no need to import any projects.
  2. This should pull the whalesay image and launch up a workspace with it configured.stacks9.png
  3. Click on the play icon next to the cowsay command and you should whalesay in the terminal saying boo!stacks10.png

Bonus Step: Use Macros

  1. We can also use macros in commands, try editing the command to have the whale say the name of the selected file like this:
    • Name: cowsay
    • Command: cowsay ${explorer.current.file.name}

    A full list of macros can be found here: https://www.eclipse.org/che/docs/ide/commands/#macros

  2. Then test the workspace again, this time including one or two example projects. Click on a file, then run the command. This time whalesay should say the name of any selected files. stacks11

And that’s it, an unchanged Docker image easily integrated into a new runtime stack!

whalesaystack

Getting Started With Eclipse Che on Windows 10

chewindows

Over the last year or so there has been a reasonable amount of evolution of the best way to get Eclipse Che running on Windows. The latest set of instructions are reasonably straightforward though not a complete no-brainer. This post records my snapshot of the step-by-step install process for Eclipse Che 5.2.2 including Docker install, dead ends & false starts.

  1. The aim is to set up a local install of pre-built Che on Windows 10 Home Edition. Starting point, scan documentation at https://www.eclipse.org/che/docs/setup/getting-started/
  2. Install prerequisite Docker. Download & install ‘Docker for Windows’. Helpful error message redirects me to use Docker Toolbox. dockerforwindows
  3. Download Docker Toolbox for Windows. Run installer. Requires 77.6M, and also wants to install Git.  The reason it needs to install a full source control system is merely down to the fact that the Git install comes with a nicely packaged bash & terminal implementation. Nevertheless I already have git installed so no need to here.
  4. Install completes, use shortcut ‘Docker Quickstart Terminal’ doesn’t work as it can’t find bash.exe (The shortcut expects Git to be in a certain default location).
  5. Update shortcut to point at my installed version of Git’s bash.exe (after another false start trying to use Git’s bash directly, fiddling with running things as Admin, etc).
  6. Success & I can run docker run hello-worldhelloworlddocker
  7. Now to the actual Eclipse Che part, try default command line
    docker run -it eclipse/che start

    Not so hasty there, a helpful error message tells me I’m missing a mandatory parameter.cherun1

  8. Add in missing parameter, try again
    docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock \
        eclipse/che start

    Get another useful message about needing to mount a directory to save data:cherun2

  9.  Create a temporary directory and rerun the command specifying a data directory. Get a warning about data folder needing to be in %userprofile% (now you tell me). cherun3
  10. Make a different directory and run command again
    docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock \
        -v /c/Users/tracy/tmp:/data eclipse/che start

    Success!cherun4

  11. Launch up browser, point at http://192.168.99.100:8080, Eclipse Che is up and running.chewindows

Diversity means Open Source for a New Generation

img_20160511_183256

Last month the Eclipse Foundation announced a new Diversity & Inclusion Champion, Thea Aldrich. I did cartwheels in my head when I heard the news. Why? Because I can’t do cartwheels in real life. But seriously, this is really a BIG DEAL.

When it comes to open source communities like Eclipse, you don’t need big data to know there is a diversity problem, you just have to show up to an EclipseCon and look around.

While this has been the case for years, recently there has been a change. Maybe it’s down to more consciousness of the issues or maybe more maturity in the community, but people have noticed and want to do something about it.  About a year ago, a grassroots effort led by Alex Schladebeck was started with the renewed goal of improving diversity in the community and at conferences. I was honoured to be part of the small but effective team. One big goal of ours was simply to raise awareness and start an open conversation about issues surrounding diversity. We did just that, through:

  • Writing: blog post, after blog post, after blog post
  • Speaking: I do a diversity talk ‘7 Habits of Highly Diverse Communities’, which has been well received and requested by other open source communities.
  • Online Discussions: we have a dedicated Mattermost channel at Eclipse for the topic, plus several web conferences.
  • In Person Discussions: we have had Diversity BOFs at EclipseCons, as well as addressed the Eclipse Board of Directors & Members’ meeting.

And we’ve been learning a lot along the way, such as when we tried and failed to secure a woman keynote speaker for one of the conferences. However,the best thing about all this has been how the rest of the community has responded. People have shown up, got involved, asked questions, challenged things (I expect nothing less of developers!) and offered support.

Throughout this, and even from the beginning, we have always wondered how we can sustain these efforts and indeed how we can expand them to do more. There is no quick fix for promoting diversity, more just a continuous and determined set of steps in the right direction. And in reality it would never work long term without someone dedicated to spearheading the changes.

So that is why, in less than a year after we implored the Foundation to make this happen, they listened, took us seriously and did it! Not just that, the Eclipse Foundation now becomes an Open Source Foundation investing in change, with a dedicated role to diversity. We can hope this becomes a must-have role for every open source foundation out there.

I’ve always said how much I love this community because of the ability to adapt to changing environments, and here is more proof. Thea has already kicked off efforts on multiple fronts:

  • Identifying ways in which all Eclipse events are inclusive events and welcoming,
  • Rolling out an ambassador program to involve the community in welcoming newcomers into the ecosystem,
  • Reaching out to established projects to see how we can support their efforts,
  • Making all Eclipse Foundation websites and resources easier to navigate for native non-English speakers,
  • Providing the community with a direct path to Foundation staff for ideas, complaints, feedback and other issues that our community or members may encounter.

We look forward to working with Thea, the Foundation and community on all these aspects to keep making improvements and bring about a real change.I will continue to do what I can, which includes running for a seat on the Board of Directors. Having this new focus and investment in diversity means we can look forward to bringing open source to a whole new generation of developers.

The Open Source Advent Calendar

oscalendar

It’s that time of year again. But I don’t really like chocolate. And there’s only so many times you can do a Lego advent calendar. However, I do really love open source software. So for a change, it’s time to spread some open source software magic.

The Open Source Advent calendar works like a normal advent calendar, but instead of the usual countdown, in this case each day has a specific ‘act of open source goodness’. Whether you are an existing open source committer, a contributor or even a lurker looking to get involved, the open source advent calendar will get you right into the spirit of things. It’s like the Kindness Calendar but for open source software.

We’ll be working as a team at Kichwa Coders to step through the calendar, focusing on the projects we love.  We’re using the hashtag #OpenSourceAdventCalendar to share our acts of open source. We invite you to join us, create an open source ripple effect and make a difference to open source projects everywhere this December!

Download the Open Source Advent Calendar here.

Pushing the Limits of Xtext for C/C++ Linker Scripts

simplemem

Xtext is the popular Eclipse language development framework for domain specific languages. Its sweet spot is JVM-languages and it is excellent for languages where you can define the grammar yourself. But how well can Xtext cope with a non-JVM language that has undergone decades of evolution?

In our case, we want to see if we can take advantage of Xtext to create an editor for C/C++ linker scripts in CDT. Linker scripts are used to specify the memory sections, layouts and how code relates to these sections. Linker scripts consist of the ld command language, and this is what a simple typical script might look like:

MEMORY {
 RAM : ORIGIN = 0x0, LENGTH = 0x2000
 ROM : ORIGIN = 0x80000, LENGTH = 0x10000
}

SECTIONS {
 .text : { *(.text) *(.text.*) } > ROM
 .rodata : { *(.rodata) *(.rodata.*) } > ROM
 .data : { *(.data) *(.data.*) } > RAM
 .bss : { _bss = .; *(.bss) *(.bss.*) *(COMMON) _ebss = .; _end = .; } > RAM
}

Alternatives to Xtext

Besides using Xtext, its worth considering some of the other options there are for this task:

  • Roll-your-own – the existing C/C++ Editor in CDT does this, gives full control, best error-recovery and supports bidirectionality, recreating source from abstract syntax tree (AST), but it is a last resort as it would be an incredible amount of work that would take a long time to get right.
  • Antlr – write your own antlr grammar, but since antlr is already used in Xtext, may as well use Xtext and get benefits of Eclipse editor integration
  • Reuse linker’s bison grammar – would give perfect parsing, but it is a no-go because i) it’s GPL ii) it generates C code not Java & iii) requirements for editing are much more strenuous than for linking and this for example, would not support bidirectionality (i.e you can’t recreate the linker file from the AST).

Benefits of Xtext

The Xtext framework additionally provides these nice features we are interested in:

  • Parsing, lexing & AST generation
    • serialisation support is particularly important to support bidirectionality and preserve users comments, whitespace etc.
  • Rich Editor Features
    • syntax highlighting
    • content assist
    • validation & error markers
    • code folding & bracket matching
  • Integrated Outline editor
  • Ecore model generation which can be used for integration with UI frameworks such as EMF Forms, Sirius, etc.

Linker Script Parsing Challenges

When we talk about the ld command language being a non-JVM language, here are some specific challenges related to what that means.

  1. Crazy Identifiers! The following are valid identifiers in linker scripts:
    • .text
    • *
    • hello*.o
    • “spaces are ok, just quote the identifier”
    • this+is-another*crazy[example]
  2. Identifier or Number? Things that appear to be identifiers may actually be numbers:
    • a123 – identifier
    • a123x – number
    • 123y – identifier
    • 123h -number
  3. Identifier or Expression?

In the grammar 2+3, for example, depending on context, can either be an identifier or an expression:

SECTIONS {
 .out_name : {
  file*.o(.text.*)
  2+3(*)
  symbol = 2+3;
 }
}

The first 2+3 is a filename, so almost anything that can be a filename is allowed there. The second 2+3 is an expression to be assigned to symbol.

Resolutions

Here’s what we did to support the linker language as far as we could:

  1. Custom Xtext grammar – as extending the XType grammar does not make sense, the main job is to craft the grammar to understand all the linker script identifier and expressions specifics. This involves iterating as we add in more and more language feature support, here’s the work in progress.
  2. Limited Identifier Support – in some cases we opted to not support certain identifiers unless they are escaped (double-quoted). While linker scripts theoretically support such identifiers (e.g. 1234abcd) we have not found a single case yet of an identifier that would actually need escaping. If one did crop up, the user could adjust it to work with the editor (e.g. “1234abcd”).
  3. Context Based Lexing – knowing the difference between an identifier or expression would require context based lexing rules. However this will not work with the antlr lexer. We have the option to replace it with a custom or external lexer. This is an option that can be considered in the future if desirable.

Conclusion

Xtext is a great language development framework. While Xtext may not be able to support every theoretical case of the long-lived linker script command language, it can be used to provide a very high level of support for the common features. Support for context based lexing in the future would enable a higher level of language support. Xtext can be used to provide a rich language editor with syntax colouring, command completion, integrated outline view & more in a relatively short space of time. A powerful linker script editor is another great feature for C/C++ developers that use CDT, the reference C/C++ IDE in the industry.