CDT has been reformatted!

Eclipse CDT has just had all its source code reformatted. That means I just pushed a massive set of commits with a huge git shortstat:

$ git diff --shortstat  3cf0297769..HEAD
 10806 files changed, 697885 insertions(+), 593225 deletions(-)

That also means that any commit/patch you have on CDT before the reformat will almost certainly not apply cleanly.

Therefore I have created this guide for you to reformat your commit and make it easier for you to submit your updated patch to gerrit.

To be able to rebase your commit onto the current master you need to create a version of it with the new formatting. At a high-level the steps we are going to do are:

  1. Reformat your commit with CDT’s new coding style
  2. Create a commit against the pre-formatted CDT that has only the files your commit changed formatted – but without your other changes

The diff between 1 and 2 above is an updated version of the diff for your real change.

Step-by-step this breaks down as:

Pre-requisites

  • Modern bash shell with sed, awk, grep, git, python, etc
    • I am using Ubuntu 18.04 LTS and python 2.7
  • Eclipse EASE with Py4J Python language support
  • Eclipse workspace setup for CDT development, follow the standard setup on the wiki and perform these extra steps:
  • Copy releng/scripts/cleanup.py somewhere outside of your gitroot so you can access it as the branches change later
  • Set ECLIPSE environment variable to path to eclipse on your machine to make some of the commands below easier to copy/paste

Step 1: Checkout the commit to rebase

First step is to checkout in CDT repo the commit you want to rebase onto current master. For example if you are updating a commit in gerrit, you could press Download in the top right and copy/paste the Checkout command:

download

Step 2: Reformat commit to new style

Start by creating a new branch to work on:

git checkout -b commit_to_rebase
Then rebase your change onto commit 35996a5c5ca, where all the per-project formatter settings have been applied, but the code has not been formatted yet. If your original commit is not too far behind this will finish without conflicts. If you have any conflicts, resolve them now.
git rebase 35996a5c5ca5c254959ba48241eaada6dbf8628d

Reformat the code to current standard using the following steps:

Step 2a: Close open editors in Eclipse

The EASE script will run on all open editors, so make sure to start with no open editors.

Step 2b: Open all the Java files modified in your commit

Using git’s diff-tree command we can get all the Java files as a list and then open them with Eclipse.
git diff-tree --no-commit-id --name-only -r commit_to_rebase -- *.java | xargs $ECLIPSE

Step 2c: Run cleanup.py

With Eclipse EASE run cleanup.py:

  1. Create a new Run Launch Configuration of type EASE Script
  2. In Script Source browse the filesystem to where you saved cleanup.py earlier
  3. Ensure Execution Engine is set to Python (Py4J)
  4. Press Run

Each open editor will have the clean up actions run, and then be closed.

Step 2d: Remove trailing whitespace in files

CDT’s code standards require trailing whitespace to be removed from all files marked with remove trailing whitespace in .gitattributes. To do this, run this bit of shell:

git show master:.gitattributes | awk '/# remove trailing whitespace/{getline; print $1}' |
    while read i ; do
        echo "Removing trailing whitespace on $i files"
        git diff-tree --no-commit-id --name-only -r commit_to_rebase -- "$i" | xargs --no-run-if-empty sed -i 's/[ \t]*$//'
    done

Step 2e: Save/commit cleanup changes

Update the change to now contain the formatted files.
git add -u
git commit --amend --reuse-message=HEAD

Step 3: Create a base commit to compare against

In this step create a commit that has only the files modified in the original commit cleaned up, but no other ones. This will be the base for the diff we create later.

Step 3a: Create a new branch

Start by checking out to a new branch the same commit as above with the formatter settings

git checkout 35996a5c5ca5c254959ba48241eaada6dbf8628d -b commit_to_format

Step 3b: Cleanup the modified files

Repeat Step 2a to Step 2d above to cleanup the files.

Step 3c: Save/commit cleanup changes

Save the files which are now formatted, but without the change you wish to rebase in a commit
git add -u
git commit -m"formatted files"

Step 4: Create a new commit that can be cherry-picked to master

Apply the change so you have a history on commit_to_format branch that is two ahead of 35996a5c5c, the first being the formatted files created in step 3, the second being the change we are trying to get onto master, which will be created now

Step 4a: Diff and apply the change

Diff the two branches we just made, that diff is the real work you are trying to get on master, and apply that diff
git diff commit_to_format..commit_to_rebase | git apply

Step 4b: Commit the change

Save the edit, reusing your original commit message (and therefore Change-Id!)
git add -u
git commit --reuse-message=commit_to_rebase

Step 5: Cherry-pick the new commit onto master

You now have a commit, marked with commit_to_format, that can be cherry-picked onto master. If other commits have made a conflicting edit since the reformat you may have some conflicts to resolve when you do this cherry-pick
git checkout master
git cherry-pick commit_to_format

Step 6: Push commit to gerrit

Push your commit now that it is ready, if you are using gerrit you can do:

git push origin master:refs/for/master

Step 7: Clean up your branches

Once you are done make sure to remove the branches you don’t need anymore.

git branch -D commit_to_rebase
git branch -D commit_to_format

All done!

And that is it. Hopefully you have now recreated your patch with just a few automated steps, instead of a painful manual copy+paste job. These instructions are also in the CDT repo in releng/scripts/rebase_helper.sh.

This set of steps shows that there are still some more automatable things that could be done. In particular being able to run Code Clean Ups from the command line in Eclipse would be really nice. Eclipse’s Java code formatter can be run from the command line.

And finally, join in on the cdt-dev mailing list!

Women speakers pass the tipping point at Monkigras, and why this matters for tech.

pia_cropped

At Monkigras last week there were more women speaking than men. It rocked my world. The ever-inspiring Dr. Lucy Rogers was the only familiar face. Dormain Drewitz, Pia Mancini, Mazz Mosley and Joni Saylor were revelations in the curated line-up. Then there was the unscheduled Linda Peng who seized the opportunity to promote codebuddies. And that’s just the half of it.

My normal status quo these days is to be all-too keenly aware of the gender of tech speakers. Probably because I am constantly being reminded I am a women-in-tech whether I want to be or not. But somehow with a dozen or so women speaking at Monkigras that simply didn’t happen; somehow I forgot to tune into the gender of the speakers at the time.

In hindsight it was a great snapshot of terrific women working in tech, women who are so confident in what they do and comfortable sharing this knowledge with the audience. For one thing Jesslyn Rose is a great speaker and gave us a great visual language for understanding burnout.

I have never done pager-duty in my life but Charity Majors still had me hanging on her every word as she laid out tech leadership as it should be, driving points home in ways no-one could forget in a hurry.

I noticed Majors’ talk also featured major ‘kerb-appeal’ for my daughter: blue & purple hair, unicorn stickers and name-dropping ‘Rainbow Dash‘.

Incidentally, the talk that would probably appeal most to my son would be Catherine Dixon and her magical ability to make you care about the size of the letter ‘O’ and see how that makes lettering come alive and seem balanced.

And while we are picking favourites, mine was definitely Mandy Whaley’s talk which transported me to the life and times of Marie Curie and was neatly tied in with open source & crowdfunding campaigns. It was not lost on me that it took the tragic death of Pierre Curie to propel Marie out of the shadows and onto the mainstage. Fortunately we’ve come a long way since then. I think.

And what I discovered while listening to the talks, not tuned into gender, was that I could focus on something else: the overarching theme was a call for sustainability in tech & other domains. For instance, I learnt that distressed jeans are not so great for the people creating them. I’m not ready to give up on jeans but I’d like to be better about where I get mine from (babysteps to Patagonia maybe?).

In tech itself we need to figure out how to bring people together to build technologies that last and how to take care of ourselves and each other so we can keep doing that. These are issues for everybody: men as much as women. We need to do this together.

And that’s what I experienced happening at Monkigras. Once tech figures out how to deal with gender imbalances, we can truly all move together onto new horizons. Building sustainable worlds and resilient societies.

Beware diversity-led marketing (& three cheers for these companies doing diversity right)

IMG_20171025_133846

There’s an increased level of discussion on improving diversity in tech communities. Or maybe it was always there and I just started paying attention. Either way, it’s a really good thing. But in the mix there is definitely a certain amount of diversity-led marketing e.g “We need more women-in-tech! Women, sign up for this tech course here…”.  Some are more obvious than others.

What about tech conferences? Take the case of a tech conference getting some promotion off the back of free diversity tickets. Is that just diversity-led marketing and a bad thing? After all it’s great to get underrepresented folks into conferences, right? The answer is if it is done in isolation then it is probably just self-serving marketing and pr. Underrepresented folks aren’t merely props for your agenda

Diversity is complicated. It’s easy to get it wrong and end up like the conference organizers who violated their own code-of-conduct and had a speaker cancel their talk.  Or fall into the Google case of  trying to inspire teen girls to code while simultaneously systematically limiting the careers of women in your company.  Again diversity is complicated, so we all want to focus our energies on those doing it the right way.

Open source has a worse record than most when it comes to diversity.  At this year’s Eclipsecon Europe, as with every tech conference I attend, I did my own evaluation of what’s being done well and not so well. This year, I noticed a few companies who are doing something right as evidenced by the women and underrepresented minorities that are in leadership positions.

  1. IBM – IBM had a noticeably improved presence at this year’s conference. I learnt that they actively encouraged speaker proposals. If a talk is accepted, there are good company policies in place to ensure speakers can travel and attend the conferences. As a result we had four awesome women speakers from IBM, not just any speakers, but experts in their respective fields: Eclipse JDT leads Noopur Gupta and Sarika Sinha, Eclipse SWT committer Lakshmi Shanmugam and Eclipse Microprofile committer Emily Jiang.
  2. OBEOOBEO specialize in graphical modelling and are well respected in the community. Melanie Bats is one of the rockstars in the community, doing terrific and imaginitive tech talks and also recently took over as the Eclipse Planning council lead. OBEO recently promoted Melanie to CTO, which is written about beautifully here: Zero-to-CTO.
  3. BREDEX – BREDEX specialize in testing and are well represented at EclipseCon by the indefatigable Alex Schladebeck. Alex can be found leading the highly enjoyable Kahoot quiz at EclipseCon as well as heading up the Project Quality day. Doing great things in the testing world, it was great to learn that Alex has been promoted to  ‘Head of Software Quality and Test Consulting’ at BREDEX.

These three companies set a great example for the rest of us, not to mention make us better at our work as a community. Which brings me to the picture at the top of this blog post. I like to get setup really early before I do a tech talk, especially one in a huge room with a massive screen. So while getting setup, Jonah Graham of Kichwa Coders and Sarika Sinha of IBM got into a discussion about debugger protocols and threading issues. To discuss the finer points my laptop was commandeered and out came the code. It was one of those serendipitous moments and I didn’t want my pre-talk nerves to stop them. So I took a seat and took pictures while taking deep breaths. I think my talk went well anyway. That one conversation really informed our thinking on our work on the future of debuggers. And really it reminded me in a powerful way how things are always better the more different types of people you get involved. And little moments like these make it all worthwhile, and worth doing right, in the best way possible.

The Future of Developer Tools for IoT, ThingMonk 2017

ThingMonk is an annual London conference that brings together the people building and shaping the Internet of Things. This year I spoke at the conference on ‘The Future of Developer Tools for IoT’. This talk looks at emerging and future trends in the developer tools space. Check out the slides and feedback from the audience, as well as reference links at the end. Plus thanks to Marcel Bruch & Codetrails for input on AI tools. Be sure to share your thoughts on how you see developer tools shaping up to scale for building the Internet of Things.

Continue reading “The Future of Developer Tools for IoT, ThingMonk 2017”

What is it like to work in Open-source?

Open-source software (OSS) is computer software with its source code made available with a license in which the copyright holder provides the rights to study, change, and distribute the software to anyone and for any purpose. – Wikipedia

Yannick_opensource

I am Yannick Mayeur, a French computer science student currently gaining work experience at Kichwa Coders in the UK, and this is how I feel about working with Open-source.

Why Open-source ?

Let me tell you a story. A company asks someone in their software team to build some software to do a certain task. It takes him a lot of time but he manages to do it. He is the only one working on the project so there are no comments in the code nor any documentation to help maintain the code. He later leaves the company, the software slowly becomes useless as nobody else knows how to use it.

If this company had created an Open-source project instead, this problem wouldn’t have occurred.

Help spread Open-source – or ensure a job for life by using this guerrilla guide on how to write unmaintainable code. But seriously don’t.  Continue reading “What is it like to work in Open-source?”