The Future of Developer Tools for IoT, ThingMonk 2017

ThingMonk is an annual London conference that brings together the people building and shaping the Internet of Things. This year I spoke at the conference on ‘The Future of Developer Tools for IoT’. This talk looks at emerging and future trends in the developer tools space. Check out the slides and feedback from the audience, as well as reference links at the end. Plus thanks to Marcel Bruch & Codetrails for input on AI tools. Be sure to share your thoughts on how you see developer tools shaping up to scale for building the Internet of Things.

Continue reading “The Future of Developer Tools for IoT, ThingMonk 2017”

Woohoo! Java 9 has a REPL! Getting Started with JShell and Eclipse January

With Java 9 just around the corner, we explore one of its most exciting new features – the Java 9 REPL (Read-Eval-Print Loop). This REPL is called JShell and it’s a great addition to the Java platform. Here’s why.

With JShell you can easily try out new features and quickly check the behaviour of a section of code. You don’t have to create a long-winded dummy main or JUnit test – simply type away.  To demonstrate the versatility of JShell, I am going to use it in conjunction with the Eclipse January package for data structures. Eclipse January is a set of libraries for handling numerical data in Java, think of it as a ‘numpy for Java’.

Install JShell

JShell is part of Java 9, currently available in an Early Access version from Oracle and other sources. Download and install Java 9 JDK from http://jdk.java.net/9/ or, if you have it available on your platform, you can install with your package manager (e.g. sudo apt-get install openjdk-9-jdk).

Start a terminal and run JShell:capture1

As you can see, JShell allows you to type normal Java statements, leave off semi-colons, run expressions, access expressions from previous outputs, and achieve many other short-cuts. (You can exit JShell with Ctrl-D.)

Using JShell with Eclipse January

To use Eclipse January, you need to:

1. Download January:

Get the January 2.0.2 jar ( or older version January 2.0.1 jar).

2. Download the dependency jars:

The January dependencies are available from Eclipse Orbit, they are:

3. Run JShell again, but add to the classpath all the jars you downloaded (remember to be the in the directory you downloaded the jars to):

Windows:

"c:\Program Files\Java\jdk-9\bin\jshell.exe"  --class-path org.eclipse.january_2.0.2.v201706051401.jar;org.apache.commons.lang_2.6.0.v201404270220.jar;org.apache.commons.math3_3.5.0.v20160301-1110.jar;org.slf4j.api_1.7.10.v20170428-1633.jar;org.slf4j.binding.nop_1.7.10.v20160301-1109.jar

Linux:

jshell --class-path org.eclipse.january_2.0.2.v201706051401.jar:org.apache.commons.lang_2.6.0.v201404270220.jar:org.apache.commons.math3_3.5.0.v20160301-1110.jar:org.slf4j.api_1.7.10.v20170428-1633.jar:org.slf4j.binding.nop_1.7.10.v20160301-1109.jar

Some notes:
Some version of jshell the command line argument is called -classpath instead of --class-path
If you are using git bash as your shell on Windows, add winpty before calling jshell and use colons to separate the path elements.

capture2

Then you can run through the different types of January commands. Note JShell supports completions using the ‘Tab’ key. Also use /! to rerun the last command.

Import classes

Start by importing the needed classes:

import org.eclipse.january.dataset.*

(No need for semi-colons and you can use the normally ill-advised * import)

Array Creation

Eclipse January supports straightforward creation of arrays. Let’s say we want to create a 2-dimensional array with the following data:

[1.0, 2.0, 3.0,
 4.0, 5.0, 6.0,
 7.0, 8.0, 9.0]

First we can create a new dataset:

Dataset dataset = DatasetFactory.createFromObject(new double[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 })
System.out.println(dataset.toString(true))

This gives us a 1-dimensional array with 9 elements, as shown below:

[1.0000000, 2.0000000, 3.0000000, 4.0000000, 5.0000000, 6.0000000, 7.0000000, 8.0000000, 9.0000000]

We then need to reshape it to be a 3×3 array:

dataset = dataset.reshape(3, 3)
System.out.println(dataset.toString(true))

The reshaped dataset:

 [[1.0000000, 2.0000000, 3.0000000],
 [4.0000000, 5.0000000, 6.0000000],
 [7.0000000, 8.0000000, 9.0000000]]

Or we can do it all in just one step:

Dataset another = DatasetFactory.createFromObject(new double[] { 1, 1, 2, 3, 5, 8, 13, 21, 34 }).reshape(3, 3)
System.out.println(another.toString(true))

Another dataset:

 [[1.0000000, 1.0000000, 2.0000000],
 [3.0000000, 5.0000000, 8.0000000],
 [13.000000, 21.000000, 34.000000]]

There are methods for obtaining the shape and number of dimensions of datasets

dataset.getShape()
dataset.getRank()

Which gives us:

jshell> dataset.getShape()
$8 ==> int[2] { 3, 3 }

jshell> dataset.getRank()
$9 ==> 2

Datasets also provide functionality for ranges and a random function that all allow easy creation of arrays:

Dataset dataset = DatasetFactory.createRange(15, Dataset.INT32).reshape(3, 5)
System.out.println(dataset.toString(true))

[[0, 1, 2, 3, 4],
 [5, 6, 7, 8, 9],
 [10, 11, 12, 13, 14]]


import org.eclipse.january.dataset.Random //specify Random class (see this is why star imports are normally bad)
Dataset another = Random.rand(new int[]{3,5})
System.out.println(another.toString(true))

[[0.27243843, 0.69695728, 0.20951172, 0.13238926, 0.82180144],
 [0.56326222, 0.94307839, 0.43225034, 0.69251040, 0.22602319],
 [0.79244049, 0.15865358, 0.64611131, 0.71647195, 0.043613393]]

Array Operations

The org.eclipse.january.dataset.Maths provides rich functionality for operating on the Dataset classes. For instance, here’s how you could add 2 Dataset arrays:

Dataset add = Maths.add(dataset, another)
System.out.println(add.toString(true))

Or you could do it as an inplace addition. The example below creates a new 3×3 array and then adds 100 to each element of the array.

Dataset inplace = DatasetFactory.createFromObject(new double[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 }).reshape(3, 3)
inplace.iadd(100)
System.out.println(inplace.toString(true))

[[101.0000000, 102.0000000, 103.0000000],
 [104.0000000, 105.0000000, 106.0000000],
 [107.0000000, 108.0000000, 109.0000000]]

Slicing

Datasets simplify extracting portions of the data, known as ‘slices’. For instance, given the array below, let’s say we want to extract the data 2, 3, 5 and 6.

[1, 2, 3,
 4, 5, 6,
 7, 8, 9]

This data resides in the first and second rows and the second and third columns. For slicing, indices for rows and columns are zero-based. A basic slice consists of a start and stop index, where the start index is inclusive and the stop index is exclusive. An optional increment may also be specified. So this example would be expressed as:

Dataset dataset = DatasetFactory.createFromObject(new double[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 }).reshape(3, 3)
System.out.println(dataset.toString(true))
Dataset slice = dataset.getSlice(new Slice(0, 2), new Slice(1, 3))
System.out.println(slice.toString(true))

slice of dataset:

[[2.0000000, 3.0000000],
 [5.0000000, 6.0000000]]

Slicing and array manipulation functionality is particularly valuable when dealing with 3-dimensional or n-dimensional data.

Wrap-Up

For more on Eclipse January see the following examples and give them a go in JShell:

  • NumPy Examples shows how common NumPy constructs map to Eclipse Datasets.
  • Slicing Examples demonstrates slicing, including how to slice a small amount of data out of a dataset too large to fit in memory all at once.
  • Error Examples demonstrates applying an error to datasets.
  • Iteration Examples demonstrates a few ways to iterate through your datasets.
  • Lazy Examples demonstrates how to use datasets which are not entirely loaded in memory.

Eclipse January is a ‘numpy for Java’ but until now users have not really been able to play around with it in the same way you would numpy in Python.

JShell provides a great way to test drive libraries like Eclipse January. There are a couple of features that would be nice-to-have such as a magic variable for the last result (maybe $_ or $!) and maybe a shorter way to print a result (maybe /p :-). But overall, it is great to have and finally gives Java the REPL and ability to be used interactively that many have gotten so used to with other programming languages.

In fact we will be making good use of JShell for the Eclipse January workshop being held at EclipseCon France, see details and register here:  https://www.eclipsecon.org/france2017/session/eclipse-january

eclipseConV2

Diversity means Open Source for a New Generation

img_20160511_183256

Last month the Eclipse Foundation announced a new Diversity & Inclusion Champion, Thea Aldrich. I did cartwheels in my head when I heard the news. Why? Because I can’t do cartwheels in real life. But seriously, this is really a BIG DEAL.

When it comes to open source communities like Eclipse, you don’t need big data to know there is a diversity problem, you just have to show up to an EclipseCon and look around.

While this has been the case for years, recently there has been a change. Maybe it’s down to more consciousness of the issues or maybe more maturity in the community, but people have noticed and want to do something about it.  About a year ago, a grassroots effort led by Alex Schladebeck was started with the renewed goal of improving diversity in the community and at conferences. I was honoured to be part of the small but effective team. One big goal of ours was simply to raise awareness and start an open conversation about issues surrounding diversity. We did just that, through:

  • Writing: blog post, after blog post, after blog post
  • Speaking: I do a diversity talk ‘7 Habits of Highly Diverse Communities’, which has been well received and requested by other open source communities.
  • Online Discussions: we have a dedicated Mattermost channel at Eclipse for the topic, plus several web conferences.
  • In Person Discussions: we have had Diversity BOFs at EclipseCons, as well as addressed the Eclipse Board of Directors & Members’ meeting.

And we’ve been learning a lot along the way, such as when we tried and failed to secure a woman keynote speaker for one of the conferences. However,the best thing about all this has been how the rest of the community has responded. People have shown up, got involved, asked questions, challenged things (I expect nothing less of developers!) and offered support.

Throughout this, and even from the beginning, we have always wondered how we can sustain these efforts and indeed how we can expand them to do more. There is no quick fix for promoting diversity, more just a continuous and determined set of steps in the right direction. And in reality it would never work long term without someone dedicated to spearheading the changes.

So that is why, in less than a year after we implored the Foundation to make this happen, they listened, took us seriously and did it! Not just that, the Eclipse Foundation now becomes an Open Source Foundation investing in change, with a dedicated role to diversity. We can hope this becomes a must-have role for every open source foundation out there.

I’ve always said how much I love this community because of the ability to adapt to changing environments, and here is more proof. Thea has already kicked off efforts on multiple fronts:

  • Identifying ways in which all Eclipse events are inclusive events and welcoming,
  • Rolling out an ambassador program to involve the community in welcoming newcomers into the ecosystem,
  • Reaching out to established projects to see how we can support their efforts,
  • Making all Eclipse Foundation websites and resources easier to navigate for native non-English speakers,
  • Providing the community with a direct path to Foundation staff for ideas, complaints, feedback and other issues that our community or members may encounter.

We look forward to working with Thea, the Foundation and community on all these aspects to keep making improvements and bring about a real change.I will continue to do what I can, which includes running for a seat on the Board of Directors. Having this new focus and investment in diversity means we can look forward to bringing open source to a whole new generation of developers.

The Sound of the Universe @ EclipseCon Europe 2016

At EclipseCon this year I heard the sound of the universe. And it was awesome and breathtaking. To be precise, it was the sound of two black holes colliding over a billion years ago,  part of the enthralling final keynote from Dr Benno Wilke on detecting gravitational waves. It was a fitting way to end a conference that had kicked off with another amazing keynote:  Stephen Carver delivering a powerful and emotional story of the people and tech behind the space shuttle disasters, framed in profound lessons on real communication and avoiding silo thinking.

For the very first time at EclipseCon Europe we held a CDT summit. Over 10 years ago I had the honour of being the first developer from Europe involved in CDT, so to bring the summit to Europe was a particularly special moment for me, especially with our renowned project co-lead Doug Schaefer in attendance. The summit was a success, particularly welcoming contributors from the wider community into the fold, and will definitely something we will be doing again next year.

As this year’s focus there was also a big community focus on diversity and raising awareness on this topic. The activity included my talk on ‘7 Habits of Highly Diverse Communities‘, addressing the board on the topic and a diversity BOF session. The discussions were great, lots of good energy, practical suggestions and I am so proud to see the community work together to ensure we can be as open and inclusive as possible.

The Science Working Group had good reason to celebrate at the conference: we have just completed our very first simultaneous release of five projects. A significant milestone for this nascent group, and was terrific to talk about the projects to the rest of the community.

There was an incredible amount on at the conference this year, the best way to get a quick taste was hearing what people enjoyed: language servers, Xtext, Sirius, scripting, IoT & testing were topics that kept coming up. On a personal level, it was my most intense EclipseCon yet with three talks, a BOF and a summit to organize. On the whole it was the busiest conference yet with a record attendance of 619. The most important thing is always the people: lots of new and old friends to talk to and exchange energy. At EclipseCon this year I heard the sound of the Eclipse universe. And it was awesome and breathtaking.

(A version of this article was first published on jaxenter.com: https://jaxenter.com/eclipsecon-europe-at-a-glance-129883.html)

Suspicious Semicolon: CDT at EclipseCon France 2016

before_suppress

The CDT project was well represented in Toulouse this year.

CDT Latest & Greatest

Jonah Graham gave a CDT overview with CDT: Latest & Greatest Tooling for C/C++. Mostly hands-on, Jonah used the C-implementation of the Python interpreter to demonstrate how to set-up, build and debug a substantial sized project with CDT.

 

This included showing some of the new features in Neon like suppressing Codan warnings and the enhanced memory view. Jonah also talked about the upcoming features including the new GDB console and improved multicore breakpoint support. Continue reading “Suspicious Semicolon: CDT at EclipseCon France 2016”