Using Eclipse Common Navigator Framework for the NeXus Navigator

Requirement:

The Eclipse common navigator framework (CNF) is used as the basis for the NeXus Navigator within the DAWN Science platform. The nexus navigator provides a focussed view on nexus files, but in doing so removes the directory and project structure. However, more typically users need to focus on their nexus files within the context of their directory structure (often very deliberately set up and related to the experiment).

Pre-existing navigator with flat directory structure flat
Pre-existing navigator with flat directory structure flat

So the requirement is to show the existing directory (and project) structure but provide a filter such that only Nexus files are shown.

Solution:

Based on this requirement  and the existing codebase, the actual work requires more deprecating code than writing it.
There was a specific content provider class written to flatten the structure. This and the corresponding label provider are no longer required: displaying files in the view is delegated to the default Eclipse resource content and label provider. That way the files always appear the same in any navigator view.
solution-image-1

A filter is required, and the common navigator framework provides powerful extension point that allows it be implemented completely statically using xml. The filter automatically shows up in the view customizations but could be hidden if needed.
solution-image-1

Rabbit holes:

Based on past experience, using expressions in Eclipse plug-in extension points can be tricky – if they don’t work, it’s not straightforward to debug why. Usually referencing the documentation coupled with a working example is a good starting point. Here’s the filter expression for this case (which came with a certain amount of trial and error):

<filterExpression>
 <and>
  <not>
   <adapt type="org.eclipse.core.resources.IFile">
    <test
      property="org.eclipse.core.resources.extension"
      value="nxs">
    </test>
   </adapt>
  </not>
  <adapt
     type="org.eclipse.core.resources.IFile">
  </adapt>
 </and>
</filterExpression>

In particular, note it is best to use the tag rather than to ensure the filter works not just for files but anything masquerading (or adapting) to the IFile interface.

 

Testing:

There is no Java code provided that requires unit tests. Using the extension point takes advantage of the pre-existing testing and repeated use by the open source community. The expression can be verified and so tests for the navigator are naturally rolled into tests for upcoming features intended for the navigator.

 Resources:

  1. Common navigator framework – http://wiki.eclipse.org/index.php/Common_Navigator_Framework (the blog resources are particularly useful)
  2. Eclipse expressions reference – http://wiki.eclipse.org/Command_Core_Expressions

P2 Update Site for the DAWN project – the final piece of the puzzle

The Problem

The DAWN Science project has an update site, but an install of the product does not have the site automatically appearing in the ‘Available Software Sites’. This makes it difficult for users to upgrade their DAWN installation. 
Before: Sites list is empty

Workaround

Users can manually add in the site, but would need to know where to find the site. So even if users have the site URL, this is still an unnecessary hassle and definitely a significant barrier to upgrade. 

Complications

Over the years the Eclipse upgrade mechanism has changed dramatically, culminating in the P2 framework. A simple google search provides lots of out-of-date information mixed in with the key information (hint: anything referring to features and update sites is pretty much deprecated). P2 is a powerful and therefore complex mechanism. Even P2 developers admit things could be better – from the Eclipse help on P2 metadata:
 Disclaimer: Authoring p2 metadata is something we expect to support with better tooling in a future release. 
Most update site tutorials available do not address this final step, probably because its a product deployment issue, beyond the scope of the tutorials in question. 

Solution:

P2 can be a complicated beast, but the key information is out there if you know where to look.
The solution is to create a touchpoint advice file, place it the appropriate location and configure it with the site information. 
For DAWN that boils down to creating a new p2.inf file which is colocated with the org.dawnsci.base.product file. The file is configured with the update site information – here’s the key bit of magic.
This is essentially adding in a line with the instructions.configure key and the value in the specified format – see here for details
After: Sites list automatically populated with DAWN update site

Testing:

As ever, testing is the trickier part of the job. Using the existing Squish framework, two tests are added in. 
The first test launches DAWN and makes sure an update site is listed. This test ensures the list is not empty, it does not check the content of the list. While a check could be added to check the value of the site, this would make the test unnecessarily fragile and require updating every time the site is updated.  However, the content of the list of sites is added to the test logs to provide room for further checks if desired in the future. This is a regression test to ensure the codebase never reverts to the original problem. 
The second test actually provides a smoke test for the update site itself. This test launches a previously released version and then runs a full update against the DAWN p2 update site. The test passes if the p2 infrastructure returns success and DAWN restarts successfully. (Note this test is pending as it first requires the actual site locations to be approved).

Resources:

Crowdfunding PyDev

Kichwa Coders is now a silver sponsor for the PyDev project.

We’ve been using PyDev since 2008, and it is truly the best Python development environment out there. With such superb features like its context-sensitive code completion, rich debugging and testing integration it is a tool that has helped us focus on the real programming problems at hand. The development effort, led by Fabio Zadrozny, is highly responsive and always forward-looking.

We are dedicated to supporting such excellence in open-source projects for our and the benefit of the whole developer community.

We look forward to the continued development of PyDev, backed by the Python community. We are also excited by the new features and the LiClipse project.

Join us to support this worthwhile cause at indiegogo.
Read more about PyDev and Fabio’s blog where he explains more about the history and motivation for crowdfunding.

Eclipse London DemoCamp November 2009

The London Eclipse DemoCamp hosted by SkillsMatter was held on the 24th of November near the Barbican. Ralph Mueller, Director of the Eclipse ecosystems in Europe kicked off proceedings by giving the 40-strong audience an insight into Eclipse’s history as well as its future. In particular highlighting how major companies like Airbus may be turning to Eclipse in order to solve the problems they face, particularly the lack of continuity in front of very long lifecycle products. This is very interesting to see how the relationship between companies and open source continues to evolve with open source and Eclipse playing an ever increasingly important role.

There were four Eclipse demos on show. First up was Paul Gibbons from Diamond LightSource. Paul was demoing the latest version of the GDA, the scientific software used to run experiments at the UK synchrotron. Paul also announced that the GDA is going open source and the framework will be available to other synchrotron facilities worldwide. Kichwa Coders developed the initial prototype of the GDA on Eclipse and it was great to see how it has been progressing, having now gone live and having more and more functionality including 2D and 3D graphing.

Next, I was up and gave a demo of CyanIDE2 which we have been working with Cyan on. It all went smoothly as I walked through creating a new project through to getting LEDs flashing on the board. That was great as it highlighted the tools aim of being able to get a program running on a board in just a few minutes. Also it highlighted how Eclipse is allowing chip companies to drive technology forward and develop new solutions for systems-on-a-chip (SOCs).

The other two demos featured Miles Sabin showing us the latest from the Scala IDE and Neil Bartlett demoing Sun’s Microsystems tooling for JavaFX within Eclipse. Both shared the common theme that they were extending the JDT to integrate their respective languages in. The Scala talk highlighted how the team have gone down the route of using AspectJ to monkey-patch Eclipse JVM code out of necessity. They highlighted how they have not been able to get patches integrated by the core Eclipse Java team, for understandable reasons and so have had to resort to the binary-code modification route. Seems very severe, but very interesting – certainly a use of aspect oriented programming I’d never envisaged! Neil’s talk also include a nice demo of using OSGi for runtime modularity for JavaFx which was good to see.

Afterwards, we all convened to the nearby pub, ‘The Slaughtered Lamb’ for a post-mortem and drinks courtesy of Ralph Mueller. Kudos for Neil Bartlett for organizing the event and keeping it running to time. All in all, a very interesting and stimulating evening with the Eclipse community.

A Tale of Three Start-ups

Actually it was three stories of three semi-conductor start-ups being told at the Cadence VC Forum. The third annual occurrence of this event was held at the Institute of Physics in London.

Jed Hurwitz, CTO of Gigle Semiconductor spoke of how you had to be ‘Just Mad Enough’ – too much and no-one takes you seriously, too little and no-one wants to invest in you. I particularly enjoyed hearing about how he was able to take his experience in one field (video and imaging) and use it to great benefit in another area- home networking. Gigle have just successfully launched a Gigabit powerline modem in June 2009, and Jed spoke about the path that led them there, including advice for up and coming start-ups. He reminded all when coming up with their product ideas to ask themselves ‘Could someone else do it?’ – he answered this well saying yes anybody could do it, but ‘Can someone do it?’ is a different matter. No-one else may be in the right position today to do what you can.

Next we had ‘The Challenges of a 21st Century Start-up’ presented by Mirics Semiconductor’s CEO Simon Atkinson. He talked about Mirics road to success and the course corrections that they had to go through to ultimately be successful. Today Mirics have the only demodulator that covers all the worlds standards for broadcast TV – they do this with a combination of dedicated hardware and software running on the main PC or laptop CPU. Also interesting was hearing about the hardware acceleration for the next generation by using the GPU. Afterwards we chatted to Simon and discussed power utilization due to using non-dedicated hardware. While the power consumption is higher with the general purpose CPU approach, it is nearly as low as dedicated hardware (only a couple of minutes less TV watching). However, more importantly, the power consumption fits within the threshold that a consumer is happy with.

The last start-up story was about Phyworks – their CEO Stephen King talked about the ups and downs of the company. Going through many iterations and lots of fund-raising in difficult years, they also eventually found their way. Stephen King also shared his dos and don’ts of which the most memorable was his sentiment of ‘pushing people’ to get the most out of them, whether it was employees, partners or customers!

It was a well organized event, in a great location. The food on offer this time was a huge improvement over last year. The only downside was that the talks did all overrun which resulted in a very long intense session, and less time for talking over dinner afterwards. All in all well worthwhile and looking forward to next year’s event!