Open Source - for developers

(Kev Mayfield) #1

The tag ‘for developers’ is important for me, not because I’m a developer but I need to make a distinction between what open source is perceived as compared to what developers think it is.

For example:

Looking at this thread, An Open Clinical Terminology? As a developer I don’t need a terminology browser but I do need an API or something like to access SNOMED. This occurs whether I’m implementing a NoSQL, SQL or form solution. Rory suggestion of using GitHub - IHTSDO/snowstorm: Scalable SNOMED CT Terminology Server using Elasticsearch sounds like it fits this requirement and as a developer I could probably help with this part of it snowstorm/using-the-fhir-api.md at master · IHTSDO/snowstorm · GitHub

Similarly with the current push to use FHIR, as a developer I want to check my work is correct. For transfer of care NHSD have developed a test system but I need to be near to the end of the project before I can start using them. A developer (working on FHIR) needs to be able to test their work early on. You wouldn’t expect a builder to wait until the house is nearly complete before checking the doors open or the cement is holding the bricks together.

We have some tools which are described here HOWTO Check you FHIR code is correct (Transfer Of Care, LHCRE, Digital Child Health, Care Connect, GP Connect, etc) but that’s just part of toolset (how about checking the first floor is done). That project is now complete but as a developer I’m going to want to extend it, fix faults, move it to the latest international validation software (it should move to the r5 validator :slight_smile: ).

How as a (NHS/supplier) developer do I get to make these changes? It sounds quite simple, I know the HAPI FHIR libs and can do the changes myself but it takes time. The time is an issue though, I could be working in a hospital and spending several days/weeks needs to be justified - I’d be employed to support the hospital not (international/UK) open source.
In addition open source is viewed as something that’s run at a project level (a vertical) but the open source I mention here runs across them (a horizontal) (both the term and FHIR validation are horizontals, they are not for a specific project (/vertical) but may be used in many of them)

How do we get around this? Should suppliers contribute (to gain PR), could trusts somehow get funding to support NHS wide development efforts.

1 Like
(rory) #2

Thanks for the call out @mayfield.g.kev and my two pennies worth.

My experience with running open source repositories that squarely sit in the horizontal world of terminology is that we get greatly appreciated contributions from individual developers and a very few niche suppliers, but we get almost no contribution from the larger players. We know the repositories are being used, but, sadly, the approach from many is that open source is a one-way street. We use open source software ourselves, but always contribute back anything that could be useful to others, but this is not as common as it should be in healthcare software development.

Terminology is a good example of a horizontal technology stream (irrelevant of whether that be SNOMED CT or other terminology standards) that should make use of open source platforms such as the one I suggested. However, terminology is also a little easier than other horizontal domains. There are not really many possible variations on the theme, and if organizations like SNOMED International actively fund the open source development, then there is no dependency on people working outside of the day job to maintain the software, we’ll take any feedback and improve/change where we can.

If major players were willing to contribute back any changes or improvements they make to any open source code, this would only help everyone using the horizontal services.

1 Like
(Marcus Baw) #3

This is a great and very interesting thread. It would be interesting to see how we can encourage greater participation from the larger players, and encourage them to contribute back.

(Pablo Pazos Gutiérrez) #4

IMO there are many ways of contributing with an open source project or product that are not related with direct coding. Just testing tools and reporting issues or ideas for improvements, is a big contribution, also asking for new features and engaging on discussions about those, another big contribution.

There are some products that are vertical to one specific need, context or type of user, but in an integrated context still need to use common underlying services, like terminology servers, clinical data repositories, message buses, demographic repositories, multimedia repositories (PACS), authentication and authorization, data query and analysis, data validation and processing, clinical decision support (rules, alerts, reminders, suggestions, …), etc. These services are horizontal and form an integrated platform every hospital has or should have (if they want to make sense of the data and have an integrated environment).

On the other hand, many open source developers focus their projects on the vertical part (apps) not on the horizontal part (service platform). Users need the former, developers need the latter. As an open source developer myself, it is difficult to market a product on the service platform side and reach other developers that might need to integrate services we develop into their architectures. For instance, Rory mentioned repositories. There are no much generic, standardized and dev oriented clinical data repositories out there, and I’ve been struggling first developing and second sharing info about the one I created, the EHRServer. But slowly some people find it, try it, and start asking questions, reporting issues and proposing new features, and even some contributed with code.

1 Like
(Kev Mayfield) #5

That’s exactly the kind of feedback I was grateful for. Were we developing the correct product, what should we be doing, etc…
At times we would think is anyone using this code? [I’ve since changed roles and found out that’s a resounding yes… all the hospitals and trusts in the region have been recommended to use it]

(Emlyn) #6

I’d say the problem is a wider issue for open source in general but because we’re fishing in such a small pool we notice it more in health related open source (even more so if one limits it to NHS open source).

There are some interesting stats around open source contributions here:

It’s a couple of years old but I suspect most of it still applies.

I’d agree with Pablo that this relates to more than just the developer role. Good documentation, how-to guides etc go a long way to “convincing” people to pitch in. As does a well managed list of a project’s open issues for folk to get stuck in to with a clear, well described PR process to back it up. It needs a broad range of skills to do properly.

As Kev says though - it needs dedicated time from at least a few individuals. It’s chicken and egg though I guess? Until NHS organisations see the benefit of open source they or their suppliers won’t give the time (or even hire the skills in some cases), but without the time/skills the benefits never happen.

I’ve no idea what the answer is. Perhaps NHSx are going to kick it in to life? Perhaps we need to dream up a way ourselves?

(wolandscat) #7

I’ve said it many times in the past, maybe just to be unpopular, and I’ll say it again here: open source is no guarantee of goodness. Most software is unremittingly bad, being built by hackers with poor modeling skills weak analysis, and only a superficial grasp of requirements. Most database schemas and data are the same. Thus, most open source software is bad. The only good open source projects are those rare ones with a proper design and quality culture, documentation, and with a functioning community - just like the only good commercial products (they do exist).

Open source does one useful thing that closed source does not: it allows good quality developers from multiple companies and/or other orgs to all work together on something common, and it allows that common thing to live outside any particular company or institution.

But to do good software that will actually matter to anyone, you have to do good software engineering. Proper requirements, design and so on (Boehm’s spiral model, for those that care about such things).

As some people here know, I am mainly concerned with open platform (back-end) software, where not only the implementation is open source (or may not be), but the entire architectural specifications are open. With a well-designed, open platform architecture, good quality, long-lived implementations are possible.

Good quality apps can result as well, if they use the APIs and semantic models of the platform.

Most other software, hacked together on private models and data structures will have a short life and attract very few developers; ultimately, its investment will be wasted. Open source or not.

In summary: there is no short-cut to quality.

(Emlyn) #8

I don’t think that makes you unpopular. I hope that nobody is suggesting that open source magically makes software good. It’s a licence not a methodology.

I’d also agree that those bad software practices are rife -and that is exactly why I’d prefer not to trust any software that I can’t do an independent code review on. That’s maybe just me though?

Ignoring quality, I’d say the real value of open source is the sharing of experience and the creation of a low entry barrier for folk wanting to build quality software - commercial or otherwise. That is particularly true when talking about the horizontal/underpinning services that Kev originally spoke about. When we’re taking about software used in public funded organisations that value is even stronger.

In my view - it’s oxymoronic to want something to be widely adopted and at the same time try to protect commercial interests in that thing. MongoDB inc. may disagree right now but “even Microsoft” seems to be coming to that conclusion.

(Kev Mayfield) #9

It’s also the feedback loop and culture. I don’t think anyone would disagree if I say the culture of the NHS is bureaucratic and on top of that we have human nature.

So when we ask ‘Send Reinforcements, we’re going to advance’ it often becomes something else ‘Send three and fourpence. We are going to a dance’

For example: I work with many API’s and systems and I want a standardised API, probably RESTful using JSON. So have other developers and the request goes up the chain as a common requirement. As it’s gone up the chain it’s attracted other interested parties, many naturally want to push their own requirements or products.
Naturally the requirement has morphed into something different, it may sound similar but is now a more generic requirement but we’re not advancing now and have a choice of open dances to go to.

That’s an example of a vertical, it had feedback as it ascended from architects, suppliers, standards organisations, clinicians, modellers, etc but naturally that concentrated on their own areas and the culture supports this.

So how to bridge this?

  • Forums - have as few as possible (if I want to talk to other developers I’ve over 10 different channels: slacks, email, zulip, confluence)
  • Generative culture - we can’t change the organisation but can we get support to be more open? How do we encourage dev teams to be more open (and get staff to feel more comfortable to share).
  • Hackathons - Need more (need more NHS developers to attend and them to be encouraged to attend)
  • Workshops/Hackshops - Feel we don’t have enough of these at a developer level. Many of those labelled as tech tend to match the hierarchy of the NHS and so IT manager focused.
(wolandscat) #10

Well, to go a bit further, I would add something I have advocated for a long time: any large organisation has to devise its own platform, i.e. a single coherent information architecture. Naturally they don’t want to have to literally develop it all (or even much) from scratch, but they do need to do non-trivial work to source and integrate scientifically developed and proven architectural elements (models, languages, methods, processes etc) to create the organisational platform. Some things they need to develop on their own (at least clinical models of information and process). The resulting architecture needs to be completely coherent at all levels: typing, languages, tooling, semantics, or it will just be a recipe for endless chaos.

If such a platform existed in the NHS, all developers would just be working either on different parts of it, or on applications that use it. Every little discovery by a developer would have a place to be recorded and add to the platform.

This is not however the history of the NHS nor other similar organisations, e.g. US Veterans Administration. Instead, it deludes itself that they can go on a shopping trip for standards via which they will perform magical ‘interoperability’ work that will make all systems talk to each other.

Unfortunately, nearly all standards are highly self-inconsistent (due to design-by-committee) and mutually incoherent (due to lack of a platform vision in industry + political motivations of SDOs) - the idea being that grabbing standards ‘already out there’ is a zero or low cost way to obtain the information architecture needed by the organisation.

This has always failed. We are no closer to a coherent information platform for the NHS than since I first started working in the domain here in 1994. That’s a lot of lost time.

A far more realistic scenario for an organisation like the NHS would be to create its own information engineering department - probably the equivalent of a small university - that takes seriously the need to understand the problem domain, and pursue the numerous clinical and engineering R&D activities that are needed to come up with a computing platform to serve its workers and patients. That is what it would take.

In this approach, interoperability is an outcome, not a (permanent, expensive, non-scaling) work item.

(Kev Mayfield) #11

I like the last couple of paragraphs.

It is very difficult for a new entrant (supplier) or new developer (to health) to get to understand the domain. I know when I moved from primary care to secondary I wanted to find out more, it was difficult:

Other than that, it was difficult to get more information. What about reporting, NHS data dictionaries, how does this all fit together. I had experience from primary care which helped a lot.
I looked at courses, you have stats modules at open university (good for SQL Devs), Health Informatics degrees but that’s geared up for IT project staff not technical.

(wolandscat) #12

For fun, I expanded on this topic in a blog post.

1 Like
(colin.brown99) #13

Great work, building on some of your earlier blogs. Love “… go on a shopping trip for standards …”

Re “ … the NHS to create … the equivalent of a small university.”

Your proposal is for the engineering and informatics domains, which are shared not only UK-wide but internationally.

And may I add: so is clinical practice, essentially an international platform enterprise based on Open Source principles, as per @pacharanero :

open-source-is-the-only-way-for-medicine

We need a strategy to manage the clinical content models already in use worldwide, but outside of
OpenEHR because they are proprietary. For example, most clinical systems have Tobacco Use and Family History models, developed in-house before OpenEHR published its archetypes: we need to learn from the best features in
these.

Their clinical creators never consented for their input to be monetized, and ethically it remains in The Commons of clinical practice, so we should reclaim the clinical practice already modelled in closed IT systems.

Could this gap in knowledge management be closed by a virtual “International Public Academy of Clinical Informatics”?

Some parts exist:

The Digital Commons Academy offers a model for Education, and NHS Digital Academy
for the management & leadership for Health IT in UK.

But to do the work of reclaiming all those models of Clinical Practice already out there will require substantial resource.

Is this a task for our current real University Depts – and if not, who?

(Adrian Wilkins) #14

And also due to design in isolation from any implementation.

I always rolled my eyes at HL7 v3 because there was no reference implementation when I was working on the NHS MIM tools. We had guidelines to explicitly avoid things like the GTS datatype and use more specific types, because it was a lovely abstract idea but no consideration was given to how to implement it.

Design-by-implementation isn’t always great, as you can see in the contrast between OpenDocument and Microsoft Office Open XML (or MOO-XML as I usually call it) - hundreds of pages of reasonably clear specs vs thousands that reveal mostly that they just hastily wrote an XML serializer for the internal structs of Office and published the documentation as “a standard”.

But standardizing on stuff that you know works because you have a working implementation is probably much better than designing something like a Klein bottle or Escher staircase that’s beautiful but impossible.