skip to main content

Deep Dive into Implementation and Testing

Tailored for technical systems developers and test leaders.

The goal of implementation and testing is to ensure that the developed IG (Implementation Guide) is translated into real-world software and that this software provides value to end users. To achieve this, the IG developers often need to coordinate with workflow subject matter experts, end users, and software vendors to develop open-source demonstration software including reference implementations and develop testing tools to support software adoption. These resources will help speed the implementation process, guide developers in adapting the software to real-world instances, and excite end users about the potential benefits of the community’s use case.

Here, we will walk through the development processes identifying common, readily available, free tools that the community can use to manage, test, and update the community’s software.

Software Development Lifecycle (SDLC)

The SDLC guides software through key steps in the development process:

  • Requirement gathering
  • Software Design
  • Coding
  • Testing
  • Deployment
  • Maintenance and Support

While software vendors will oversee this process when they adapt the community’s IG for local instances of the IG software, the community will need to follow this process for any of its open-source software, such as reference implementations or testing tools. This discussion provides a detailed overview of the process and methodologies (waterfall, agile, and others) used to guide software development teams through the SDLC phases.

Development Recommendations

Requirements Gathering

See the Use Cases & Planning page for guidance on requirement gathering.

Software Design

Software design typically encompasses both the function of the software and the user interface. However, what end users see and how they interact with the data is typically the purview of software vendors. As such, discussions on the community's role in software design focus on the design of standardized data exchange and IG development. See the Standards Development section and the associated Standards Development Deep Dive for guidance on IG development.

Coding

Development Environment

It is strongly recommended that the community develop open-source software within an environment that allows a team of developers to effectively manage the code. Key personnel like developers can and often do change over the course of the project. As a result, it is critical that there is a single repository for the community’s code, ideally one that can be made public when software is ready for release. There are many options for code management and software repositories, the most common repository for open-source projects is GitHub. To ensure that developers can work collaboratively, yet independently, it is also recommended that they use project management software such as Trello, Jira, or Microsoft Teams to track development goals and resolve bug fixes in a timely manner.

FHIR Artifacts

A common starting point for many looking to jumpstart their development is leveraging existing FHIR software artifacts such as the HAPI FHIR software, which provides working JAVA software for many basic FHIR resources. HAPI FHIR, HL7's Confluence page, and other FHIR open-source repositories provide basic FHIR software that can be adapted for the community’s purposes.

Testing

Software testing can be broken down into three software states as described in the Implementation & Testing section:

  • Initial testing - Initial development and testing stage conducted by developers
  • Prototyping – second round testing conducted with willing potential end users
  • Real-world implementation – implementation and testing of instances of software within live software systems using real-world workflows and data

The first two states are the primary focus of community software development. The final state, real-world implementation relies on community partnerships with software vendors or healthcare organizations to implement instances of software within local systems. Discussions should focus on the community's role in initial testing and prototyping and leave real-world implementation decisions to software vendors.

It is vital to develop testing tools for the development team and the community to ensure that the IG and associated implementations function as intended. Properly developed, IG testing tools can facilitate uniform adoption of the community’s IG standards and simplify individual implementations by helping developers identify issues with their software or with the IG. One tool that can be adapted to test the community’s IG is the Inferno testing toolkit, which is used across the FHIR community to test numerous IGs, and by the Assistant Secretary for Technology Policy / Office of the National Coordinator (ASTP/ONC) to verify EHRs meet certification requirements for interoperability. Another testing tool option is AEGIS Touchstone, a commercial testing framework that is also used across the FHIR community.

Initial Testing

Much of the development and testing strategy will rely on developers coordinating with use case subject matter experts. One critical need in this stage of testing is for accurate synthetic patients to test workflows. To avoid the time-consuming, monotonous process of manually creating test patients, developers should use software to generate synthetic patients, like Synthea. Synthea is open-source, and used widely by medical development communities, at Connectathons, and in private industry because it can generate populations of realistic patients with medical data such as medications, allergies, conditions, and clinical notes. For more information on Synthea’s current capabilities and instructions for generating patient data see Synthea’s GitHub page for more details.

Prototyping

To both guide community members in their individual implementations and demonstrate the software to end users, the community should develop an open-source demo, otherwise known as a reference implementation, of their IG. A reference implementation is a system-agnostic implementation of an IG that serves as an idealized representation of how the data exchange should function. Often it is a simple client-server data exchange with a few workflow examples to demonstrate software functionality and utility. FHIR IG creators use reference implementations to test and get feedback from both developers and end users in dedicated implementation feedback sessions and during Connectathons.

  • HL7 maintains an instance of the HAPI FHIR server for everyone to use. It should be noted, however, that the public HAPI FHIR server has millions of FHIR resource instances and can be heavily loaded, particularly during Connectathons. Since these resource instances are public, testers should never use real patient data on the HAPI FHIR servers or any other open-access environment. Finally, because of the large testing community using the HL7 HAPI FHIR server, HL7 may occasionally purge test data to ensure continued server functionality for all.
  • HealthIT.gov also provides a FHIR Sandbox that includes FHIR(R4) servers and the Inferno testing tool.
  • HL7 and Logica Health provide free services for developing SMART-on-FHIR apps, graphical front-ends, Electronic Health Record (EHR) simulators, and OAuth authentication support.

In this stage of development, the Inferno testing toolkit and AEGIS Touchstone, as described above, are also great tools for testing throughout this stage.

Real-world Testing

There is an overlap between real-world testing and deployment as real-world testing deploys software to local systems. The goal of this stage is to verify the efficacy and efficiency of the community's software, identify any lingering issues with IG development, and gather feedback and prepare the community for larger scale implementations. This step is largely overseen by software vendors who are best equipped to make implementation and testing decisions within their software systems.

Deployment

See the Adoption subsection of Adoption & Value for high-level guidance on promoting adoption of software.

Maintenance and Support

Software Versioning

There is one constant in life, change. FHIR, its underlying standards, security protocols, and the community’s IG will all have regular updates, some of which introduce major changes, to ensure continual improvement and refinement of data exchange processes. When creating software development plans, it is a necessity to have a well-established version control process within GitHub or whatever code repository management software the community chooses to use. See this article for specific recommendations and guidance on maintaining version control for the community’s software releases.

Over time, the HL7 community has developed multiple versions of FHIR profiles, including the foundation for all United States-based FHIR Implementation Guides, US Core. HL7 releases new US Core versions regularly, often every year and regulations are being proposed to require a minimum version of US Core that certified medical software vendors must use. HL7 is creating draft guidance on how to navigate how IGs can support multiple versions of US Core.

Back to top

© Copyright 2025 MITRE Corporation. All rights reserved.