Data Curation, Validation, and Supported FHIR Resources
HL7 Engine
The Centaur® Data Platform's HL7 Engine routes HL7 ORM(Order) messages from Athena to JCI and Sectra and HL7 ORU(Results) messages from JCI to Sectra and Athena. It uses the MLLP protocol. The engine reprocesses the message if it fails to send it to the destination or if there is any other exception apart from the valid rejects, i.e., ACK returns as AE (Application Error) or AR (Application Reject).
- The engine stores files as source/success or source/error for source requests.
- The engine stores files as destination/success or destination/error for destination requests.
HL7 Engine Retry Logic
The retry logic begins for all failed messages except valid rejects, i.e., ACK returns as AE or AR. The HL7 engine will retry for the first attempt after 2 seconds. If it fails, it stores the message in the folders mentioned below. When a message is put under the “retry” folder (or subfolders), it will be deleted from this location after a successful retry.
- Error Folder -
<container_name>/<destination>/error/<year>/<month>/<date>/<messageid>.hl7 - Retry Folder -
<container_name>/retry/1_min/<messageid>_<destination_name>.hl7
| Scheduler Running | Processing Directory | Max Attempts | Next Directory |
|---|---|---|---|
| Every 1 Minute | retry/1_min | 5 | retry/5_min |
| Every 5 Minute | retry/5_min | 3 | retry/15_min |
| Every 15 Minute | retry/15_min | 2 | retry/1_hr |
| Every 1 Hour | retry/1_hr | 24 | retry/error |
Note: If reprocessed successfully, messages will be deleted from the “retry” folder, and the success message will be created in the destination “success” directory.
Source Data Processing
The Centaur® Data Platform's pipelines process source data every day from sources such as New Direction, Facets, Enterprise Data Warehouse (EDW), Athena, and Altruista Guiding Care.
| Sources | Format | Source Location | Destination Location |
|---|---|---|---|
| Facets | CSV | rawzone/CDH_FACETS | cont-fhir-convertor/Facets |
| EDW | CSV | rawzone/CDH_EDW | cont-fhir-convertor/EDW |
| New Direction | FHIR JSON | rawzone/CDH_NEWDIRECTION | cont-fhir-convertor/NewDirection |
| Altruista Guiding Care | Database | rawzone/CDH_GUIDINGCARE | cont-fhir-convertor/GuidingCare |
| Athena HPDE | CCDA, CSV | rawzone/CDH_ATHENA | cont-fhir-convertor/Athena |
Historical Data Processing
The previous system’s endpoint is used to export historical FHIR bundle data, which is then leveraged by the Centaur® Data Platform and divided into individual resource types for each bundle. Centaur® Data Platform’s bulk data import is set up to work, and it can quickly ingest all the historical data within minutes. A trigger is used to commence Bulk Importing, which involves taking multiple FHIR bundles at a time and importing them to the FHIR server.
Member Record Processing
Centaur® Data Platform’s convertor reads the source data from the storage for the conversion process. During conversion, the platform verifies whether the file has. hl7, .xml, and .json extension. The convertor will generate a list of resources from the FHIR R4 specification for a particular source file. Below is the list of generated resources:
- Account
- AllergyIntolerance
- CarePlan
- CareTeam
- Communication
- Condition
- Coverage
- Device
- DiagnosticReport
- DocumentReference
- Encounter
- EpisodeOfCare
- FamilyMemberHistory
- Goal
- Immunization
- Location
- Medication
- MedicationRequest
- MedicationStatement
- MessageHeader
- Observation
- Organization
- Patient
- Practitioner
- PractitionerRole
- Procedure
- Provenance
- RelatedPerson
- ServiceRequest
Centaur® Data Platform has a built-in FHIR data mapper, which uses predefined templates to transform data from various formats into FHIR R4 bundles. These templates consist of specific mapping rules that dictate how data elements from the source format should be placed within the FHIR R4 structure. Mapping templates can be saved, reused, and easily integrated with other systems and tools – drastically reducing timelines associated with the mapping process.
In the intricate landscape of healthcare data interoperability, converting clinical data (like HL7 ADT, CCDA, XML, JSON, etc.) into FHIR R4 resources is a critical step. Centaur® Data Platform assures that no data is lost in translation, ensuring that every element finds its place in the resulting FHIR R4 resource.
Curation and Validation of Member Records in FHIR Server
The Centaur® Data Platform uses a curation process to retrieve unique identifiers mapped to specific source files and the FHIR resource IDs for each FHIR resource. This process is enabled by two configuration files: resourceConfig and sourceConfig, which have all the info related to resource identifiers and source-related configurations. These files can be customized based on source format or business needs.
The resourceConfig file contains all the unique identifiers based on the source and FHIR resource and is used only if the mapping attribute present in sourceConfig is set to identifier; otherwise, it will be a direct ID search with the FHIR server without any identifiers being used. Meanwhile, the sourceConfig file contains priority lists with primary and secondary sources and registry templates based on source. These unique identifiers can then fetch the unique ID from the FHIR server.
Apart from the identifier configurations, sourceConfig will contain another set of EMPI configurations containing all the necessary information about the EMPI Processor.
For resources such as Patient / Organization / Practitioner, the unique FHIR resource ID will be replaced with the help of the EMPI Module, which is done through an API call.
During the Validation process, the Centaur® Data Platform checks the Implementation Guide (IG) Validation to validate incoming resources. The platform also follows the HL7 FHIR specifications, which help validate the structure, cardinality, value domains, and coding / codeable concept bindings of the resource.
In the Reference Handler component of the Centaur® Data Platform, references from FHIR resource data are verified using identifiers, and a valid FHIR ID is updated for each reference from the FHIR server.
As mentioned in the curation process, the platform uses the resource identifiers retrieved from the resourceConfig to search for unique identifiers in the FHIR Server if the mapping is set to identifier else if the mapping is ID, then a direct call to the FHIR server is made using a resource type and resource ID without using any identifiers. Based on the FHIR ID present in the JSON, the message is inserted or updated into the FHIR Server.
The Retry Reference Handler feature of the Centaur® Data Platform will keep searching and processing references in the FHIR resource until it reaches the maximum retry count of 6. If the reference is found and replaced during the retry process, the FHIR resource will be updated accordingly.
HL7 FHIR Resources Supported
Centaur® Data Platform v3.1.3 supports USCDI V3 and FHIR R4 Standards