Skip to main content
Version: 3.1.3

Management Portal, Pipeline Visibility and Observability

Data Observability Tool​

Centaur® Data Platform’s Data Observability Dashboard provides real-time visibility into each pipeline's data processing stage, enabling proactive monitoring and effective management of data processing and quality. From initial data ingestion to final output, Centaur® Data Platform empowers users to track progress, identify bottlenecks, and optimize workflows and data quality.

The dashboard is utilized for real-time monitoring of data processing, failure detection, and capturing and displaying Azure performance metrics. Set alerts, integrate data sources, and scale seamlessly for reliable performance.

Centaur® Data Platform's Data Observability Dashboard consists of the following components:

  • Overview: This section provides an overview of the total number of files processed by the Centaur® Data Platform and the different file types received for conversion.

    • Total Source Files Processed: Number of source files processed in the specified time.
    • Total FHIR Resources Processed: Number of FHIR resources processed (including reprocessing references) in the specified time.
    • Total Source Formats: Number of CCDA, HL7™ Message, JSON or CSV files ingested in the specified time.

  • Service and FHIR Resource Overview: This section provides an overview of success and failure count of each component included in Centaur® Data Platform.

    • HL7™ Engine Processor: Number of files successfully processed and failed to process, retrieved from the Azure service bus queue.
    • HL7™ Validator: Number of files that have been successfully checked for validation, and the count of files which failed validation. Typically, the error codes 171 and 172 indicate validation failures.
    • Conversion: Number of files successfully converted into FHIR resources, and the count of files that failed conversion.
    • Curation: Number of files that have been successfully curated, and the count of files that failed to be curated.
    • Validation: Number of FHIR resources that have been successfully compared and validated against the HL7™ FHIR IG specifications, and the count of resources that failed validation.
    • Reference Handling: Number of references that have been successfully checked in the FHIR resources, and the count of resources that failed reference checks.
    • Retry Reference: Number of references that have been successfully handled, and the count of resources that failed retry reference.
    • FHIR Ingestion: Number of FHIR resources that have been successfully ingested into the FHIR server, and the count of resources that failed ingestion.

  • Success Metrics: This section displays the successful outcomes of each component as a time series graph, with the number of files represented on the Y-axis and time represented on the X-axis.

    • Error Trends: This section displays the error trends of each component as a time series graph, with the number of files represented on the Y-axis and time represented on the X-axis.
    • Service Bus view: This section provides a count of each queue message from the Azure service bus, which is synced from the Azure account.
    • System Information: This section provides the number of replica scales used by each component container app from Azure.
    • Memory Usage: This section displays the memory usage of each Centaur® Data Platform service container app, which is synced from Azure.

Management Portal

Management Portal​

The Centaur® Portal v3.1.3 offers platform management for all the components of the Centaur® Data Platform, including Platform Monitoring. The platform management section is currently focused on HL7™ Engine data traceability. It allows users to view channels, configure channels, view messages, and retrigger messages if they encounter errors. The platform monitoring section provides quick access to the dashboards related to the Centaur® data processing from source to destination, as well as a dashboard specifically for HL7™ Engine success/error metrics.