Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

APHL

CDC

Peraton

Dari Shirazi:

Megan Light:

Erroll Rosser: X

Vanessa Holley : X

Teresa Jue:

Kristin Peterson : X

Brooke Beaulieu: X

Cheri Gatland-Lightener: X

Tom Russell: X

Mel Kourbage:

Rachael Aubert

Scott Rank: X

Gretl Glick: X

Dawn Sievert

Laura Carlton: X

Patrick Caneer: X

Sam Abrams: X

Kim Del

Michelle Meigs

Emily Augistini: X

Goals

  • Update on Data Lake Migration Status

...

Item

Notes

Overall Status Updates

  • Project Status:

    • CELR DL Migration to CDC Platform

    • 2 main pieces to migration:

      • 1- Capability to process data in Azure at CDC

      • 2-Integration with DEX at CDC Front Door

    • Peraton: Overview

      • 1-CELR Pipeline-Validation of HL7, CSV for CV19 info into merged Lab Record from both formats

        • 4 major areas: Focus on Pipeline--Key Activities

        • 1-Re-engineer CELR CSV and HL7 Pipelines from Amazon Workspace (AWS) to CDC Azure Cloud.

        • 2-Splitting pieces: Migrate Ingest, Validation, and Redact functionality for COVID-19 Lab Records submitted to CDC from CELR to Data Exchange (DEX).

        • 3-Develop transformation and lab result algorithms for Covid-19 Lab Results submitted to CDC

        • 4-Re-engineer CELR Portal within CDC Azure Cloud

      • 2: DEX Integration:

        • Option 1: SDS (Send to DEX Service) / Upload API

        • Connectivity test successfully completed w/ AIMS & DEX Team

        • In Progress:

          • AIMS: Confirm & identify ongoing operations, resources, support/training for AIMS>DEX, Upload API: Tentative: Late August

          • DEX: Continue internal DEX performance/load testing: Target readiness for Mid August 2023

          • AIMS: Send CELR Production data (n = 24 hours~) to DEX STG Environment: Target: ~August 21, 2023 (will confirm dates as we assess readiness)

          • DEX can accommodate routing and validation of HL7; DEX can accommodate routing of CSV files, but cannot currently validate (dev in progress)

          • Dependency: CELR Migration to CDC Platform

DL Migration Updates: CELR

  • Status Updates and Timeline:

    • Peraton Update:

      • Key Activities:

      • 1-Re-engineer CELR HL7 pipeline into Azure cloud: Currently in QA Testing

      • 2-Re-engineer CSV pipeline into Azure Cloud: Currently in QA Testing [Separate from DEX integration]

      • Migrate HL7 & CSV pipelines into DEX

        • HL7: Ingest validation, redaction--currently in CELR (Aims)--migrating into DEX platform: Completed

        • CSV: Ingest, validation, redaction: Will be migrated to DEX, dev in progress (can currently accept CSV, validation dev in progress): Timeframe: TBD (DEX evaluation for dev/ start date in progress) [70% of data for CELR is CSV uploads)

      • 3: Re-engineer portal: In QA testing for HL7 & CSV

      • 4: Generating final data set and transmission of data set from AIMS>HHS (Redshift DB): Not yet started (Implementation plan in place): Timeframe: Start July 31, 2023)--ER to provide update on status/timelines ongoing

      • 5: Implement processing pipeline in EZDX Data bricks: Includes processing of data, move into EZDX, integrate with DEX pipeline to accommodate different data formats: Dev in progress to implement change in schema: Started 7/3/23

      • Date to cutover to full production for both HL7 & CSV ON CDC: TBD--Will shift dependent on CSV development: ~6 months/Jan 2024 (will shift dependent on ongoing development)

        • HL7 is ready

        • CSV--unknown: dependent on being able to process CSV through DEX, process through data bricks

        • Potential to phase in HL7 while continuing to work on CSV development? Risk: 2 different sources (AIMS for CSV and CDC Platform for HL7): Complexities would be high-risk

DL Migration Updates: DAART

  • Status Updates and Timeline:

    • Peraton: For AR, PROCESS:

      • Currently at architecture/design phase

        • development yet to be started

      • Completed analysis for where/type of architecture to include i EZDX platform

      • Created tech architecture design to support AR Reporting

        • For CELR, would add on additional phase: multi-elr processing for inclusion in data bricks solution

    • Timeline for DAART Migration: TBD:

    • CGL: DAART is priority, additional discussions on down-stream processing for readiness; Goal is to create centralized, scalable DL for ELR reporting cross program areas/reporting streams

    • DEX: There would also be a DEX component for DAART for ingest (CSV and HL7 validation)

      • ER: Would also need to discuss onboarding process for DAART

Questions

  • CGL: Is the 6 mth timeline dependent on DEX support in production? ER: Yes, partially, but also dependent on migrating Redshift data into Azure cloud and reconciliation. Questions outstanding: Are we migrating historic data (potentially 4 billion records) and need to reconcile? Or are we focusing on current data set? Both options would impact timeline

  • PC: Is design phase discussions internal to Peraton?

    • ER: Yes, internal to Peraton; wanted to ensure integration with CELR/other ELR reporting platforms; Additional discussions to be held

Next Steps & Action Items

...