Structured vs Unstructured Data
Structured data fits into predefined fields, such as a discrete lab value or a coded diagnosis, and is easily searched and analyzed. Unstructured data, such as free-text clinical notes, scanned documents, photographs, and audio recordings, lacks a predefined format and is much harder to govern, search, and analyze. RHIA candidates should understand the governance challenges unstructured data creates.
Why Unstructured Data Is Hard to Govern
- It cannot easily be validated against a defined set of allowable values.
- It often contains protected health information embedded in free text, making it difficult to apply automated redaction or classification.
- It is harder to index and search, complicating both clinical use and legal discovery (release of information requests).
- Duplicate or near-duplicate documents, such as multiple scanned copies of the same consent form, can accumulate without automated deduplication.
Natural Language Processing and Governance
Natural language processing (NLP) tools can extract structured data elements from unstructured clinical notes, such as identifying a documented allergy or a specific symptom. Governance oversight of NLP tools is important because extraction errors, false positives or missed information, can affect clinical decision support and quality reporting accuracy.
Governance Strategies for Unstructured Data
- Document type standardization: Assign structured metadata (document type, date, author) to every unstructured document to enable search and retrieval.
- Access controls: Apply the same role-based access rules to unstructured content as to structured data.
- Retention tagging: Tag scanned and unstructured documents with retention metadata so lifecycle policies can still be applied.
- Quality review of NLP output: Periodically audit NLP-extracted data against source documents to verify accuracy.
- Redaction and classification tools: Use automated tools, validated by human review, to identify PHI embedded in free text for appropriate handling during disclosures.
Scanning and Legacy Paper Conversion
When converting legacy paper records to scanned images, governance policies should specify indexing standards, image quality thresholds, and quality control sampling to ensure scanned documents are complete, legible, and correctly linked to the right patient record.
As EHRs accumulate ever-larger volumes of free-text and imaging data, unstructured data governance is an increasingly important and testable RHIA topic.