Companies dealing with vast amounts of scanned paperwork, like medical forms, insurance claims, and financial records, have a persistent compliance issue: removing personally identifiable information (PII) before sharing or further processing documents. Manual redaction isn’t practical as it takes a lot of staff time, increases the likelihood of mistakes, and leaves the company open to compliance risks.
In addition to being a detection issue, precision is also a concern when it comes to redaction. A webpage can hold several names, dates, and addresses, with certain ones being relevant to the specific situation. Conventional redaction techniques combine optical character recognition (OCR) with pattern matching or bespoke machine learning (ML) models.
Though, these methods have constraints when text is of poor quality, unable to express business logic at the field level, and demand ML knowledge to develop and retrain bespoke models when document formats evolve. This article showcases the automation of end-to-end PII detection and blacking out from documents and images at a large scale on AWS.
We create a personalized plan for protecting sensitive information and demonstrate a serverless approach for handling large-scale tasks using Amazon Bedrock Data Automation (BDA), AWS Step Functions, and AWS Lambda. The workflow for serverless PII redaction is illustrated in Figure 1.
Figure 1 depicts an end-to-end process for removing personally identifiable information. The use of generative AI in document comprehension allows for the removal of limitations associated with conventional redaction techniques.
