AI Document Processing: Extract Useful Data with Review and Traceable Evidence

An invoice arrives from the supplier, a colleague examines it, and then a different person types it into the screen of an accounting system. This seems like a straightforward procedure until the invoice contains a barely legible decimal point, two purchase orders, or a table that stretches across multiple pages. A computer program can process this task faster than people can. A computer program could even pass an error to the next computer program.

The first step of any intelligent document processing system should thus be the following common sense question: what sort of decision-making can be trusted when it is based on the data from the document? Assigning a supplier's name to a review queue and assigning a bank account number or releasing payments are two different processes involving some pieces of text with quite different consequences.

This is a guide to intelligent document processing planning for business operations. Invoices will serve as an illustration because they present various issues: inconsistent layout, unclear fields, repeated submission, and approval. This kind of planning can be used for other documents as well, although privacy, retention, and business rules will have to be known for those specifically. The goal is to create usable records with traceability, not a screen that looks like it knows what it is doing.

Start with one document and one useful result

The first document must undergo a well-established procedure in the initial phase. For example, the system creates the first draft of the invoice for an accounts reviewer. The aim is to reduce manual retyping while showing where each proposed value came from. The system doesn't have to decide anything about whether the supplier is legitimate, whether the goods were delivered, or whether the payment is right.

State the result in terms of the person executing the task. The reviewer will have to check the number against the source page, change it, and proceed to the next step without having to use four different applications. This task formulation reveals the integration and interface requirements that cannot be discovered with the help of a demo of text extraction.

Create a representative document collection

Do not test the whole process exclusively on a set of PDF samples that you have prepared especially for the demonstration. Collect a set of examples allowed to use that are similar to everyday input documents: scans, photos, multipage invoices, rotated pages, unusual layouts, and documents with empty fields. Use both easy and hard samples. A collection of hard cases only is just as biased as the set of easy ones.

Record reasons for significance in each sample. Perhaps one vendor has the purchase order number adjacent to the delivery address, while another has the number in a footnote. Hand-written changes on scanned images could occur. Omit or mask any confidential data according to needs of the project. Keep the source collection separate from samples for further analysis, so that successive refinement does not covertly transform a test into a practice exercise.

Separation of reading and interpretation

Optical character recognition identifies text in the document. Several approaches to extraction can recognize tables, fields with names, or couples of labels and values. It is explained in Amazon’s Textract documentation. This is useful when evaluating the services, but this does not guarantee that the particular invoice will be identified correctly.

Keep the extracted text and proposed business fields as distinct items. In the event that the system suggests an invoice date, keep the original text used and the interpretation rules. The person who follows up may realize that it's actually the delivery date. It's far easier to correct the mistake when the different layers are kept separated, rather than only having the value and a green tick at the end.

Defining fields before selecting prompts

Agree on the meaning of every field. The invoice number might have letters, slashes and zeroes in front. Converting it to a numeric field would lose some information in terms of the identity of the item. The invoice date does not need to be the same as the service date. A total may contain or exclude tax, or be a balance forwarded. Define these things in the accounts department before the developers get started.

Define also how the system must behave in the case of missing field. An empty field, an uncertain value, and a certain value represent three different states. If all empty fields are filled with a guess, then one will get a clean database but the process will be unreliable. A document without purchase order number might need a review and not an invented purchase order number.

Keep evidence from the sources for every proposal

There should not be a necessity to go through a long PDF to understand the reason behind the decision made by the system. One should store sufficient information to find the source of this proposal: name of the document, page number, surrounding text, and also a visual region if possible. Present this evidence near the proposed field.

Another way evidence helps is in the case where a customer disputes a record after several weeks have passed. This allows the support team to recognize the original document, the machine proposal, and the reviewer's acceptance value. The process history should be easy to understand without looking at the raw output of the model. It is especially useful in cases where an extraction setup has changed, as old documents still contain evidence of the process used.

Confidence is the right review signal

Confidence is not a guarantee of the correctness of a field. Instead, it should guide the review process which has already been validated against the business' document set. Even a high confidence rating on the wrong date label will still be the wrong date. Compare confidence against the results of fields you know about and see how overconfident or overly careful your extraction setup is.

Treatment can vary between fields. Search descriptions will allow uncertainty where an amount associated with a transaction will not allow that luxury. Work out the validation policy together with the operator of the downstream process. Begin with a conservative approach; note any changes made by the reviewer and revise the thresholds based on the results. Do not simply use the threshold provided in an example from one of your vendors and apply it everywhere.

Check meaning as well as format

It is important to perform format checks, but it is not enough. Dates might be formatted correctly yet still be completely unrealistic. Totals can be numeric yet contradict line item totals. Supplier references can be correct and still refer to a completely different supplier within your company's system. Meaning must be checked as well as format.

Do not cover up the problems of validation; make sure that they are visible and can be solved. If there is a conflict between the total sum and particular elements, show the difference and sort this record based on a certain rule. You cannot invent the balancing item for the sake of bypassing one of the rules in the database. The reviewing process should identify whether the error occurred during the extraction, was in the document itself, or in the submitted data.

Be intentional about duplicates

There are legitimate reasons for resending documents. Someone may never have responded to an email, an upload attempt might have failed, or the supplier might have sent a new version of the same document. The mere fact of sending the same document again does not create a second payable. Define unique document and business identifiers that allow the process to distinguish resubmissions.

Comparison of an original document copy and a revised version of the document. Whereas a duplicate may involve an invoice with similar references but differing in amounts, requiring further checking before being discarded, the application should maintain the relationship between the documents and present this information to the checking authority. There should be reasons why the application considers the document duplicate, and the authorized user should have a way of clarifying this issue.

Untrusted input

A submitted document can include commands, links, or anything else that would not have any rights or powers over the application itself. In case if a language model is used to interpret a submitted document, its output must be treated as a suggestion within a controlled workflow. The text in the PDF must not be able to change the permissions, choose any destination, or instruct the application to give out any money or customer data.

Ensure that the extraction of the documents is limited to those documents permitted by the extraction policy. Check the output against the expected document format, and validate any business requirements other than what has been modeled. Writing to the downstream systems should be done based on defined application policies and controlled access. Just because the document has been obtained from a known vendor does not necessarily make all the instructions on the document true.

Design a review queue for people to work on

The queue needs to clearly identify what needs to be done and why. Arrange the fields by the documents that support them, allow finding issues in progress easily, and keep the changes performed while the reviewer browses the pages. The reviewer should see whether the record is pending extraction, review, export approval, or has some problems with integration. These are separate tasks assigned to separate users.

Do not require the reviewer to approve each and every field individually if your process does not demand it. Nor should you use an approval button that masks the controversial amount. Test your system using real documents and distractions. The user returning from a phone call must have a clear understanding of what was approved, what is not yet known and if any of his colleagues took action.

Linking approval to an export

Data extraction and data extraction approval are not necessarily two events that happen together. The record can be ready for approval but not for export. The reviewer should find the conditions under which the integration writes to the accounting, CRM or operational systems. He should note the values being approved and the user approving them.

The export needs its own result. In case if the destination does not accept the supplier reference, the document has to be traceable, and there will be no need to begin from zero. Give the operation a stable identity so that a retry after a timeout does not create a duplicate record. Compare the local state of the export operation with the real result on the destination side in case if the result is ambiguous.

Document protection during its lifecycle

Access should be provided according to the purpose of the document. The user who is allowed to upload the document will not necessarily require access to each and every processed document. The needs of the reviewers, the support and administration staff can be quite different. The access should be extended to the original file, extracted text, preview, and export of the document.

Agree on retention and deletion policies with the business before production. Temporary processing documents, evidence documents, and final documents can have different requirements. Logging complete document content to save time can ruin a tightly controlled document repository. Ensure diagnostic operations pinpoint the faulty document and process without logging the document content or screenshots of it in a place where they are widely visible.

Prepare for processing delays and failure

Document processing can be asynchronous. The user submits a document but receives a result at a later stage. Provide useful status information and don't lead the user to think their document submission is gone just because the extraction is still going on. Define appropriate retry policy for temporary failures and distinguish temporary failure from unsupported documents or those that require resubmission by the user.

Design constraints need to be included in the design. File size, number of pages, computing power and reviewing power all have impacts on the user experience. It is not addressing the bottleneck if a queue takes out the documents very quickly, leaving reviewers with an unworkable backlog of documents. Track the age of the outstanding work and let the team know how you respond when the demand goes beyond the operating capacity.

Evaluating the fields and workflow independently

Field level performance needs to be measured when evaluating the extraction process. The success rate for documents does not address a system that accurately captures supplier names but fails repeatedly at capturing totals. Compare the proposed values with reference values to track what types of errors occur within the process. The samples used for evaluation need to reflect present submissions.

And then assess the workflow. Think about review time, number of corrections, number of exceptions that are not resolved, duplicate exporting, and number of records getting into the destination successfully. Each of these metrics answers its own question. Decreased number of keystrokes does not necessarily mean decreased total effort required, especially in case the review interface adds investigation, and the integration needs assistance in resolving unclear exports.

Cost of review and continuous change

Costs of utilizing AI for documents include much more than just the extraction API. Storage costs of the document, the cost of preview generation, integration development, review effort, and evaluation costs have to be estimated as well. Compare these costs with the costs of your current process and value of the result you need. Small and clear workflow can bring more benefits than complicated solution.

Be prepared for any changes in the document structure and business logic. The invoice form of your supplier may be changed, while your accounting department will require implementation of yet another field into the reference. You have to make sure that somebody is responsible for all changes and the pipeline will be checked before being released. Versions of your configuration will be linked to process history allowing you to track the change without guessing what exactly was used when you performed your check.

Prepare acceptance checks before the pilot

Choose strong checks for the pilot run. The reviewer will be able to see where the total number comes from, correct the date if needed, cope with the problem of duplicates, and recover from export failure. All the decisions made by the reviewer must be remembered after the page refresh. There has to be a document impossible to process.

Specify what the pilot does not allow you to do. An invoice drafting record does not imply permission for spending approval or any changes to the vendor's bank account details. Make sure that these limitations are clearly communicated to the employees as well as other parties involved. A narrowly scoped pilot project will help to identify what type of automation and decision-making needs a responsible person, which is a good foundation for the next version of the application.

How Digitonix can help in document processing

An AI solutions service from Digitonix can be taken into account in scoping an extraction workflow, whereas software development is about linking the fields suggested to the business rules and existing applications. Initiate the discussion of your project with the relevant permitted samples of documents, field definitions, review criteria, and destination of the data. Those aspects are more useful than requesting an AI dashboard with no accuracy target.

The review platform may need Python programming for handling services and React for handling source comparison and exceptions. The appropriate architecture depends on the workflow and limitations of the team. Please reach out to Digitonix using the Digitonix contacts and ask for the scope that includes evaluation criteria, integration ownership, security scope, and the ones that need to be passed before the production implementation. See React interface for more information.

Frequently asked questions

Is it possible for AI document processing to replace manual review completely?

Everything depends on what document set we have, what domain we operate in, and what could go wrong because of the mistake. Do not assume that the successful extraction will allow you to automate everything else. Start with a review process, based on the business goal, analyze the changes, and then go to automation. There will always be some scenarios when human intervention is necessary.

Is invoice data extraction the same as invoice approval?

No. Extraction extracts information from a document. Approval checks whether the company agrees to accept a document or approve some other action. Distinguishing between these two actions will help to prevent confusion between a readable invoice and acceptance of products, the trustworthiness of the supplier, or approval to pay.

Which documents should be included in the pilot?

Inclusion of valid examples according to the actual process including normal documents and interesting exceptions is needed. Interesting exceptions may include changes in layout, faint scanned documents, multiple-page tables, duplicates, and lack of references. Each example should be explained in the documentation and the separate evaluation set should be prepared if possible. A pilot should show common operation difficulties rather than the best looking documents.

How should extraction accuracy be evaluated?

Compare the values of fields proposed against the values of fields referenced in reviews and note down any errors field-wise and document-wise. The second metric which needs consideration is the one which relates to the process itself and includes the efforts put into review process, exceptions and export quality. One number can obscure many issues. Determine what is an acceptable outcome with all stakeholders of the business process and test again when layouts, rules, providers and configurations are changed.

What do we need to have in place before we sit down for the discussion of our project?

We need to have an overview of the current process, permitted sample documents for evaluation, list of fields needed, destination systems, and decisions requiring approval. We also need to have the understanding of access and retention requirements, load profile, and possible exceptions. This information will help us create a realistic first release and test scenario.