Auto Document Classification

Document classification is the first and the most important step of the document and data capture processes. Prior to extraction, the classification of the document has to be done because different types of documents normally have different extraction fields. Once the classification has been successfully carried out, the document type-specific field extraction can be started.

There are several approaches and methods to pass this step automatically. There is no one “the best” approach for now.  AI (Machine learning methods) tools are more popular than rule-based systems for the sales and marketing teams these days.

While the rule-based systems mostly used for a limited number (less than 1000) of document types such as insurance, banking, operational documents and forms, AI tools is used for unlimited and unstructured documents (not the forms) such as emails or social media shares.

AI tools must be trained with hundreds of thousands of documents. Machine learning methods needs at least 1000 true and 1000 false pre-classified samples for each type of documents. The neural network is created with the help of these training sets. If the system is trained incorrectly, you have to start from scratch which is almost impossible to find out the miss-classified documents. On the other hand, there are premade neural networks for AI tools to classifies emails and spam blockers. You have to accept some error rate for AI systems.

CaptureFast’s well-known and successful auto-classification system is a rule-based system. Adding a document type into the system is less than 10 minutes. Although it is extremely easy to use, it is quite advanced and accurate. The accuracy target is %100 for CaptureFast Automatic Classification system.

Capturefast’s new hybrid (AI + Rule) auto-classification system!

The existing rule-based system almost perfect for a capture system. But it still needs some human touch to set it up. On the other hand, when the number of document types increases, more time needed to fine-tune the classification for each document type. CaptureFast engineers decided to add an AI layer for the existing system.

The workflow of the new hybrid system is not changed. But how?

The rule-based classification system uses a DEF table for each document type which must be created by an analyst. Here is a very simple DEF table ingredient of a loan document.

IDPOINT~180

OBJECT~200~0~(0,0,250,250)~image-234

BARCODE~200~0~(0,0,1000,250)~reg exp

ID~200~0~(0,0,1000,330)~”UNIFORM RESIDENTIAL LOAN APPLICATION”~95~reg exp

ID~100~0~(0,0,1000,250)~”AGENCY”~100~ [A,B,C][0-9]{10}

ID~100~0~(0,0,1000,1000)~”LENDER”~100~reg exp

ID~100~0~(0,0,1000,1000)~”PURPOSEOFLOAN”~97~reg exp

ID~100~0~(0,0,1000,1000)~”ORIGINALCOST”~98~ ^-?(?:0|[1-9]\d{0,2}(?:,?\d{3})*)(?:\.\d+)?$