Login
By 2025, IDC projects that unstructured data will account for 80% of the data collected globally.
For those of you who know what it takes to wrangle unstructured data, that figure might seem terrifying. And in fact, Dell reports that nearly half of IT decision-makers fear their infrastructure won’t handle it.
But technology is closing the gap. In particular, advances in artificial intelligence (AI) are making it possible to overcome a seemingly insurmountable challenge — automating the processing of documents containing unstructured data. This was a focus at the SIFMA Operations Conference earlier this month. The theme for the event was "get your operations future ready" and many attendees were evaluating new solutions and technologies geared towards driving greater operational efficiency.
In this article, we’ll explore how AI is transforming processing unstructured data and what it takes to turn AI from a buzzword into a powerful tool for your business.
What is unstructured data?
Unstructured data is information without a predefined format or organization, such as text, image, videos, and audio.
Unstructured data formats we commonly encounter in financial services include emails, faxes, PDFs, and scanned documents. These formats are ubiquitous throughout the industry, from fund administrators and custodians to buy-side and sell-side firms.
The challenge of unstructured document processing
Unstructured data is much harder to process and utilize than structured data, which arrives in predictable, standardized formats that make automation easy. We face three main issues in processing unstructured documents at scale:
Variability and lack of standardization
Unstructured data comes in diverse formats. It’s hard to find a single solution that can handle everything from images to PDFs to faxes with the required level of accuracy.
Lack of contextual understanding
You may be able to extract data with optical character recognition (OCR), but without contextual understanding, it’s still just data. Your OCR tool can’t tell whether it’s looking at a contract or an invoice. It can’t trigger an automated process the way we can easily do with structured data.
Handling multiple languages and scripts
In the financial services industry, we need to be able to handle unstructured data in a wide variety of languages and scripts. And if you don’t have contextual understanding in English, you certainly don’t have it in Japanese, Hebrew, Mandarin, and so on.
How AI transforms unstructured document processing
Fortunately, we finally have a solution to these challenges. It’s everyone’s favorite buzzword: AI.
AI can automatically process any unstructured document — no need to manually train your model on each new format. With AI’s cognitive capabilities that provide confidence levels, you can go beyond extraction to analysis and only surface documents for manual review when necessary.
Why AI is the right tool for the job
Over the past few years, AI has gone from being a specialized tool for data scientists to a universal technology with a wide range of business applications. Here’s why:
- AI is trained on a wide range of data, much wider than traditional machine learning (ML) models developed for narrowly scoped use cases. That means AI can support a larger variety of use cases.
- AI has become much easier to use with the rise of tools and applications designed for business (i.e., non-technical) users. You can now put AI in the hands of people who know the business best (while freeing up your technical teams for more complex tasks).
- Scalability, interoperability, and flexibility are table stakes for today’s AI applications. You never have to worry about transaction volumes or whether a tool or model will work with your other systems.
AI isn’t automation in itself — but it fast-tracks your ability to derive insights from unstructured data and put it to use in automated processes.
AI in practice
Here are just a few examples of how financial services organizations are using AI to accelerate documenting processing workflows.
Trade confirmations
Xceptor clients process hundreds of trade confirmations daily across multiple brokers and asset classes, through multiple channels, sources, and formats. Reconciling details from the confirmation against the booking system takes a lot of slow, error-prone manual work.
AI can extract key data fields such as trade date and currency and provides a confidence level for each. You can trigger manual review if confidence falls below a certain threshold. The AI also extracts and summarizes key clauses, making it much easier to consume and review large documents.
Loan notices
Agent notices often come through emails and are different for each agent bank. Though all loan notices require the same transactional details, layouts vary based on the line items or the loan notice type.
Previously, operations team members had to read each email, view the attachments, make determinations, and manually compare the data to internal systems. AI can automate this entire process, augmented with human review as needed.
Invoices
Similarly to loan notices, every vendor submits invoices in a unique format, though each invoice must contain certain fields (such as invoice number, date, value, and tax). AI can automate this process to reduce the manual burden on operations team members, as well as the likelihood of errors.
Best practices for implementing AI
What you need to succeed
AI is a powerful capability, but you need certain elements in place for it to be successful:
- A strong orchestration layer helps you manage complex workflows end to end, including handoffs between AI, people, and systems.
- Control is non-negotiable. Automation, validation, and workflow capabilities go beyond simple "human in the loop" review to allow management of business actions and outcomes.
- The old adage "garbage in, garbage out" still applies. Effective data management ensures quality and accuracy.
3 steps for AI success
- Identify use cases and focus on control up front. Getting your organization comfortable with AI requires winning over compliance. Often, the best initial use cases focus on risk reduction, such as finding and routing customer emails or finding and flagging clauses that can’t be accepted.
- Get the business involved early on. If we’ve learned one thing from digital transformation, it’s that engaging the business drives the best results. This is even more important with AI. Start off strong with tools that provide a non-technical interface and appropriate controls so the business can quickly iterate on AI use cases.
- Don’t get hung up on individual AI components. No AI tool is a silver bullet. You may need to combine multiple models, as well as traditional approaches, for the best outcome. Disappointing early results can be demotivating, so encourage stakeholders not to fixate on the output of a single AI component in isolation. Instead, focus on experimentation and learning.
