Technology Provider: Automated Insights is a provider of natural language generation platform, Wordsmith, that converts big data into narratives or summary understandable by humans
User Company: Associated Press is a multinational, not-for-profit news cooperative.
Industry: Media
Application: Business intelligence and analytics, Process automation
Problem
Associated Press (AP) reporters spent a significant amount of time and effort to manually glean insights from quarterly financial reports released by public companies in the US. This involved extracting and analyzing relevant financial data, such as profit, revenue growth, tax expenses, etc., and converting them to financial recaps (news narrative or prose summary).
Owing to limited time and manual resources, AP reporters produced only 300 such articles every quarter, which left out thousands of potential companies that published their quarterly corporate earnings.
Actions Taken
AP employed Automated Insights’ natural language generation (NLG) platform, Wordsmith, to auto-summarize the quarterly financial recaps. This NLG platform was configured to write according to the editorial standards of AP. The configuration process involved feeding AP’s editorial rules and relevant financial data into the platform and tweaking the system to adjust the expected output summary over time.
According to Laura Pressman, an Automated Insights spokesperson, the initial configuration of Wordsmith for AP involved some hard work. Zacks Investment Research in partnership with AP provided the financial data, which served as the input data file to be fed to Wordsmith. Automated Insights then transformed this data file according to its standardized template and guidelines, which are rule-based instructions that Wordsmith uses to conjure the output autosummary. These guidelines also included the AP style guide.
The GIF below shows the structure of the AP input data to and the expected output summary from Wordsmith.
Below are some sample screenshots of said rule-based instructions, which Wordsmith uses to produce the output autosummary:
According to Laura, the Wordsmith configuration period took about two months, during which the AP reporters worked in tandem with the Automated Insights team to monitor the errors produced by Wordsmith and continually tweak the system to minimize errors and improve the overall performance. Automated Insights claims that the AP outputs are now completely automated although AP writers can add quotes, context, and opinions.
The GIF below demonstrates the capabilities of Wordsmith, including the ability to feed the required data for narratives, produce auto-summary based on certain rules and input data, and show previews of the output.
The summary output can be exported as CSV or JSON files and stored as on-premise data or it can be exported to the cloud, such as the Google Drive, via an API.
It should be noted that at the time of this implementation, Automated Insights only had their managed solutions offering. The company now offers a self-serve version of this solution, which presumably doesn’t involve the same kind of implementation and hand-holding as we see in this case study.
Results
According to Automated Insights, the number of published financial recaps at AP rose from 300 to 4,400 per quarter, resulting in a 12-fold increase. The company claims that while its NLG platform hasn’t yet displaced any reporters, it has freed up the equivalent of three full-time employees across AP. It also reports that 20% of employee time that was spent on manual production of financial recaps has been “freed up.”
A National Public Radio article compared the writing styles of an AP reporter and Wordsmith configured according to AP guidelines, and the time it took for both to produce the summary articles. The article claims that the AP reporter finished the summary in seven minutes, whereas Wordsmith produced the output in two minutes.
Here are the outputs according to the NPR article (Story #1 is from Wordsmith; Story #2 is from the AP reporter):
Although the NPR article mentions that Wordsmith took about two minutes to produce the output in this scenario, Laura claims that it took lesser time for Wordsmith to actually write the article. Much of those two minutes reportedly involved the data entry required for Wordsmith to produce the output.
According to Laura, Wordsmith can reportedly process massive amounts of data to produce any number of articles in seconds, once the system has been configured.
Transferable Ideas
While automated journalism is a rather unique AI use case, there are a number of lessons from this AI implementation that can be important lessons for business leaders.
For a task to be automated by AI, the boundaries of the task must be definable. Writing unique political commentary about complex international affairs topics is unlikely to be a task that AI can conquer in the near future. The necessary context for such writing involves a knowledge of historical events, a psychological assessment of global leaders, and the ability to connect disparate events and actions to formulate new ideas. This kind of work is challenging for machines.
In the case of Associated Press, some of their writing involved far less context. With a uniform type of data, and a prescribed kind of “output” (in this case, article), the task was conquer-able by machines.
It’s also important to note the investment time required to get these systems running. It’s nearly impossible for human beings to set enough structures and rules to consistently product a desired result from an AI system immediately.
In the case of AP, the company collaborated with Automated Insights over two initial months – finding and addressing errors in the machine’s output, tweaking the process gradually to achieve a consistent, quality result. Any business aiming to automate a process with AI should expect some degree of similar training and adjustment.
