| Northrop Grumman Mission Systems has developed a problem-solving methodology that automates the process of identifying unknown unknowns. These are factors affecting a decision that are not known to exist at the beginning of the evaluation of a situation. The toughest problem in decision making in any field is coping with unknown unknowns. Analysts with lots of time, well-structured data, and broad domain expertise appear to manage this task reasonably well. Students discover, assimilate, and learn to reason about entirely new and previously unexpected domains of knowledge. For computers, this automated reasoning capability is more difficult. In fact, until now it has been an unresolved research problem.
We developed an overall approach to automate the discovery of unknown unknowns. We used automated reasoning and learning algorithms separately and in combination to process the content of a knowledge base. Our solution leveraged extremely rich knowledge structures. We used this knowledge representation to specify what is known to a knowledge base. Differences between what was specified as known, and what new evidence suggested, were computed. These differences, dubbed unknown unknowns, were of three types: new hypotheses that might explain a situation, new links that explain previously unknown relationships between facts, and new story fragments that describe the significance of previously unidentified collections of factors that might drive a decision. The basic approach enabled us to prototype many different methods for the automated discovery of unknown unknowns, depending on the characteristics of the decisions. We successfully demonstrated these methods in three domains: strategic planning and analysis, information operations (contractually), and anti-terrorism.
|
| Patrick J. Talbot has extensive experience with Northrop Grumman, most of which is in Independent Research and Development efforts. He is currently the Chief Technologist at the Colorado Springs Engineering Organization. He has performed on Minuteman, spacecraft projects, Battle Management tasks, Anti-Satellite systems, and the development of decision algorithms. In addition, he has managed a Delivery Order titled “Technology Insertion Studies and Analysis” at the Joint National Test Facility in Colorado Springs, Colorado, where his research included parallel computing, simulated human behavior, and strategic planning. He holds Bachelors of Science degrees in Applied Mathematics and Physics from Penn State and a Masters of Science degree in Physics from Penn State. |