Inventor: Dr Botty Dimanov, Computer Lab
AI (artificial intelligence) or machine-based learning is now in widespread use, being used in areas as diverse as the development of self-driving cars, image, speech and sound recognition, improving search algorithms, and searching for new drug molecules. The key to the method is that the programmer develops training algorithms and allows the AI to “learn” by providing data and feedback loops. The software uses the data to populate and evolve a node-based neural network. The limitation is that an algorithm is only as good as the data it is trained on – using low quality or confusing data leads to a low-quality AI. An AI is also constrained by the task it has been asked to carry out – it will have no capacity to come up with “new ideas” relating to the input data.
Despite the widespread use of AI techniques, there are currently no methods to test what the AI has learned other than to see how it reacts to sample data or new situations. In areas such as self-driving cars or medical diagnoses, a badly-trained AI could have a direct impact on human life, so AI safety is becoming an increasingly important issue. This is the focus of Dr Dimanov’s research.
Dr Dimanov has developed software algorithms which identify what a computer-based neural network focuses on when presented with new input, allowing the programmer to see what the AI used as the basis for a particular decision. He is also able to observe individual nodes within a neural network and see what they are doing. This allows the computation of what inputs will have the maximum influence on that particular node.
In other words, his software can help the programmer understand what a neural network means by an individual concept such as “hair”, and how it has built that up.
The aims are twofold – firstly to identify what an AI has actually learnt about a particular concept, and then to alter the programming principles, or provide more targeted data, to correct or enhance that learning.
The challenge for the i-Team is to look at the diverse uses of machine learning algorithms to identify where a tool of this kind would have the greatest impact.