New algorithm enables effective machine learning with symmetric data | MIT News

If you rotate an image of the molecular structure, humans can tell that the rotating image is still the same molecule, but the machine learning model may think it is a new data point. In computer science, the molecule is “symmetric”, which means that the molecule’s basic structure remains unchanged when undergoing certain transformations, such as rotations.

If the drug discovery model does not understand symmetry, inaccurate predictions of molecular properties may be made. However, despite some empirical success, it is not clear whether there is a computationally effective way to train a good model that respects symmetry.

A new study by MIT researchers answers the question and demonstrates the first approach to machine learning with symmetry that is very effective in both the amount of computation and data required.

These results shed light on a fundamental problem that they could help researchers develop more powerful machine learning models designed to deal with symmetry. From discovering new materials to identifying astronomical anomalies to revealing complex climate patterns, such models will be useful in a variety of applications.

“These symmetry is important because they are some kind of information that nature is telling us about data, and we should take it into account in machine learning models. We have now shown that machine learning can be effectively used with symmetric data,” said Behrooz Tahmasebi, a graduate student at MIT and a study author of the study.

Co-leader writer and MIT graduate student Ashkan Soleymani joined him on paper; Stefanie Jegelka, associate professor of electrical engineering and computer science, a member of the Institute of Data, Systems and Society (IDSS) and the Laboratory of Computer Science and Artificial Intelligence (CSAIL); Dugald C. Jackson, professor of electrical engineering and computer science, principal investigator in the Laboratory of Information and Decision Systems (LIDS), and senior writer Patrick Jaillet. The study was recently presented at the International Machine Learning Conference.

Study symmetry

Symmetric data appear in many fields, especially natural sciences and physics. For example, a model that recognizes symmetry can recognize an object, such as a car, wherever the object places it in an image.

Unless a machine learning model designed to handle symmetry, it may be less accurate and prone to failure when dealing with new symmetric data in reality. On the other hand, models that utilize symmetry may be faster and require less training data.

However, training a model to process symmetric data is not easy.

A common approach called data augmentation, where researchers convert each symmetric data point into multiple data points to help the model generalize better to new data. For example, one can rotate the molecular structure multiple times to produce new training data, but if the researchers want to ensure that the model respects symmetry, they can be overefficient in computation.

Another approach is to encode symmetry into the architecture of the model. A well-known example is a graphical neural network (GNN), which inherently processes symmetric data due to the way it is designed.

“Graphic neural networks are both fast and efficient, and they all take good care of symmetry, but no one really knows that these models are learning or what they do. Understanding GNNS is the main motivation for our work, so we do theoretical evaluations from what happens when data is symmetrical,” Tahmasebi said.

They explored statistical computational tradeoffs in machine learning using symmetric data. This trade-off means that methods that require less data are computationally more expensive, so researchers need to find the right balance.

Based on this theoretical evaluation, the researchers designed an efficient algorithm for machine learning using symmetric data.

Mathematical combinations

To do this, they borrowed ideas from algebra to narrow down and simplify the problem. They then re-formed the problem using the idea of effectively capturing the geometric shapes of symmetry.

Finally, they combined algebra and geometry into an optimization problem that could be solved effectively, resulting in a new algorithm.

“Most theories and applications focus on algebra or geometry. Here we just combine them together,” Tamasaibi said.

The algorithm requires fewer training data samples than the classical method, which will improve the accuracy of the model and the ability to adapt to new applications.

By proving that scientists can develop efficient machine learning algorithms through symmetry and demonstrate how it is done, these results may lead to the development of new neural network architectures that may be more accurate and resource-intensive than current models.

Scientists can also use this analysis as a starting point to examine the internal operation of GNNs and how their operations differ from algorithms developed by MIT researchers.

“Once we know, we can design more explainable, more powerful, and more efficient neural network architectures,” Soleymani added.

The research is funded in part by the Singapore National Research Foundation, the Singapore National Laboratory, the U.S. Naval Research Office, the National Science Foundation and Professor Alexander von Humboldt.