Atom2Vec - Stanford AI recreates chemistry’s periodic table of elements

June 25, 2018
Stanford AI recreates chemistry’s periodic table of elements

In a first step toward generating an artificial intelligence program that can find new laws of nature, a Stanford team created a program that reproduced a complex human discovery – the periodic table.

https://news.stanford.edu/2018/06/25/ai-recreates-chemistrys-periodic-table-elements/

By Ker Than

It took nearly a century of trial and error for human scientists to organize the periodic table of elements, arguably one of the greatest scientific achievements in chemistry, into its current form.
Periodic table and brain

A Stanford team has developed an artificial intelligence program that recreated the period table of elements; they aim to harness that tool to discover and design new materials. (Image credit: Claire Scully)

A new artificial intelligence (AI) program developed by Stanford physicists accomplished the same feat in just a few hours.

Called Atom2Vec, the program successfully learned to distinguish between different atoms after analyzing a list of chemical compound names from an online database. The unsupervised AI then used concepts borrowed from the field of natural language processing – in particular, the idea that the properties of words can be understood by looking at other words surrounding them – to cluster the elements according to their chemical properties.

“We wanted to know whether an AI can be smart enough to discover the periodic table on its own, and our team showed that it can,” said study leader Shoucheng Zhang, the J. G. Jackson and C. J. Wood Professor of Physics at Stanford’s School of Humanities and Sciences.

Zhang says the research, published in the July 25 issue of Proceedings of the National Academy of Sciences, is an important first step toward a more ambitious goal of his, which is designing a replacement to the Turing test – the current gold standard for gauging machine intelligence.

In order for an AI to pass the Turing test, it must be capable of responding to written questions in ways that are indistinguishable from a human. But Zhang thinks the test is flawed because it is subjective. “Humans are the product of evolution and our minds are cluttered with all sorts of irrationalities. For an AI to pass the Turing test, it would need to reproduce all of our human irrationalities,” Zhang said. “That’s very difficult to do, and not a particularly good use of programmers’ time.”

Zhang would instead like to propose a new benchmark of machine intelligence. “We want to see if we can design an AI that can beat humans in discovering a new law of nature,” he said. “But in order to do that, we first have to test whether our AI can make some of the greatest discoveries already made by humans.”

By recreating the periodic table of elements, Atom2Vec has achieved this secondary goal, Zhang says.

Potassium is to king as …

Zhang and his group modeled Atom2Vec on an AI program that Google engineers created to parse natural language. Called Word2Vec, the language AI works by converting words into numerical codes, or vectors. By analyzing the vectors, the AI can estimate the probability of a word appearing in a text given the co-occurrence of other words.

For example, the word “king” is often accompanied by “queen,” and “man” by “woman.” Thus, the mathematical vector of “king” might be translated roughly as “king = a queen minus a woman plus a man.”

“We can apply the same idea to atoms,” Zhang said. “Instead of feeding in all of the words and sentences from a collection of texts, we fed Atom2Vec all the known chemical compounds, such as NaCl, KCl, H20, and so on.”

From this sparse data, the AI program figured out, for example, that potassium (K) and sodium (Na) must have similar properties because both elements can bind with chlorine (Cl). “Just like king and queen are similar, potassium and sodium are similar,” Zhang said.

Zhang hopes that in the future, scientists can harness Atom2Vec’s knowledge to discover and design new materials. “For this project, the AI program was unsupervised, but you could imagine giving it a goal and directing it to find, for example, a material that is highly efficient at converting sunlight to energy,” Zhang said.

His team is already at work on version 2.0 of their AI program, which will focus on cracking an intractable problem in medical research: designing just the right antibody to attack antigens – molecules capable of inducing an immune response – that are specific to cancer cells. Currently, one of the most promising approaches to curing cancer is cancer immunotherapy, which involves harnessing the antibodies that can attack antigens on cancer cells.

But the human body can produce more than 10 million unique antibodies, each of which is made up of a different combination of about 50 genes. “If we can map these building block genes onto a mathematical vector, then we can organize all antibodies into something similar to a periodic table,” Zhang says. “Then, if you discover that one antibody is effective against an antigen but is toxic, you can look within the same family for another antibody that is just as effective but less toxic.”

Other Stanford co-authors include graduate students Quan Zhou and Shenxiu Liu, and postdoctoral researcher Peizhe Tang. Funding for the research was provided by the U.S. Department of Energy.

Found a link to the Atom2Vec journal article:

Learning atoms for materials discovery
Quan Zhou, Peizhe Tang, Shenxiu Liu, Jinbo Pan, Qimin Yan, and Shou-Cheng Zhang
PNAS June 26, 2018. 201801181; published ahead of print June 26, 2018. https://doi.org/10.1073/pnas.1801181115

   Contributed by Shou-C
http://www.pnas.org/content/early/2018/06/25/1801181115

Summary and Outlook.

We introduce unsupervised learning of atoms from a database of known existing materials and show the rediscovery of the periodic table by AI. The learned feature vectors not only capture well the similarities and properties of atoms in a vector space, but also show their superior effectiveness over simple empirical descriptors when used in ML problems for materials science. While empirical descriptors are usually designed specifically for a task, our learned vectors from unsupervised learning should be general enough to be applied to many cases. We anticipate their effectiveness and broad applicability can greatly boost the data-driven approaches in today’s materials science, especially for the recently proposed deep neural network methods (29⇓⇓–32), the same as the huge success of word vectors in language modeling (22, 23, 33, 34). Several directions related to the feature learning methods here are worthy to explore in the future. For example, the element–environment matrix can be generalized to a higher-order tensor, where the extra orders depict different parts of the composition. Such a tensor should contain finer information than the matrix, and how to extract features from this high-order object is still an open question. Also, more reasonable environment descriptions are necessary for improvement in both model-free and model-based methods. Structural information has to be taken into account to accurately model how atoms are bound together to form either environment or compound, where the recent development on recursive and graph-based neural networks (30, 32, 34) might help.

PNAS wrote:Atom2Vec workflow to learn atoms from the materials database. Atom–environment pairs are generated for every compound in the materials database, based on which atom–environment matrix is constructed. A small dataset of seven compounds is used here as an example. Entries of the atom–environment matrix denote the numbers of atom–environment pairs. Inset shows the unit cell of compound Bi2Se3 and the pair corresponding to the entry of target atom Bi and environment (2)Se3. Only compositional information is considered, while structural information is ignored. Atom2Vec learning algorithms extract knowledge of atoms from the atom–environment matrix and encode learned properties in atom vectors.

MBL Version:
https://www.nevyns-lab.com/mathis/app/mbl/mbl.html?mbl=Bi2Se3&align=X&atom=nucleus

This .pdf is pretty good as well:

S1. Statistics of compound data
There are 60605 inorganic compounds in total in Materials Project database[1]. Because environments with too many atom types usually depends on structures heavily, they do not help and even impair atom learning if only chemical formulas are taken into account. Hence, as mentioned in Method, only binary, ternary and quaternary compounds are used to learn atom vectors in this work. The population of each symbol (equivalently, the number of environments with this symbol as target atom) is also examined. It is found that there is a sharp population drop between common and rare symbols. A threshold, 1% of the maximal population which is located near the drop, is selected to filter out rare symbols. Only atom-environment pairs of common symbols (about 100) are used to learn atom vectors.

http://www.pnas.org/highwire/filestream/816160/field_highwire_adjunct_files/0/pnas.1801181115.sapp.pdf