Google made an AR microscope that can help detect cancer

In a talk given today at the American Association for Cancer Research's annual meeting, Google researchers described a prototype of an augmented reality microscope that could be used to help physicians diagnose patients. When pathologists are analyzing biological tissue to see if there are signs of cancer -- and if so, how much and what kind -- the process can be quite time-consuming. And it's a practice that Google thinks could benefit from deep learning tools. But in many places, adopting AI technology isn't feasible. The company, however, believes this microscope could allow groups with limited funds, such as small labs and clinics, or developing countries to benefit from these tools in a simple, easy-to-use manner. Google says the scope could "possibly help accelerate and democratize the adoption of deep learning tools for pathologists around the world."

The microscope is an ordinary light microscope, the kind used by pathologists worldwide. Google just tweaked it a little in order to introduce AI technology and augmented reality. First, neural networks are trained to detect cancer cells in images of human tissue. Then, after a slide with human tissue is placed under the modified microscope, the same image a person sees through the scope's eyepieces is fed into a computer. AI algorithms then detect cancer cells in the tissue, which the system then outlines in the image seen through the eyepieces (see image above). It's all done in real time and works quickly enough that it's still effective when a pathologist moves a slide to look at a new section of tissue.

So far, the Google team has demonstrated the scope's abilities with breast and prostate cancers, showing the system is pretty accurate in its assessments. But Google says such a set up could be used to detect other cancers as well as infectious diseases like tuberculosis and malaria. It could also be used to display other data or annotations into the view. "Of course, light microscopes have proven useful in many industries other than pathology, and we believe the Augmented Reality Microscope can be adapted for a broad range of applications across healthcare, life sciences research and material science," says Google.

This proof of concept study is also described in a paper that's currently still under review. Google says that a more in-depth study is required for more robust assessments of the system's performance and shortcomings. It concluded, "We're excited to continue to explore how the Augmented Reality Microscope can help accelerate the adoption of machine learning for positive impact around the world."