Merge design and computer science in a creative way | MIT News

The speed at which new technologies can be put on the market is no help compared to the speed at which talented researchers find creative ways to use them, train them, and even turn them into something we can’t do without. One of the researchers is a graduate student at MAD Alexander HTET Kyaw, a graduate student who studies a dual master’s degree in architecture studies in the fields of electrical engineering and computer science.

Kyaw takes technologies such as artificial intelligence, augmented reality and robotics, and combines them with gesture, speech and object recognition to create human workflows that have the potential to interact with our built environment, change the way we shop, design complex structures and make physical physical things.

One of his latest innovations is Curator AI, who he and his MIT graduate student partners won first prizes in the AI ​​build of the MIT AI conference – $26,000 in OpenAI products and cash – generated voice AI Solutions, MET’s Weekly Hackathon, the final speech in New York City last fall. Working with Kyaw are Richa Gupta (architecture) and Bradley Bunch, Nidhish Sagar and Michael Won, all from the Department of Electrical Engineering and Computer Science (EECS) of MIT.

Curator AI aims to simplify online furniture shopping by using AI and AR to provide context-aware product advice. The platform uses AR to occupy the size of the room, with windows, doors and locations of existing furniture. Users can then talk to the software to describe the new furniture they want, and the system will use visual AI models to search and display various options to match the user’s prompts and visual characteristics of the room.

“Shoppers can choose from the recommended options to visualize products in AR and search modifications using natural language requirements, making the furniture selection process more intuitive, efficient and personalized,” Kyaw said. “The problem we are trying to solve is that most people don’t know where to start when providing a room, so we developed curator AI to provide intelligent, contextual advice based on the appearance of your room.” Although curator AI is developed for furniture shopping, it can be expanded in other markets.

Another example of Kyaw’s work is estimates, which he and three other graduate students created at a hackathon at the MIT Sloan Product Technology Conference in March 2024. The competition focused on helping small businesses; Kyaw and Team decided to base their work on a Cambridge-based painting company that employs 10 people. Estimates use AR and object recognition AI technologies to make exact measurements of rooms and generate detailed cost estimates for renovation and/or paint work. It also utilizes generated AI to display images of rooms or rooms as they look like them after painting or renovation and generate invoices after the project is completed.

The team won the hackathon and $5,000 in cash. Kyaw’s teammates are Guillaume Allegre, May Khine and Anna Mathy, who graduated from MIT in 2024 with a Master of Business Analytics.

In April, Kyaw will give a TEDX speech at his alma mater, Cornell University, where he will describe curator AI, estimation and other projects that use AI, AR and robotics to design and build things.

One of these projects is unjoined, and Kyaw connects AR with gesture recognition to build a software that maps the dimensions of building components from inputs from fingertips on the surface of materials and even the air. That’s how unlog is – a towering art sculpture made of gray logs on Cornell’s campus.

Unlog indicates the possibility of structures that can be constructed directly from the entire log, rather than letting the log spread to the wood mill to turn into a plank or two by four wood and then shipped to a wholesaler or retailer. This is a good representation of Kyaw’s desire to use building materials in a more sustainable way. A paper on this work, “Feedback-based mixed reality manufacturing gesture recognition, a case study of Unlog Towers”, published by Kyaw, Leslie Lok, Lawson Spencer and Sasa Zivkovic at the 5th International Conference on Computational Design and Robot Manufacturing in January 2024.

Another system from Kyaw developed integrated physics simulation, gesture recognition and AR to design active bent structures built with bamboo poles. Gesture recognition allows users to manipulate digital bamboo modules in AR and integrates physical simulations to visualize how bamboo bends and connect the position of bamboo poles in a way that creates stable structures. This work appears in the August 2023 record of the 41st Education and Research Conference in Computer-Assisted Architecture Design in Europe, “One Bend in Physics-Based Mixed Reality: Design and Manufacturing of Reconfigurable Modular Bamboo Systems”.

Kyaw last year used the Bamboo module to propose a similar idea to create a deployable structure that is Mitdesignx, a MIT MAD program that selects promising startups and provides coaching and funding to roll out them. Since then, Kyaw founded Bendshelters to build prefabricated, modular bamboo shelters and community spaces for refugees and displaced people in his native Myanmar.

“Where I grew up in Myanmar, I saw the daily impact of climate change and extreme poverty.” “There is a huge refugee crisis in the country and I want to think about how to contribute to the community.”

MIT Sandbox, PKG Social Innovation Challenge and Amazon Robots’ Social Commodity Award were recognized.

At MIT, Kyaw collaborated with Professor Neil Gershenfeld, Director of the Center for Location and Atoms, and PhD student Miana Smith, using voice recognition, 3D Generative AI and robotic arms to create a workflow that can build objects in an accessible, boarding and sustainable way. Kyaw holds a bachelor’s degree in architecture and computer science from Cornell University. Last year, he received a SJA scholarship from the Steve Jobs Archive, which funded the intersection of technology and art.

“I love exploring various technologies to design and make things,” Kay said. “Being a part of the madness has made me think about how all my work is connected and helped me articulate my intentions. My research vision is to design and develop systems and products that can interact naturally between humans, machines and the world around me.”