Anthony Simeonov

I am a Ph.D. student in the EECS department at MIT, advised by Professor Alberto Rodriguez and Professor Pulkit Agrawal. I'm interested in robot manipulation and physical interaction. I am fortunate to be supported by the NSF GRFP.

I received my S.M. in EECS from MIT and my B.S. in Mechanical Engineering from UC San Diego, where I worked with Professor Michael Yip. I have also spent time as a research intern at Disney Research Los Angeles and NVIDIA's Seattle Robotics Lab

Email  /  Google Scholar  /  LinkedIn

profile photo
Research

I'm currently interested in machine learning for perception-driven planning and control, applied to problems in robot manipulation and physical interaction. The goal is to equip robots with reusable manipulation skills, through a combination of effective perceptual representations and reactive behaviors. Previously, I have studied the design, modeling, and control of inherently compliant artificial muscle actuators.

Publications

From Imitation to Refinement - Residual RL for Precise Visual Assembly
Lars Ankile, Anthony Simeonov, Idan Shenfeld, Marcel Torne, Pulkit Agrawal
arXiv, 2024
project page

A BC + RL pipeline for precise manipulation tasks like assembly. Fine-tuning BC-trained policies with action chunking and diffusion de-noising requires specialized RL methods. We simplify such RL fine-tuning with simple residual policies trained with PPO.

Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta*, Pulkit Agrawal*(*equal advising)
RSS, 2024
project page

Learning robust manipulation policies by combining real-world imitation learning and reinforcement learning in simulation. RL in sim uses digital twin assets that reflect the geometry, appearance, and kinematics of the target deployment scene.

JUICER: Data-Efficient Imitation Learning for Robotic Assembly
Lars Ankile, Anthony Simeonov, Idan Shenfeld, Pulkit Agrawal
IROS, 2024
project page

A pipeline for learning image-based policies for precise, long-horizon assembly tasks from a small number of demonstrations by combining expressive policy architectures and various techniques for dataset expansion

Lifelong Robot Learning with Human Assisted Language Planners
Meenal Parakh*, Alisha Fong*, Anthony Simeonov, Tao Chen, Abhishek Gupta, Pulkit Agrawal(*equal contribution)
ICRA, 2024
project page

An LLM-based task planner that can learn new skills opens doors for continual learning.

Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement
Anthony Simeonov, Ankit Goyal*, Lucas Manuelli*, Lin Yen-Chen, Alina Sarmiento, Alberto Rodriguez, Pulkit Agrawal**, Dieter Fox**(*equal contribution, **equal advising)
CoRL, 2023
project page / code

Performing 6-DoF relational rearrangement between objects and scenes. Iterative pose de-noising via diffusion helps handle multi-modality and local scene cropping improves generalization.

Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation
Ethan Chun, Yilun Du, Anthony Simeonov, Tomás Lozano-Pérez, Leslie Kaelbling
ICRA, 2023
project page / code

Local point cloud encoders help 3D neural descriptor fields generalize to more diverse global shapes and improve robustness to occlusions

SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields
Anthony Simeonov*, Yilun Du*, Lin Yen-Chen, Alberto Rodriguez, Leslie P. Kaelbling, Tomás Lozano-Pérez Pulkit Agrawal (*equal contribution)
CoRL, 2022
project page / code

Applying 3D neural descriptor fields in a pairwise fashion, to enable relational rearrangement with pairs of unseen objects in arbitrary poses.

MIRA: Mental Imagery for Robotic Affordances
Lin Yen-Chen, Pete Florence, Andy Zeng, Johnathon T. Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez, Phillip Isola
CoRL, 2022
project page

NeRF enables synthesizing orthographic views from virtual camera poses, allowing search for 6-DoF placing actions represented by pixels in the rendered image (positions) and the camera pose used to generate the image (orientations)

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Anthony Simeonov*, Yilun Du*, Andrea Tagliasacchi, Joshua Tenenbaum, Alberto Rodriguez Pulkit Agrawal**, Vincent Sitzmann** (*equal contribution, order determined by coin flip. **equal advising)
ICRA, 2022
project page / code

A novel representation that models objects as 3D neural fields of descriptors. We apply this representation to enable pick-and-place on unseen objects in out-of-distribution poses from a small handful of demonstrations.

A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
Anthony Simeonov, Yilun Du, Beomjoon Kim, Francois Hogan, Joshua Tenenbaum, Pulkit Agrawal, Alberto Rodriguez
CoRL, 2020
project page

A framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation

Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Planners
Ahmed Qureshi, Yinglong Miao, Anthony Simeonov, Michael C. Yip
TRO, 2020
project page

By encoding raw observations of obstacle geometry into a latent representation, we can distill the capabilities of an expensive planning algorithm into a neural network, allowing significant speedups in finding near-optimal collision-free paths.

Motion Planning Networks
Ahmed Qureshi, Anthony Simeonov, Mayur J. Bency, Michael C. Yip
ICRA, 2019
project page

This paper is subsumed by our journal paper.

Bundled super-coiled polymer artificial muscles: Design, characterization, and modeling
Anthony Simeonov, Taylor Henderson, Zixuan Lan, Guhan Sundar, Adam Factor, Jun Zhang, Michael C. Yip
RA-L, 2018

Empirical study of the performance tradeoffs between different bundling techniques for thermally-driven, compliant artificial muscle actuators.

Stickman: Towards a Human Scale Acrobatic Robot
Morgan T. Pope, Steven Christensen, David Christensen, Anthony Simeonov, Grant Imahara, Günter Niemeyer
ICRA, 2018

We present a two DOF robot that is launched from a gravity-driven pendulum and executes a variety of mid-air stunts. The robot uses a combination of onboard sensors to perform state estimation and inform actuator timing.

This research is part of the basis of Imagineering's Stuntronics project.

Three-dimensional hysteresis compensation enhances accuracy of robotic artificial muscles
Jun Zhang, Anthony Simeonov, Michael C. Yip
Smart Materials and Structures, 2018

A new modeling scheme for capturing the coupled hysteresis between tension, strain, and input, which is used for improved control of artificial muscle actuators.

Modeling and inverse compensation of hysteresis in supercoiled polymer artificial muscles
Jun Zhang, Kaushik Iyer, Anthony Simeonov, Michael C. Yip
RA-L, 2017

We propose three new models for characterizing the hysteresis present in super-coiled polymer artificial muscles, and demonstrate their use for accurate open-loop control.

Workshop Papers

Learning to Plan with Pointcloud Affordances for General-Purpose Dexterous Manipulation
Anthony Simeonov, Yilun Du, Beomjoon Kim, Francois Hogan, Pulkit Agrawal, Alberto Rodriguez,
RSS 2020 Workshops on Visual Learning and Reasoning for Robotic Manipulation, and Learning in Task and Motion Planning


template