Cogito Interview: Daniel Lee, Bioengineer

Cogito members submitted questions to Dr. Lee from March 5 through March 18. Scroll down and read his answers.

I’m Daniel Lee, and I’m on the faculty in the Electrical and Systems Engineering department at the University of Pennsylvania, where I teach and do research. I’m also a member of the GRASP Lab at Penn, which stands for General Robotics, Automation, Sensing and Perception. 

So what does that mean? It means that I get to do some pretty cool stuff, in my classroom and in my lab. For example, I teach a class where electrical engineering students learn about embedded computing, which are like the sensors and processors in your cell phone and microwave, and then use their knowledge to create some interesting devices that work in the real world. One group of students made a trash can with a motion-controlled lid that would not open when it sensed the trash can was full. It was called Intellitrash. Another group made a remote-controlled sports car called the Nissan Sportster where the user could control the speed, flashers, horn, and steering wheel. Some other students made a xylophone that plays itself.

Daniel Lee, PhD

Hometown: Houghton, Michigan (the Upper Peninsula)
Education: PhD in Condensed Matter Physics from MIT, 1995
AB in Physics, Harvard, 1990
Research Interests: Computational neuroscience, machine learning, robotics
Free-time Favorites: ice skating (and eventually playing ice hockey) with my 5 year-old son, Jordan and 3 year-old daughter, Jessica



In my own research, I’m trying to apply what we know about how humans process sensory information such as vision and language, to build better artificial sensorimotor systems that can adapt and learn from experience. This field of machine learning is a combination of artificial intelligence, computer science, and robotics. I began this work while I was at Bell Laboratories, the research and development arm of Lucent Technologies, where I was a researcher in the Theoretical Physics and Biological Computation departments. While there I built some fun robots such as a dog that can walk around, recognize faces and “look” at you when you call it. I also built a robot that could roll down the hallway on its own, and transmit photos of what it was “seeing” back to the lab. It could then speak using speech synthesis to people outside of my office.

I arrived at Penn five years ago, and one of my projects has been to lead a soccer four-legged robot team, the UPennalizers, in the annual Robocup competition. We program a team of Sony Aibo dogs to compete in soccer games and travel around the world for competitions. Our best victory was in Italy where we placed second to an Australian team ( Currently I am involved in a big competition called the DARPA Urban Grand Challenge, where we are programming a car to drive itself and self-navigate on a city road, in the presence of other moving cars ( (You may have heard about the Lexus that can parallel park by itself — that’s a piece of cake!)

I’m on Cogito for the next couple of weeks to take any questions you have about my work in particular and about engineering in general. Ask away.

How did a physics guy like you end up in engineering?

These days, the distinctions between fields such as physics, chemistry, biology, and related engineering fields have blurred quite a bit. A lot of interesting research has emerged from taking ideas and concepts from an established area, and applying them to new problems in other fields. I enjoy thinking about how mathematical models and concepts that I learned as a physics student can be applied to understanding the brain, or building intelligence into machines.

After graduate school, I did a postdoc at Bell Labs where there was a lot of cross-fertilization of ideas between physicists and other scientists and engineers. So when I looked for academic faculty positions, I didn’t limit myself to only physics departments. I especially like how I can think about both theoretical models as well as doing practical experiments in engineering, something that is harder to do in physics where there is more of a dichotomy between theory and experiment.

“Computational neuroscience” sounds interesting. Do you work with researchers from this field on your projects? How does this field influence how you build robotic devices?

I especially like talking to neuroscientists who are trying to understand the brain as a computational system. For my research, the brain is proof of principle that an intelligent robotic device is possible. The hard part is trying to figure out how the brain really works! Just as the Wright brothers analyzed bird flight to understand the importance of ailerons and lateral stability in flight control, I would like to use ideas from neuroscience to build robots.

How do you tackle such a big project like programming a car to drive itself?

Very cautiously and carefully :-). One aspect of science and engineering is being able to work with teams of researchers, and focusing them on a common problem and goal. This is one aspect of research that is not really taught in science grad school.

What are some other projects your students have worked on other than Intellitrash, the xylophone, and the remote-controlled sports car?

The neat thing about teaching is that students always surprise you with cool ideas and projects that you would have never thought of yourself. One group made a digital sundial—they took a traditional sundial and made a digital readout for it using photodiodes. I thought this was a nice combination of a very old idea (sundial) with modern technology.

I am an 8-year-old boy and I think you’ve done some pretty cool stuff. How do you make robots though?

Robotics is a combination of many different disciplines: mechanical engineering, computer science, electrical engineering, etc. But it’s pretty easy to start—when I was a student at MIT, I just started building robots with LEGO’s. From those simple robots, I gradually began building more complex and sophisticated systems like a robotic car. So as in most things, one just has to simply start and learn as one goes.

You said you made a robotic dog that could look at you and recognize faces. That’s interesting, but what’s the point of that? How would it be used?

This research is not only applicable to robots, but also to making better human-computer interfaces. I really don’t like typing on my computer, fighting with Microsoft software trying to make it do some simple things. Someday, I would like to be able to communicate with my computer, just as I do another person: talking, smiling, gesturing, etc.

Sounds like there’s so much neat stuff being made but none of it is on the market. Why? Why can’t we buy an Intellitrash can yet? What does it take to bring something like that to the market?

That’s a good question, and something that many companies struggle with. A lot of times, very good ideas in research do not immediately become commercial products and successes. There are many factors with bringing something to market—timing, venture capital, economic conditions, liability issues, etc. If you ask people in startup companies, they would also say that plain luck has a big part as well. So as you say, it does seem that a lot of neat stuff does not immediately make it to market.

How do you balance working and your home life and kids?

It is very difficult to balance that (and unfortunately, it would still probably be more difficult to do if I were a woman). Having tenure in academia helps—I can freely set some of my priorities without having to worry so much about the consequences. But I do have to constantly remind myself what I feel are the most important things in my life.

Do you feel like your kids give you any insights into how the brain works, or how artificial intelligence should work?

It gives me a lot of humility. I’m amazed at how fast kids learn new concepts and skills—that truly puts my robots to shame. If we could emulate even a small fraction of how well children learn and adapt, that would be an amazing breakthrough in artificial intelligence. Unfortunately, having kids does not necessarily mean that you understand them…

What other job would you like to do besides this kind of engineering?

When I was a kid, I thought it would be neat to be a professional hockey player. Unfortunately, my competitive playing days are long past. But the reason I chose this area of research is that I found it fascinating, and I really enjoy working on it. No matter what you do, you should have a job that you find interesting. And as long as I feel that way about engineering, I’ll continue to teach and do research there.

Leave a Comment

Your email address will not be published. Required fields are marked *