List of Research Projects

Description: The exploration-exploitation trade-off is the central problem in decision making under uncertainty and reinforcement learning. When faced with uncertainties in its environment, an agent may either choose to learn the consequences of their actions to get a better outcome in the future or avert the risk of failing by committing to actions they believe to work just enough. A major challenge in reinforcements learning is to design algorithms balancing these two competing strategies to maximize the rewards the agent collects over time. There are a number of approaches (e.g. upper confidence bound method) proposed to balance exploration and exploitation. However, these methods are mostly limited by computational constraints and therefore are not feasible for real world applications. In our research project, we aim to design, implement and analyze computationally efficient and feasible reinforcement learning algorithms balancing exploration and exploitation. We will study computational complexity aspects (e.g. time and sample complexity) of reinforcement learning algorithms and investigate their performance in several simulated control and decision making problems.

Required Knowledge: Basic knowledge of and experience in programming (preferably in Python) is useful. It would also be helpful to have some basic knowledge of probability, linear algebra and calculus although not required.

Special Notes: Note that the mentor for this project would be entirely remote.

Description: Modern day control systems such as autonomous driving often use image and video signals to produce feedback control. Generally. This is called perception-based control in control theory and robotics literature. There are now well-known examples whereby adding adversarial noise, which can not be detected by human eyes can result in awfully wrong image label prediction. Many works have addressed this issue of adversarial robustness in the supervised learning/prediction type of tasks. But there is very little work that understand what adversarial robustness means for perception-based control. This project aims at surveying, understanding, categorizing and well as comparing the different ways of perception-based control in terms of their robustness to perception perturbation, from both theoretical guarantees to simulation comparisons.

Required Knowledge: Must feel comfortable with linear algebra, proof-based mathematics. Have a working knowledge of control dynamical systems and python is a plus.

Description: The heart of a rocket engine is the injector: it introduces and mixes the fuel(s) and oxidizer(s) into the chamber, and is the root of many failure modes on an engine, from maldistribution to pressure instabilities. For this reason, ensuring an injector operates as intended is critical. Early in this process a common technique used is hydro-patternation, where instead of the often dangerous propellants an engine would use, water is flowed through the device and performance characteristics (flow rates, mixing patterns, and stability) can be estimated. Since the actual pressure drop across the injector is usually only ~5%-20% of the combustion pressure, the water pressure is quite low and safe; it is literally rocket science you could do with just your garden hose! The project for the FSRI students would consist of design of elements of a test measurement setup for injector articles, with pressure and flow rate measurement capabilities and a high speed camera to capture the injector behavior. Furthermore, if students have time and desire they can design and water test their own injector, with guidance from myself towards which are most approachable at this level.

Required Knowledge: No hard pre-requisites are needed, but experience in basic fluid mechanics (AP Physics 1 or equivalent) will unlock more possibilities for independent development of components of the test stand and injectors. Furthermore, experience with hands-on building (FSAE, FRC, etc.) will be helpful for the assembly of the test rig. However, the most important aspect would likely be interest in rocketry, aerospace, or fluid mechanics!

Description: Application of machine learning to mathematical reasoning. Study and work with the Lean theorem prover. Experiments with other logic proof systems. Several small projects are possible.

Required Knowledge: Familiarity with programming (Python or C/C++) and Git.

Description: The goal of this research project is to give students hands on experience in STEM curriculum building by (1) building/designing one electrochemistry lab module as part of a 3-unit electrochemistry lab course to be taught in Spring 2024 (2) doing literature research on effective learning experiments to teach core concepts and apply effective learning tactics, and (3) learn and practice research relevant electrochemistry techniques to answer fundamental chemistry questions. Although the project is more focused on educational research and teaching rather than a typical ““research”” focused project with the goal to discover and understand new science, this project would be suitable for anyone interested in a career in teaching and in learning a breadth of fundamental electrochemistry techniques (that would be useful for future research experiences). By the end of the assignment, students will have a tangible product, have developed some teaching pedagogy, will gain experience being in a TA-like position, and gain recognition for developing this course.

Required Knowledge: Some lab experience in chemistry is useful but not necessary. Enthusiasm for effective teaching. Interest in learning experimental electrochemistry.

Description: We would like to perform a systematic study to evaluate whether blood vessel network images acquired by candling 5-days old eggs (candling simply refers to shining a strong light beam through the egg) can be used in combination with a deep learning network to predict embryo gender. We believe this hypothesis is worth testing because deep learning has previously been demonstrated to be capable of highly accurate gender identification when applied on human retinal images of the blood vessel network (~90% over several independent studies) – these subtle pattern differences are remarkably beyond our human ability to observe/describe and is a strong indication that there is a link between gender and the way blood vessel networks are organized.

Required Knowledge: No pre-requisites.

Description: This project will involve analyzing eye tracking and choice data. Subjects in this experiment were extensively trained on the value of a subset of a large stimulus set. Then they made chose between two stimuli either from the hyper trained stimuli or the regular exposure stimuli. The goal is to understand the differences in choice and visual search patterns between the two training conditions. We will begin with reading two papers that show how what one looks at changes choice behavior and identify key figures. Students will be provided with cleaned eye tracking data and asked to reproduce those figures. They will also identify other questions specific to this dataset as a group, discuss how to operationalize them and make figures for those. If there is time I will walk them through the statistical analyses checking for effects in the figures they have produced and familiarize them with how to code up the experiment that collected the data.

Required Knowledge: Familiarity with or willingness to learn Python or R visualization packages

Description: We will investigate how COVID19 affected the performance of high school athletes in the United States. We will leverage the abundance of data online by developing Python code that processes the data and helps us answer questions like: Were certain areas of the country affected more than others, for example, in colder climates where outdoor athletics were not feasible year-round? Did elite athletes maintain the same level of performance while mid-tier athletes saw declines? How did the various public school shut-down policies affect the athletes? Have the performances seen improvement following reopening and what performances can we expect this upcoming year? Did the event type (eg sprinters, jumpers, throwers, or distance) matter for the results?

Required Knowledge: Prior course in statistics

Description: Synthetic biologists combine engineering principles with biology to manipulate and understand biological systems. An increasingly popular tool in the synthetic biology toolbox is synthetic cells. These cells are typically compartments formed by lipid bilayers and containing a solution of interest, and they can range in size from 1 to 100 micrometers. Synthetic cells are used to explore the origins of life on Earth, to observe biological processes in simplified settings, and for real-world applications, such as targeted delivery of anti-cancer drugs. Although synthetic cells can perform basic functions like transcription and translation (TX-TL), they currently lack the ability to perform complex computations like living cells. This project will focus on programming synthetic cells with DNA strand displacement circuits for complex computation. Specifically, this project will focus on (1) determining if the system is compatible with existing TX-TL systems and (2) if the system can be easily scaled for computation of even greater complexity.

Required Knowledge: An introductory knowledge of biology is necessary, although AP Biology (or equivalent) is not needed. Previous lab experience is also not necessary, but students must be motivated, willing to learn, and capable of maintaining a detailed lab notebook. Experience with programming in Python is a plus.

Description: Learning mappings between function spaces, such as solving differential equations, often involves discretizing infinite-dimensional spaces into finite-dimensional grids, where the input is a coefficient function and the output is a solution function, and standard learning models such as neural networks can be applied. But the learned neural network model may not generalize well to different discretizations, beyond the discretization grid of the training data. To overcome this, we may use a recently developed deep-learning framework called neural operators, which directly maps between function spaces on bounded domains. Because neural operator is designed on function spaces, they can be discretized by a variety of different methods, and at different levels of resolution, without the need for re-training. Neural operator has broad applications in science and engineering where learning mappings between function spaces are crucial

Required Knowledge: Basic skills in Python, familiar with Linux.

Description: Flowing fluids may seem disordered, but from them emerges highly organized coherent structures, like vortices in the ocean and atmosphere or at the tip of airplane wings. This complex movement - turbulence - is one of the most hotly researched topics in applied physics and engineering. Understanding how ordered structures arise within chaotic flowing fluids is highly relevant to a wide range of disciplines, ranging from aircraft design to oceanography. In this program, students will handle numerical turbulent flow data; they will learn tools to visualize and process this data with the goal of answering the following questions: can you use learning algorithms to predict the course of a chaotic flow? And in case these fail, what other tools can be used to analyze a complex fluidic system?

Required Knowledge: Some familiarity with coding and plotting would be helpful, though most of code will be provided.

Description: The blood-brain barrier (BBB) serves as a significant hurdle in delivering treatments to the brain. Adeno-associated viruses (AAVs) have become a preferred tool in this area because of their safety, specificity, and ability to transduce various cell types. However, current versions of AAVs, such as PHP.B, PHP.eB and 9P31 engineered in rodents, don’t work effectively in non-human primates and face difficulties in clinical therapy due to differences in receptor binding among species. In response, I employed a different strategy, developing a new method to evolve AAVs to specifically bind to human BBB cellular receptors carbonic anhydrase IV (CA4; Car4). FSRI students will be involved in this work to select new AAV variants that can bind human CA4 and do individual validation to characterize their in vitro and in vivo performance.

Required Knowledge: Cloning technique, Cell culture experience,

Description: We will start with the characteristic differential equations describing enzyme kinetic systems. Then we shall identify the rate constants by comparing the differential equations to data from the literature. We plan to use recently collected data for the main protease of COVID-19.

Required Knowledge: None. We shall teach what is needed.

Description: Quantum transduction converts quantum bits (qubits) between microwave and optical frequencies, for large-scale quantum networks. In the Faraon lab, we fabricate and measure chips with the aim of making an efficient transducer. To this end, we need a variety of optical and electronic equipment with dedicated, easy-to-use control. We have a variety of projects mostly involving instrumentation and experiments, including: coding an application to interface with lab equipment, to verify the status of the on-going experiments; making an optical setup to fiber-couple a Fabry-Perot cavity, useful as a filter; integration of an optical switch into a setup and coding an interface for remote control (among others, depending on interest).

Required Knowledge: Just enthusiasm and willingness to get your hands dirty! The projects will be focused on coding/setups, and less about theory. Some coding background will be helpful for the coding projects, but not required.

Description: Firefighters currently use handheld anemometers to determine wind conditions during wildfires. But, these devices provide limited information about the overall wind conditions, which can make it difficult to assess how the fire is likely to spread. Therefore, we propose to use computer vision to estimate wind conditions using a single camera. This approach would leverage the fluid-structure interactions of vegetation with the wind to provide a coarse map of the wind in real time. This would allow firefighters to make better decisions about how to fight the fire and prevent it from spreading.

Required Knowledge: Coding in matlab or python

Special Notes: Note that the mentor for this project would be remote the last few weeks.

Description: Autonomous systems are commonly placed in unknown and unstructured environments and are expected to not fail in completing their tasks. The notion of safety in systems can be interpreted as having undesirable things not happen or good things happen. For an autonomous ground robot, this could be not colliding with obstacles or navigating towards a desired target. A common tool to encode safety in a system are control barrier functions for which several constructive techniques have been developed.

Required Knowledge: Student should have some experience in: Coding in C++, Python, or Arduino; designing using CAD programs such as Solidworks; working with electronic components.

Description: How do living systems self-stabilize their own dynamics? Our work has focused on macroscopic self-organized waves of greenness, linking biological ‘clocks’, light and morphogenesis in a photosynthetic unicellular alga, Caulerpa. The complex morphology exhibited in this organism strikingly resembles differentiated organs in multicellular plants. Our findings tie the discovered waves to one of the mysteries of development in single-cell organisms, morphogenesis. (Afik et al. 2023) Project goals are to 1. develop a system for reactive spatio-temporal illumination, infer dynamics of organismal morphology from time-lapse imaging, set the illumination state to depend on live-feedback from the algal samples. This makes the light part of the dynamical system, to study emergent modes of adaptation and anticipation. To this end will program a Raspberry Pi computer to control and coordinate (i) a DLP mini-projector, and (ii) a camera imaging the living samples. In this way different regions of the single cell will be exposed to various temporal illumination protocols. For image segmentation we will compare several approaches, Active Contours Without Edges, Random Forests Classifier, and Segment Anything Model (SAM). Finally, programming a dynamical model, we will work towards coupling the illumination state to that of the organism. In our manuscript, ‘Dynamical States of Self-Organized Waves in a Giant Single-Celled Organism Feeding on Light’, we report measurements and analysis of macroscopic self-organized waves, linking biological ‘clocks’, light and morphogenesis in a photosynthetic unicellular alga, Caulerpa. The complex morphology exhibited in this organism strikingly resembles differentiated organs in multicellular plants. Our findings tie the discovered waves to one of the mysteries of development in single-cell organisms, morphogenesis.

Required Knowledge: The project is at the interface between Physics and Biology. Great candidates would be those who are self-motivated and independent, passionate about living systems, having programming experience and physical thinking. Experience with the scientific Python programming ecosystem, image processing and linux is an advantage.

Special Notes: Note that the mentor for this project would be remote the last few weeks.

Description: The embryonic development of the chicken is scientifically and commercially an important. In the poultry industry the males are mostly culled after hatching, which is not only an economic loss but is also an animal welfare issue. In collaboration with Professor Changhuei Yang’s Biophotonics lab at Caltech, we aim to develop tools to image eggs and then use artificial intelligence to detect male embryos automatically.

Required Knowledge: Good biology background and some lab experience with working with animals.