Shubham ShrivastavaTechnical Expert and Lead - 3D Perception @Ford AutonomyIEEE Senior Member
Stanford UniversityGraduate Program - Artificial Intelligence
GPA: 4+/4.0Courses in Machine Learning, Meta-Learning, Multi-Task Learning, Deep Generative Models, Natural Language Processing, Computer Vision, and 3D Reconstruction
August 2020 - Present
The University of Texas at ArlingtonMaster of Science in Electrical Engineering
August 2014 - August 2016
Visvesvaraya Technological UniversityBachelor of Engineering in Electronics and Communication Engineering
GPA: 4.0/4.0, First Class with Distinction, Aggregate Percentage: 86%
August 2014 - August 2016
Keynotes and Media Coverage
Keynote talk @ Auto.AI USA 2023, delving into the dynamic world of vision-centric perception algorithms, breakthroughs, and bridging the academia-industry gap. 🌍✨ A key message to the research community: It's not just about building models that can do more, but rather empowering them to do "More with Less."
we.CONECT interviews Shubham Shrivastava on the landscape of vision-centric perception for autonomous vehicles.
[Keynote Talk @In.Cabin Sensing Europe] From the outside to the inside – Rethinking the implications of autonomous driving for the communication with the driver (link)
[Panel Discussion @In.Cabin Sensing Europe] On the way to smart cabin – How to find the balance between in-cabin sensing, communication, privacy and user experience? (link)
Ford Greenfield Labs, Palo Alto, CA
Leading a team of talented machine learning and robotics engineers towards building vision-centric 3D perception solutions at Ford Autonomy. My work includes building an end-to-end 3D Perception stack for Ford L2+ vehicles on the road, and the development of a flexible and scalable machine-learning framework for all ML tasks within Ford.
My research includes computer vision and advanced machine learning methods including convolutional neural networks, generative adversarial networks, variational autoencoders, and 3D perception; with significant emphasis on object detection, semantics learning, 3D scene understanding, multi-view geometry, and visual odometry.
In addition to researching and developing novel methodology, I built the complete cloud-based MLOps pipeline including intelligent data sourcing, auto-annotation, training, model optimization, and deployment (TensorRT C++) for putting our prediction engine in production.
Two major projects for which I built end-to-end perception pipelines are (1) Ford Autonomous Shuttles (2) Ford Factory of the Future - Infrastructure-based autonomous vehicle marshaling through assembly plants.
[Topics of Research]
Monocular RGB camera and LiDAR-based 3D object detection, Classification, and Tracking in both indoor and outdoor environments.
Unsupervised and Semi-Supervised Object 6-DoF Pose Estimation to reduce the cost of manual data annotation from millions of dollars down to zero.
Multi-headed multi-task neural networks for scene understanding incorporating Sim2Real methods for zero-cost training of the networks.
Generative Adversarial Networks for realistic image generation with semantic and cycle consistency from simulation data to fill the gap between both worlds.
A combination of computer vision and traditional methods including non-linear optimization for object pose estimation.
Automated global localization of multiple spatially distributed sensors within the infrastructure to a common coordinate frame using an autonomous robot.
Perception system for localizing objects of various classes to within 10 centimeters and sub-degrees orientation accuracy within Ford’s Factory of the Future.
Renesas Electronics America, Inc.Applications Engineer, Perception R&D - ADAS and Autonomous Driving
March 2017 - September 2019
I worked as part of a very small team towards building the ADAS & Autonomous Driving perception reference platform “Perception Quick Start” which includes end-to-end solutions for camera and LiDAR based road feature and object detection.
Developed complete lane detection pipeline from scratch. Pipeline includes lane pixel extraction using a combination of classical computer vision method and deep learning, lane detection, polyfit, noise suppression, lane tracking, lane smoothing, confidence computation, lane extrapolation, lane departure warning, lane offset, lane curvature, and lane types.
Developed C based Computer Vision Library for basic image processing functions like image read/write, hough transforms, edge detection (canny, horizontal, vertical), colorspace conversions, image filtering (sharpen, gaussian smooth, sobel, emboss, edge). Created advanced math library for functions like least-squares polyfit and matrix operations.
Stereo Camera calibration, rectification, disparity map generation, flat road free-space estimation, Object Detection using V-Disparity and 3D density-based clustering, 3D point cloud rendering along with 3D bounding-box, and depth perception.
Developed dynamic image ROI stabilization module for correcting rotation and translation using angular pose/velocity data by computing and applying homography at run-time. Developed a general-purpose positioning driver for bringing in GNSS/IMU data into the perception stack.
Optimized embedded implementation of algorithms for parallel computing on R-Car SoC HW Accelerators.
Developed the complete V2V Solution from Scratch for the Renesas’ V2X Platform including CAN Framework, GPS/INS driver, GPS+IMU fusion for localization, Concise Path History computation, Path Prediction, CSV and KML logging module, 360-degree lane-level Target Classification, Basic Safety Applications, and an HMI using QT for displaying Warnings, Vehicle Tracking, Maps, and Debug Information.
Changan US Research & Development Center, Inc.Intelligent Vehicle Engineer (Connected Autonomous Vehicle Research Group)
August 2016 - March 2017
Worked within Changan's Connected and Autonomous Vehicle research team to design and develop various vehicle safety models for 360-degrees target classification and warnings notifications with and without line-of-sight requirements.
DELPHI (now known as APTIV)Embedded Software Engineer (Intern)
May 2016 - August 2016
Worked with application teams, forward systems algorithm group, and controller design groups to define the functionality, develop algorithms, and implement them in accordance with the V-Model Software Development Life Cycle.
BlackBerry QNXSoftware Development Intern (Board Support Package)
January 2016 - May 2016
Developed the BSP (Board Support Package) for custom hardware with i.MX6 Solo Processor and several peripherals. Worked towards low-level board bring-ups and provided support for the following peripherals.
Support for RAM file system to manipulate files during runtime.
Support for SPI NOR Flash and Parallel NOR Flash mounted as a filesystem at startup.
Support for Removable storage (SD, microSD, USB Flash). Also, provided support for its auto-detection and auto mounting at the attachment.
Support for USB OTG to be used for the Console Service, Mass-Storage Device, and USB-to-Ethernet Adapter attachments.
Added new features to the QNX OS for the BSP including auto-detection and switching between device stack to provide console service, host stack to provide auto-mounting of mass storage devices, and host stack to provide networking with USB-Ethernet Adapter based on the type of attachment.
The University of Texas at Arlington Research InstituteResearch Intern
August 2015 - December 2015
Designed and Developed the control GUI for a prosthetic system used to help rehabilitate post-stroke patients. It used an Arduino controller for adaptive adjustment of the air bubble pressure at the desired psi value for various points on the leg. Also designed a GUI which allows the user to enter the desired psi values for each air bubble and simultaneously measure current bubble pressure and display it on the GUI in real-time.
Used two Arduino UNO, one for sending the signals to 32 solenoids controlling airflow from Alicat Mass Flow Controller into the respective air bubbles, and one for receiving sense signal from 32 corresponding air pressure sensor. Signals were also sent to the Solenoids for deflating the air bubbles when required.
Indian Institute of Science (IISc)Trainee Engineer
January 2014 - May 2014
Designed and developed a 2 Dimensional plotter (Smart XY Plotter) at the Mechatronics Lab, IISc, capable of plotting any 2D image using a pen, which was controlled by means of a Solenoid and two stepper motors (responsible for x, y, and z directional movement).
The control system was governed by an ARM Processor (STM32F4 Discovery Board) to plot images which features were extracted using MATLAB.
Developed the GUI in MATLAB which allows a user to either upload the image of their choice or select any other plot (Arbitrary interpolated curve, texts, shapes).
Used two timers for controlling and synchronizing the parallel movement of X and Y motors to provide any curve of any desired slope.
The solenoid setup was brought back to its initial position after every plot. Limit switches were used to detect its arrival at the desired reset position.
Papers and Publications
[Granted | us 11107228] S Shrivastava. “Realistic Image Perspective Transformation Using Neural Networks”. A system based on a deep neural network to synthesize multiple realistic perspectives of an image.
[2 Papers] The IEEE International Conference on Computer Vision (ICCV) 2023
The IEEE International Conference on Robotics and Automation (ICRA) 2023
The IEEE Robotics and Automation Letters (RA-L) 2022
[2 Papers] The IEEE International Conference on Robotics and Automation (ICRA) 2022
[2 Papers] The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022
[2 Papers] ASME Journal of Autonomous Vehicles and Systems 2021