Software Engineer - Data Engineering

We work in a distributed team with a remote-first mindset. By that we just mean that we work to ensure everyone is on the same playing field in terms of collaboration and information, regardless of where they work. We collaborate frequently in many different forms including daily stand-ups, planning meetings, and many ad-hoc discussions/brainstorming/troubleshooting/pairing sessions over zoom or slack. Our team is operationally responsible for the services we own, so we do have an on-call rotation in place where each member will take a turn serving as the front-line to incidents for our services. Rotations last a week each, during business hours. We value maintaining a healthy work/life balance and prefer sustainable development over heroic efforts. 

What you’ll do:  

We are seeking an experienced Software Engineer highly skilled in implementing and maintaining cloud-based data management solutions at petabyte scale.  In this role you will be responsible for working closely with engineers to drive the technical road map of AWS-native ETL and storage solutions that power data driven analytics, simulation, and training across the enterprise. 

• Create robust and resilient ETL pipelines to process massive daily volumes of data created by vehicle fleets 
• Scale Torc’s data lake through a distributed storage system, data crawling and discovery 
• Promote and protect the integrity of data through validation, versioning, and data provenance 
• Support the expansion of Torc’s data lake through acquisition of additional data sets from internal and external sources 
• Govern data access throughout the data lake at table, column, and row levels 
• Assist in architecture and development of cloud-based solutions for all aspects of software build, test and deploy processes.  
• Collaborate with teams specializing in perception, planning, control, mapping and vehicle testing to develop solutions that support their development efforts.  
• Support the implementation of emerging cloud-based capabilities that can extend our technology stack and improve our ability to build, deploy and test safety-critical software for self-driving vehicles.   

Here’s a list of some of the technologies we use to make all of the above happen: 

• Managed services powered by AWS (Lambda, SFN, Batch, EventBridge, Athena, Glue) 
• Terraform 
• Python 
• On-Call Tooling (PagerDuty, Datadog, AWS Cloudwatch) 

What you’ll need to Succeed:  

• BS/MS Degree in Computer Engineering, Computer Science, or related field
• New graduates with data engineering academic degree with training in full stack development will be considered   
• 4 plus years of experience preferred building and maintaining workloads in public cloud environments  
• Deep knowledge of different database architectures, including but not limited to relational and NoSQL databases, data warehousing and clustered, distributed data stores 
• Implementation of workflow patterns using directed acyclic graphs (Apache Airflow, AWS Step Functions) 
• A strong commitment to test-driven development patterns, continuous integration and delivery, and infrastructure as code 
• Practical experience with Python libraries for applied data science (Pandas, Plotly, Matplotlib, Dask, TensorFlow) 

Bonus Points!  

• Strong organizational, time management, and communication skills working with a team orientation and collaborative style  
• Deep knowledge of AWS serverless architectures (Lambda, Batch, ECS Fargate, Glue, Athena)  
• Experience with data storage and acquisition patterns for robotics and advanced driver assistance systems     

Perks of Being a Full-time Torc’r

Torc cares about our team members and we strive to provide benefits and resources to support their health, work/life balance, and future. Our culture is collaborative, energetic, and team focused. Torc offers:  

• A competitive compensation package that includes a bonus component and stock options
• 100% paid medical, dental, and vision premiums for full-time employees  
• 401K plan with a 6% employer match
• Flexibility in schedule and generous paid vacation (available immediately after start date)
• Company-wide holiday office closures
• AD+D and Life Insurance  

#IND