Robert Half International Lead Data Architect in PHILADELPHIA, Pennsylvania
We are looking for a Technical Lead for a large graph representation of the access network project. As part of this project, the candidate would be responsible for extracting data from various disparate data sources through ETL scripts and storing them in S3 and eventually importing them into AWS Neptune database. They would then also be responsible for developing APIs to expose to consume data from the Neptune DB. In this role, you will be responsible for design and architecting the E2E solution and providing guidance to junior data engineers to get the work done.
Must be very proficient in Python, especially using libraries like Pandas, etc. Needs experience working with Databricks. Should be very familiar using AWS services.
• Developing large scale data pipelines exposing data sources within our large company to our team of data analysts and data scientists.
• Developing REST APIs utilizing AWS lambda and API Gateway.
• Developing Spark streaming and batch jobs to clean and transform data.
• Writing build automation to deploy and manage cloud resources.
• Writing unit and integration tests.
Some of the specific technologies we use:
• Programming Languages (Python, Scala, Golang, Node.js)
• Build Environment: GitHub Enterprise, Concourse CI, Jira, Serverless, SAM
• Cloud Computing (AWS Lambda, EC2, ECS)
• Spark (AWS EMR, Databricks)
• Stream Data Platforms: Kinesis, Kafka
• Databases: S3, MySQL, Oracle, MongoDB, DynamoDB
• Caching Frameworks (ElasticCache/ Redis)
Top three Must have technical skill sets: Python using Pandas, Spark (AWS EMR, Databricks), and Cloud Computing (AWS Lambda, EC2, ECS).
Preferred skill sets: Scala, AWS Neptune, as well as QuickSight and Tableau.
Soft Skill requirements (team fit/personality requirements)
Should have worked as a lead before on data engineering projects. Should have had experience with data architecture.
10+ years working as a software engineer.
5+ years working within an enterprise data lake/warehouse environment or big data architecture.
Can do attitude!!!
Should not be scared of new things and should be able to learn new things quickly and find a path forward
Robert Half Technology matches IT professionals with remote or on-site jobs on a temporary, project or full-time basis. From roles in software and applications to IT infrastructure and operations, we provide you unparalleled access to exciting career opportunities.
Our experienced staffing professionals can promote you to employers and advocate on your behalf. We provide access to top jobs, competitive compensation and benefits, and free online training. For more opportunities, get the Robert Half app and receive instant notifications when our AI matches you with jobs.
When you work with us, you’re working with the best. Robert Half has been recognized as one of FORTUNE’s “Most Admired Companies” every year since 1998 and was named to Forbes’ inaugural list of America’s Best Temporary Staffing Firms.
Questions? Call your local office at 1.888.490.4429. All applicants applying for U.S. job openings must be authorized to work in the United States. Benefits are available to temporary professionals. Visit https://roberthalf.gobenefits.net/ for more information.