Staff Data Engineer

Job Description

Unite USA Inc. dba Unite UsStaff Data EngineerLocation: New York, New YorkJob DescriptionThe Staff Data Engineer will lead enterprise-wide data architecture strategy and long-term dataplatform design. Evaluate and approve major technical designs, including establishingengineering best practices. Serve as technical advisor to leadership and influence product/dataroadmap. Champion data governance, democratization, and scalable data standards. Work withcross functional teams including the Solutions Delivery, Business Intelligence; PredictiveAnalytics, and Enterprise Services teams to assist with and support any data-related technicalissues and/or needs. Analyze, manipulate, and process large sets of data using statisticalsoftware. Develop ETL (Extract, Transform, Load) and data pipeline solutions using Airflow,DBT, Python, and Snowflake to load a data warehouse. Design and manage ETL workflows tosupport large-scale ingestion, transformation, and loading of data. Build and scale CI/CDpipelines and enterprise-wide data quality frameworks. Test internal data pipelines for reliabilityand performance. Test, validate, and reformulate models to ensure accuracy. Complete andimprove monitoring and validation processes to maintain pipeline reliability and data accuracy.Write new functions or applications in programming languages to conduct analysis. Buildstreaming pipelines with Confluent Kafka for event-driven processing. Build dashboards andmonitors for pipeline health, freshness, uptime, and SLA compliance. Apply data modeling toextract and analyze information from large structured and unstructured datasets. Telecommutingavailable anywhere in the United States. Annual salary between $175,000 - $200,000. Job RequirementsRequires at least a Master’s degree or foreign equivalent in Data Science, Applied ComputerScience, Computer Science or a related field. Must possess 4 years (48 months) of experiencewith all of the following: (a) developing and optimizing scalable pipelines using Apache Airflowand Python; (b) building streaming pipelines using Kafka; (c) tuning big data systems usingKafka and Hive; (d) using Spark to create jobs for batch and streaming analytics; (e) writing andusing complex SQL queries to retrieve data from data warehouses for analysis and reporting (f)using AWS for data processing and large-volume workloads; (g) automating deployment of codeand processes using Python; (h) working with cloud-based infrastructure; (i) building Datadogdashboards to monitor pipelines and system performance; (j) using Docker and Kubernetes toimprove and validate containerization and automation reliability; (k) implementing ContinuousIntegration/Continuous Deployment (CI/CD) using Jenkins; and (l) utilizing all of the following:GitHub as a code repository, Windows and Linux operating systems. Experience may be gainedconcurrently. Telecommuting available anywhere in the United States.Benefits: Paid Parental Leave; 401K Match; Flexible Vacation Time; Medical, Dental andVision Coverage; Commuter Benefits.