{"id":123857,"url":"https://alion.io/job/watchguard-senior-data-engineer","title":"Senior Data Engineer","company":{"id":8053,"name":"Watchguard","domain":"watchguard.com","url":"https://alion.io/company/watchguard","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"A","score":89,"open_postings":4,"ghost_share":0,"stale_share":0.25,"repost_share":0,"time_to_fill_p50_days":100,"computed_at":"2026-10-16T05:45:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Miami, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":105000,"max_usd":201000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1181},"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Anomaly Detection","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Bicep","optional":false},{"name":"CI/CD","optional":false},{"name":"dbt","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Platform Engineering","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Rest API","optional":false},{"name":"Snowflake","optional":false},{"name":"SQL","optional":false},{"name":"Terraform","optional":false},{"name":"Cortex","optional":true},{"name":"LLM","optional":true},{"name":"Microsoft Fabric","optional":true},{"name":"Prometheus","optional":true},{"name":"Spark","optional":true}],"status":"live","first_seen_at":"2026-06-24T13:34:08Z","employer_posted_date":"2026-06-24","last_verified_at":"2026-10-16T15:44:59Z","board_verified":true,"closed_at":null,"days_open":114,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":114},"description":"We are looking for a Senior Data Engineer to join our growing data platform team. You will own the design, build, and reliability of our cloud-native data lakehouse - from raw ingestion through to analytics-ready Gold tables. You will work closely with data analysts, analytics engineers, and product stakeholders to deliver trusted data at speed, while championing data quality and observability as first-class concerns.\nThis role sits at the intersection of data engineering and platform engineering - you will be expected to think in architectures, not just pipelines.\nWhat You Will Do\nData Platform & Pipeline Engineering\nDesign, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Apache Airflow, processing structured and semi-structured data across the Medallion architecture (Bronze → Silver → Gold).\nImplement incremental load patterns, change data capture (CDC), and event-driven ingestion to ensure data freshness across the platform.\nBuild and optimise Snowflake data warehouse objects - tables, views, dynamic tables, streams, tasks, and stored procedures - for performance and cost efficiency.\nDevelop modular, tested dbt models aligned to each Medallion layer, enforcing consistent naming conventions, documentation, and lineage across all transformations.\nData Quality & Observability\nEmbed automated data validation at every Medallion layer using Elementary (dbt's observability layer), ensuring anomaly detection, freshness checks, and schema drift alerts are in place before data reaches consumers.\nDefine and enforce data contracts between producers and consumers - row count checks, null rate thresholds, referential integrity, and value domain validation.\nBuild and maintain data quality dashboards to give engineering and business stakeholders real-time confidence in platform health.\nAzure Cloud Infrastructure\nManage and optimise Azure Data Lake Storage Gen2 (ADLS) - folder structures, lifecycle policies, access tiers, and partition strategies.\nBuild and maintain Azure Functions and Azure Logic Apps for lightweight event-driven processing, orchestration triggers, and operational automation.\nManage secrets, credentials, and environment-specific configuration securely using Azure Key Vault - no hardcoded credentials in pipelines or code.\nContribute to infrastructure-as-code practices for provisioning Azure data services (Terraform or Bicep preferred).\nCollaboration & Delivery\nTranslate ambiguous business requirements into well-defined data models and pipeline designs, working with analysts and stakeholders to validate assumptions before build.\nParticipate in code reviews, enforce standards, and mentor junior engineers on data engineering best practices.\nSupport CI/CD adoption for pipeline and dbt model deployment across Dev / Test / Prod environments.\nWhat We Are Looking For\nMust-Have\nSnowflake: Snowflake\n- Advanced SQL - window functions, CTEs, recursive queries, query profiling\n- Snowflake-native features: streams, tasks, snowpipe, dynamic tables, row-level security\n- Virtual warehouse tuning and credit cost optimisation\ndbt + Elementary: dbt + Elementary\n- Writing, testing, and documenting production dbt models\n- Elementary integration for data observability and anomaly detection\n- dbt incremental strategies, snapshots, and semantic layer\nAzure Cloud: Azure Cloud\n- Azure Data Factory - pipeline authoring, triggers, parameterisation, linked services\n- ADLS Gen2 - zone/folder design, lifecycle management, Parquet/Delta partitioning\n- Azure Key Vault - secret management, managed identities\n- Azure Functions / Logic Apps - event-driven triggers and lightweight automation\nAirflow: Airflow\n- DAG authoring, task dependencies, XCom, sensors, and connection management\n- Airflow deployment and monitoring in cloud-hosted environments\nPython: Python\n- Data pipeline scripting, PySpark basics, REST API integration\n- Unit testing pipeline logic and transformation functions\nData Quality & Medallion Architecture: Medallion Architecture:\n- Hands-on experience implementing Bronze / Silver / Gold Medallion architecture\n- Data validation checks at each layer - not just at the final Gold layer\n- Schema evolution handling and SCD Type 2 dimension management\n4+ years of professional data engineering experience with at least 2 years on Azure cloud data platforms.\nNice-to-Have\nExposure to Snowflake Cortex, dbt Semantic Layer, or Boomi Data Hub for AI-assisted data enrichment within pipeline layers.\nExperience integrating LLM-based quality checks or AI-assisted anomaly detection into data workflows.\nFamiliarity with Microsoft Fabric and OneLake as a complementary or future-state platform.\nKnowledge of data mesh or data product thinking and how it maps to Medallion layer ownership.\nExperience with Terraform or Bicep for Azure infrastructure provisioning.","description_format":"text","description_chars":4840,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Road Transport","Automotive Components"],"lifecycle":[{"event":"open","at":"2026-06-24T13:34:08Z"}],"visa":[],"liveness":{"score":30,"band":"fade","label":"Fading","p_open":1,"p_active":0.678,"p_room":0.44,"age_days":113,"expected_fill_days":100,"reasons":["conf:3","win:tail","crowd:"],"computed_at":"2026-10-16T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/watchguard-senior-data-engineer","json_url":"https://alion.io/job/watchguard-senior-data-engineer.json","meta":{"generated_at":"2026-10-16T17:40:48Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2355,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":8053},"rest":"https://alion.io/mcp/rest/get_company?id=8053"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fwatchguard-senior-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fwatchguard-senior-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fwatchguard-senior-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/watchguard-senior-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fwatchguard-senior-data-engineer"}]}