{"id":20269,"url":"https://alion.io/job/blackducksoftware-principal-data-engineer","title":"Principal Data Engineer","company":{"id":720312,"name":"Black Duck","domain":"blackduck.com","url":"https://alion.io/company/blackduck","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":87,"open_postings":6,"ghost_share":0,"stale_share":0.333,"repost_share":0,"time_to_fill_p50_days":67,"computed_at":"2026-11-02T06:13:15Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Belfast, United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":92000,"max_usd":163000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":37},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"GCP","optional":false},{"name":"IAM","optional":false},{"name":"Python","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-04-07T07:48:57Z","employer_posted_date":"2026-10-19","last_verified_at":"2026-11-02T14:03:41Z","board_verified":true,"closed_at":null,"days_open":209,"trust":{"level":"stale","repost_count":1,"flags":["stale"],"days_open":208},"description":"Black Duck Software, Inc. helps organizations build secure, high-quality software, minimizing risks while maximizing speed and productivity. Black Duck, a recognized pioneer in application security, provides SAST, SCA, and DAST solutions that enable teams to quickly find and fix vulnerabilities and defects in proprietary code, open source components, and application behavior. With a combination of industry-leading tools, services, and expertise, only Black Duck helps organizations maximize security and quality in DevSecOps and throughout the software development life cycle.\nWhat you’ll do\nLead the design and build-out of cross-product data services for multiple product lines from one governed data plane.\nDefine the “customer data plane” model: canonical customer identifiers, shared dimensions, and consistent facts used across products.\nBuild and operationalize ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources.\nOwn the operational playbook for data reliability: data contracts, quality checks, lineage, monitoring, and incident response.\nImplement and run access methods that make data usable: curated datasets, secure query interfaces, and product-ready data APIs where needed.\nProductize customer-facing data products (datasets, metrics, exports, and feeds) with versioning, documentation, and clear ownership.\nDesign data models that fit both operational systems (RDS) and analytics stores (columnar/OLAP), including performance and cost tuning.\nEnsure data products also power ML workflows: trusted training datasets, feature-ready outputs, and consistent definitions for decision-making.\nEnable AI automation by delivering reliable, low-latency, governed data products that can be used safely in automated workflows.\nPartner closely with product, engineering, and security stakeholders to align data products to roadmap priorities and customer outcomes.\nRaise the technical bar through architecture reviews, standards, and mentoring-while staying hands-on in key systems.\nRequired\nSignificant experience building and operating production data platforms at scale, including on-call and operational ownership.\nStrong SQL skills and strong Python skills, used to build pipelines, services, and automation.\nHands-on experience running cloud systems on AWS and Google Cloud (IaaS level: compute, storage, networking, IAM).\nPractical experience with both operational databases (RDS-style) and analytics stores (columnar/OLAP), including performance tuning.\nStrong data modeling ability, including schema evolution, conformed dimensions, and “one source of truth” metric definitions.\nTrack record of delivering data products that other teams or customers depend on, with clear contracts and reliability expectations.\nAbility to make sound engineering tradeoffs across latency, accuracy, cost, and security without creating brittle complexity.\nExperience with lakehouse patterns and open table formats (or similar), including governance and table maintenance.\nExperience with orchestration and streaming systems used in production (batch + real-time), and managing backfills safely.\nFamiliarity with ML data needs (training/serving splits, feature-ready datasets, evaluation datasets) and AI-adjacent workflows.\nPreferred\nExperience building self-service data platforms (catalog, discoverability, access controls) used by multiple teams.\nExperience in regulated or security-sensitive environments, including retention, auditing, and data access controls.\nWork model, location & travel\nLocation: Belfast, UK\nReports to: VP of Data Engineering\nWork model: Hybrid (details TBD)\nCollaboration hours: Flexible; overlap with UK and US time zones\nTravel: Minimal\nBlack Duck is an equal opportunity employer. We consider all applicants for employment without regard to race, color, national origin, religion, sex, gender identity or expression, age, disability, sexual orientation, veteran or military service status, or any other characteristic protected by applicable law. Black Duck complies with all applicable laws prohibiting employment discrimination in every jurisdiction where it operates and provides reasonable accommodations to individuals with disabilities in accordance with applicable law.","description_format":"text","description_chars":4256,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United Kingdom","iso":"GB","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Application Security","DevSecOps"],"lifecycle":[{"event":"open","at":"2026-04-07T07:48:57Z"},{"event":"close","at":"2026-09-21T15:12:49Z"},{"event":"reopen","at":"2026-09-23T15:15:39Z"}],"visa":[],"liveness":{"score":14,"band":"cold","label":"Long shot","p_open":1,"p_active":0.486,"p_room":0.28,"age_days":208,"expected_fill_days":67,"reasons":["conf:0","velocity","win:tail","crowd:"],"computed_at":"2026-11-02T06:13:15Z"},"pay":null,"html_url":"https://alion.io/job/blackducksoftware-principal-data-engineer","json_url":"https://alion.io/job/blackducksoftware-principal-data-engineer.json","meta":{"generated_at":"2026-11-02T15:11:43Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":21035,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":720312},"rest":"https://alion.io/mcp/rest/get_company?id=720312"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fblackducksoftware-principal-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fblackducksoftware-principal-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fblackducksoftware-principal-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/blackducksoftware-principal-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fblackducksoftware-principal-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":20269}},{"tool":"search_jobs","arguments":{"company":"blackduck.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":20269},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}