{"id":4392236,"url":"https://alion.io/job/auquan-senior-data-engineer-databricks-new-york","title":"Senior Data Engineer (Databricks) - New York","company":{"id":63554,"name":"Auquan","domain":"auquan.com","url":"https://alion.io/company/auquan","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"HiBob","truth_index":{"grade":"D","score":45,"open_postings":11,"ghost_share":0.909,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-16T05:46:25Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["New York, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":180000,"max":220000,"currency":"USD","period":"year","gross":null,"usd_annual":220000},"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Dagster","optional":false},{"name":"GCP","optional":false},{"name":"Prefect","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Amazon Kinesis","optional":true},{"name":"Apache Kafka","optional":true},{"name":"dbt","optional":true}],"status":"live","first_seen_at":"2026-11-16T13:54:31Z","employer_posted_date":"2026-11-16","last_verified_at":"2026-11-17T00:35:17Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":0,"flags":["company_stale"],"days_open":0},"description":"ALFA builds autonomous AI agents for finance workflows (credit, investment, ESG and risk) for banks, private equity and private credit firms, and asset managers. Agents are only as good as the data they run on, and that data arrives messy: filings, portfolio packs, CRM exports, spreadsheets and PDFs.The Data Engineer owns the path from a customer's raw data to clean, secure, well-modeled data that our agentic workflows and deliverables can trust. You will work in pods alongside Outcome Engineers and Product Engineers, and turn one-off customer data work into reusable pipelines and primitives as our platform grows.\nResponsibilities\nBuild ingestion pipelines for customer financial data from APIs, spreadsheets, documents and third-party sources.\nDesign and maintain data models that keep each customer's data secure and separate.\nSupply agentic workflows with reliable inputs, and keep outputs traceable back to source.\nUse LLMs to extract structured data from unstructured documents, with evals and validation checks that catch bad extractions before customers do.\nPlan and run data migrations as our platform evolves.\nWork with Outcome Engineers on live customer deliveries, then turn bespoke data work into reusable connectors, schemas and templates.\nOwn data quality, monitoring and runbooks for the pipelines you ship, and join on-call for incidents that touch data.\nRequirements\nWhat we’re looking for5+ years in data engineering, building production pipelines end to end.\nStrong Python and SQL.\nHands-on experience with Spark or another distributed processing engine.\nExperience with orchestration tools (Airflow, Dagster, Prefect or similar) and cloud data platforms (AWS, GCP or Azure).\nSolid grasp of data modelling, partitioning, and incremental or change-data-capture (CDC) patterns.\nYou take ownership, communicate clearly, and like working closely with the people who use the data.\nNice to have\nDatabrtalog,Workflows, DLT) or similar lakehouse work on Snorg/Hudi.\nExperience with financial or market data.\nExperi or AIsystems: document parsing, vector stores, retrie\nStreaming (Kafka, Kinesis) and dbt.\nCustomer empathy: you think like an analyst or risk officer and know which numbers drive their decisions\nStructured thinker: you turn noisy data into clear schemas, steps and validation rules\nComfortable in ambiguity: you make progress with imperfect data and bring structure where there is none\nProactive problem-solver: you spot a broken feed before the client does\nHustle and ownership: you own outcomes end to end, not just tickets\nCurious, fast learner: you keep up with data tooling, AI and finance developments\nClear communicator: you explain data trade-offs to senior customer stakeholders\nCommercial instinct: you see where a pipeline built for one customer can serve the next\nBenefits\nWe are an office-first company and expect regular in-person collaboration as part of this role. You should also be comfortable collaborating across time zones (US, UK, India).We offer a competitive package designed to support you both inside and outside of work:\n5 days in-person in our Soho offices, with occasional client travel\nBase Salary: $180,000 - $220,000 for New York. Factors that may impact the agreed upon salary within the range for a particular candidate include years of experience, level of education obtained, skill set, and other external factors.\nPerformance Bonus\nEquity: Meaningful stock options so you share in our growth\nTime Off: Flexible PTO\nHealth & Wellbeing: Comprehensive health coverage (US)\nRetirement: 401(k) options depending on jurisdiction","description_format":"text","description_chars":3593,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Stock options"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Agents","Financial AI","Document AI"],"lifecycle":[{"event":"open","at":"2026-11-16T19:11:27Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":30,"reasons":["conf:4","win:early"],"computed_at":"2026-11-17T05:14:41Z"},"pay":{"stated_usd_annual":220000,"is_top_pay":true},"html_url":"https://alion.io/job/auquan-senior-data-engineer-databricks-new-york","json_url":"https://alion.io/job/auquan-senior-data-engineer-databricks-new-york.json","meta":{"generated_at":"2026-11-17T05:14:41Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9691,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":63554},"rest":"https://alion.io/mcp/rest/get_company?id=63554"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fauquan-senior-data-engineer-databricks-new-york"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fauquan-senior-data-engineer-databricks-new-york"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fauquan-senior-data-engineer-databricks-new-york"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/auquan-senior-data-engineer-databricks-new-york\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fauquan-senior-data-engineer-databricks-new-york"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4392236}},{"tool":"search_jobs","arguments":{"company":"auquan.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4392236},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}