{"id":4387715,"url":"https://alion.io/job/shamrock-trading-corporation-data-engineer","title":"Data Engineer","company":{"id":3913528,"name":"Shamrock Trading Corporation","domain":"shamrocktradingcorp.com","url":"https://alion.io/company/shamrock-trading-corporation","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Dayforce","truth_index":{"grade":"A","score":97,"open_postings":88,"ghost_share":0,"stale_share":0.102,"repost_share":0.023,"time_to_fill_p50_days":44,"computed_at":"2026-11-16T05:46:25Z"}},"role":"Data Science","role_family":"Data Science","seniority":"junior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Overland Park, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":2,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon ECS","optional":false},{"name":"Amazon S3","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"AWS Lambda","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Docker","optional":false},{"name":"ETL/ELT","optional":false},{"name":"FastAPI","optional":false},{"name":"Git","optional":false},{"name":"GitHub","optional":false},{"name":"GitHub Actions","optional":false},{"name":"GitLab","optional":false},{"name":"Grafana","optional":false},{"name":"MLFlow","optional":false},{"name":"MS SQL","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Power BI","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Terraform","optional":false}],"status":"live","first_seen_at":"2026-11-16T07:00:00Z","employer_posted_date":"2026-11-16","last_verified_at":"2026-11-16T23:17:35Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"About the Role\nShamrock Trading Corporation is looking for a Data Engineer who wants to utilize their expertise in modern data platforms, microservices architectures, application programming interfaces, data pipeline development/support and analytical reporting skills by joining our Data Services team. This role is responsible for gathering and analyzing data from several internal and external sources, designing a cloud-focused data platform for analytics and business intelligence, reliably providing data to our analysts.\nThis role requires a significant understanding of data engineering and analytical techniques. An ideal candidate will have strong technical capabilities, business acumen, and the ability to work effectively with cross-functional teams\nWhat You’ll Do\nDevelop & Maintain Scalable Data Pipelines: Build, optimize, and maintain ETL/ELT pipelines using Databricks, Apache Spark, and Delta Lake.\nOptimize Data Processing: Implement performance tuning techniques to improve Spark-based workloads.\nCloud Data Engineering: Work with AWS services (S3, ECS, Lambda, Glue, etc.) to design and implement robust data architectures.\nReal-time & Streaming Data: Develop streaming solutions using Kafka and Databricks Structured Streaming.\nData Quality & Governance: Implement data validation, observability, and governance best practices using Unity Catalog or other tools.\nCross-functional Collaboration: Partner with analysts, data scientists, and application engineers to ensure data meets business needs.\nAutomation & CI/CD: Implement infrastructure-as-code (IaC) and CI/CD best practices for data pipelines using tools like Terraform, and GitHub Actions.\nWhat You’ll Bring\nBachelor’s degree in computer science, data science or related technical field, or equivalent practical experience\n2-4 years of experience in data engineering, with a focus on cloud-based platforms.\nStrong hands-on experience with Databricks (including Spark, Delta Lake, and MLflow).\nExperience building and maintaining AWS based data pipelines: currently utilizing AWS Lambda, Docker / ECS, MSK, Airflow, Databricks, Unity Catalog\nDevelopment experience utilizing one or more of the following:\nGit ecosystem (GitHub, GitLab, etc.)\nPython\nDatabricks (PySpark, SQL, Spark Declarative Pipelines/DLT, Databricks Asset Bundles)\nKafka and the Kafka Connect ecosystem (Schema Registry,Avro, etc.)\nFamiliarity with CI/CD for data pipelines and infrastructure as code (GitHub Actions, Terraform)\nStrong SQL skills for data transformation and performance tuning.\nUnderstanding of data security and governance best practices.\nEnthusiasm for working directly with customer teams (Business units and internal IT)\nWhat Will Set You Apart\nProven experience with relational and NoSQL databases (e.g. Postgres, Microsoft SQL Server, MongoDB, etc.)\nFamiliarity with data lakehouse architectures and optimization strategies.\nFamiliarity with API architectures such as FastAPI, AWS API Gateways, and OpenAPI Specifications\nFamiliarity with data visualization techniques using tools such as Databricks Dashboards, Grafana, PowerBI, and Excel.","description_format":"text","description_chars":3113,"description_truncated":false,"requirements":{"experience_years_min":2,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":true},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Logistics","Factoring & Trade Finance"],"lifecycle":[{"event":"open","at":"2026-11-16T18:03:09Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":44,"reasons":["conf:3","win:early","comp:junior,brand"],"computed_at":"2026-11-17T02:28:33Z"},"pay":null,"html_url":"https://alion.io/job/shamrock-trading-corporation-data-engineer","json_url":"https://alion.io/job/shamrock-trading-corporation-data-engineer.json","meta":{"generated_at":"2026-11-17T02:28:33Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4579,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3913528},"rest":"https://alion.io/mcp/rest/get_company?id=3913528"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fshamrock-trading-corporation-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fshamrock-trading-corporation-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fshamrock-trading-corporation-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/shamrock-trading-corporation-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fshamrock-trading-corporation-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4387715}},{"tool":"search_jobs","arguments":{"company":"shamrocktradingcorp.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4387715},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}