{"id":3294832,"url":"https://alion.io/job/bright-data-platform-engineer","title":"Data Platform Engineer","company":{"id":23363,"name":"Bright","domain":"bright.ai","url":"https://alion.io/company/bright","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":89,"open_postings":4,"ghost_share":0,"stale_share":0.25,"repost_share":0,"time_to_fill_p50_days":139,"computed_at":"2026-10-31T05:58:55Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Palo Alto, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":134000,"max_usd":254000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1343},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon Kinesis","optional":false},{"name":"Amazon S3","optional":false},{"name":"Apache Kafka","optional":false},{"name":"Argo Workflows","optional":false},{"name":"AWS","optional":false},{"name":"AWS Lambda","optional":false},{"name":"Databricks","optional":false},{"name":"Datadog","optional":false},{"name":"Delta Lake","optional":false},{"name":"DVC","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Go","optional":false},{"name":"MLFlow","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Sumo Logic","optional":false},{"name":"Terraform","optional":false},{"name":"Feature Store","optional":true},{"name":"gRPC","optional":true},{"name":"Kubernetes","optional":true},{"name":"Machine Learning","optional":true},{"name":"Time Series","optional":true}],"status":"live","first_seen_at":"2025-10-07T23:41:49Z","employer_posted_date":"2026-10-26","last_verified_at":"2026-11-01T00:18:41Z","board_verified":true,"closed_at":null,"days_open":389,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":389},"description":"We are a high-growth company transforming how businesses operate by integrating AI, IoT, and cloud-native services into scalable, real-time platforms. As a Platform Data Engineer, you’ll play a critical role in building and maintaining the data infrastructure that powers our products, services, and insights. \nYou’ll join a multidisciplinary team focused on ingesting, processing, and managing massive streams of sensor and operational data across a wide array of devices-from drones and robots to industrial systems and smart environments.\nResponsibilities\nDesign, build, and maintain scalable, reliable, and high-throughput data ingestion pipelines for structured and semi-structured data.\nImplement robust and secure data lake and SQL-based storage architectures optimized for performance and cost.\nDevelop and maintain internal tools and frameworks for data ingestion using Python, Golang, and SQL.\nBuild and maintain inference pipelines to support model serving and near real time prediction workflows\nDevelop and maintain internal tools and deployment processes to support ML development workflow , including experimentation, testing, deployment and monitoring.\nCollaborate cross-functionally with Cloud, Edge, Product, and AI teams to define data contracts, schemas, and retention policies.\nUse AWS cloud infrastructure (including Argo Workflows, S3, Lambda, Glue, Kinesis, Athena, and RDS) to support end-to-end data workflows.\nEmploy Infrastructure-as-Code (IaC) practices using Terraform to manage data platform infrastructure.\nMonitor data pipelines for quality, latency, and failures using tools such as CloudWatch, SumoLogic, or DataDog.\nBuild and maintain ingestion pipelines that land curated sensor and operational data into Delta tables on Databricks to support ML experimentation and model training.\nContinuously optimize storage, partitioning, and query performance across large-scale datasets.\nParticipate in architecture reviews and ensure the platform adheres to security, compliance, and best practice standards.\nSkills and Qualifications\n5+ years of professional experience in software engineering or data engineering.\nStrong programming skills in Python and Golang.\nDeep understanding of SQL and modern data lake architectures (e.g., using Parquet, Iceberg, or Delta Lake).\nHands-on experience with AWS services including but not limited to: S3, Lambda, Glue, Kinesis, Athena, and RDS.\nProficiency with Terraform for automating infrastructure deployment and management.\nHands-on experience with Databricks for ML workflows: MLflow experiment tracking and model registry, Model Serving, and building training datasets in Delta Lake (PySpark/Spark SQL).\nExperience working with real-time or batch data ingestion at scale, and designing fault-tolerant ETL/ELT pipelines.\nFamiliarity with DVC for dataset versioning and management.\nFamiliarity with event-driven architectures and messaging systems like Kafka or Kinesis.\nStrong debugging and optimization skills across cloud, network, and application layers.\nExcellent collaboration, communication, and documentation skills.\nBonus Points\nExperience working with time-series or IoT sensor data at industrial scale.\nExposure to gRPC and protobuf-based data contracts.\nExperience supporting ML pipelines and feature stores.\nWorking knowledge of Kubernetes concepts.\nDatabricks Machine Learning certification (Professional)\nPrior startup experience and/or comfort working in fast-paced, iterative environments.","description_format":"text","description_chars":3478,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Edge AI","Industrial AI"],"lifecycle":[{"event":"open","at":"2026-10-26T21:25:16Z"}],"visa":[],"liveness":{"score":15,"band":"cold","label":"Long shot","p_open":1,"p_active":0.524,"p_room":0.28,"age_days":388,"expected_fill_days":139,"reasons":["conf:3","win:tail","crowd:"],"computed_at":"2026-10-31T05:58:55Z"},"pay":null,"html_url":"https://alion.io/job/bright-data-platform-engineer","json_url":"https://alion.io/job/bright-data-platform-engineer.json","meta":{"generated_at":"2026-11-01T00:24:59Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":698,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-02T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":23363},"rest":"https://alion.io/mcp/rest/get_company?id=23363"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbright-data-platform-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbright-data-platform-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbright-data-platform-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/bright-data-platform-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbright-data-platform-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3294832}},{"tool":"search_jobs","arguments":{"company":"bright.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3294832},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}