{"id":94457,"url":"https://alion.io/job/qualys-senior-big-data-engineer-2","title":"Senior Big Data Engineer","company":{"id":18966,"name":"Qualys","domain":"qualys.com","url":"https://alion.io/company/qualys","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":21000,"max_usd":43000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":71},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Apache Kafka","optional":false},{"name":"Cassandra","optional":false},{"name":"Hadoop","optional":false},{"name":"Java","optional":false},{"name":"Oracle","optional":false},{"name":"Presto","optional":false},{"name":"Qualys Cloud Platform","optional":false},{"name":"Redis","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"Agile","optional":true},{"name":"Scrum","optional":true},{"name":"Unix","optional":true}],"status":"live","first_seen_at":"2026-07-20T00:00:00Z","employer_posted_date":"2026-07-20","last_verified_at":"2026-10-18T16:46:47Z","board_verified":true,"closed_at":null,"days_open":92,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":92},"description":"Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!\nDescription: \nWe are seeking a talented Sr. Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. \nWorking with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product. \nThis is a fantastic opportunity to be an integral part of a team building Qualys next generation platform using Big Data & Micro-Services based technology to process over billions of transactions data per day, leverage open-source technologies, and work on challenging and business-impacting initiatives. \nResponsibilities:\nBe the thought leader in data platform and pipeline along with Risk Evaluation. \nProvide technical leadership to the engineering organization on data platform design, roll out and evolution. \nLiason to product teams, professional services and sales engineers on solution and trade-off reviews and represent engineering in such conversations. \nDrive technology explorations and roadmaps. \nServe as a technical lead on our most demanding, cross-functional departments. \nEnsure the quality of architecture and design of systems. \nFunctionally decompose complex problems into simple, straight-forward solutions. \nFully and completely understand system interdependencies and limitations. \nPossess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices. \nLeverage knowledge of internal and industry prior art in design decisions. \nEffectively research and benchmark cloud technology against other competing systems in the industry. \nAble to document the details so it will be easy for developers to understand the requirements. \nAssisting developers with proper requirements and directions. \nAssist in the career development of others, actively mentoring individuals and the community on advanced technical issues and helping managers guide the career growth of their team members. \nExert technical influence over multiple teams, increasing their productivity and effectiveness by sharing your deep knowledge and experience. \nAble to share knowledge and train others. \nQualifications: \nBachelor’s degree in computer science or equivalent \n5+ years of total experience.\nRequired Skills: Java/Scala, Apache Spark, Kafka, Hadoop, RDBMS (Oracle or equivalent), and NoSQL databases (Cassandra or equivalent).\nExperience building scalable, enterprise-grade big data solutions using either Java or Scala. \n2+ years of relevant experience in design and architecture Big Data solutions using Spark \n3+ years experience in working with engineering resources for innovation. \n4+ years experience in understanding Big Data events flow pipeline. \n3+ years experience in performance testing for large infrastructure. \n3+ In depth experience in understanding various search solutions solr/elastic. \n3+ years experience in Kafka \nIn depth experience in Data lakes and related ecosystems. \nIn depth experience of messing queue \nIn depth experience in giving requirements to build a scalable architecture for Big data and Micro-services environments. \nIn depth experience in understanding caching components or services \nKnowledge in Presto technology. \nKnowledge in Airflow. \nHands-on experience in scripting and automation \nIn depth understanding of RDBMS/NoSQL, Oracle , Cassandra , Kafka , Redis, Hadoop, lambda architecture, kappa , kappa ++ architectures with flink data streaming and rule engines \nExperience in working with ML models engineering and related deployment. \nDesign and implement secure big data clusters to meet many compliances and regulatory requirements. \nExperience in leading the delivery of large-scale systems focused on managing the infrastructure layer of the technology stack. \nStrong experience in doing performance benchmarking testing for Big data technologies. \nStrong troubleshooting skills. \nExperience leading development life cycle process and best practices \nExperience in Big Data services administration would be added value. \nExperience with Agile Management (SCRUM, RUP, XP), OO Modeling, working on internet, UNIX, Middleware, and database related projects. \nExperience mentoring/training the engineering community on complex technical issue.","description_format":"text","description_chars":4466,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Cybersecurity","Endpoint Security","Information Security","Security Compliance"],"lifecycle":[{"event":"open","at":"2026-07-20T00:00:00Z"}],"visa":[],"liveness":{"score":26,"band":"fade","label":"Fading","p_open":1,"p_active":0.722,"p_room":0.36,"age_days":92,"expected_fill_days":59,"reasons":["conf:36","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-20T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/qualys-senior-big-data-engineer-2","json_url":"https://alion.io/job/qualys-senior-big-data-engineer-2.json","meta":{"generated_at":"2026-10-20T11:05:04Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12138,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-21T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":18966},"rest":"https://alion.io/mcp/rest/get_company?id=18966"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fqualys-senior-big-data-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fqualys-senior-big-data-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fqualys-senior-big-data-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/qualys-senior-big-data-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fqualys-senior-big-data-engineer-2"}]}