{"id":3890458,"url":"https://alion.io/job/cadre-ai-senior-ai-data-engineer-latam","title":"Senior AI Data Engineer | LATAM","company":{"id":4423642,"name":"Cadre AI","domain":"cadre.ai","url":"https://alion.io/company/cadre-4","size_band":null,"is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Gem","truth_index":{"grade":"B","score":75,"open_postings":5,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-14T05:58:10Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Colombia"],"countries":["CO"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Arize Phoenix","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Braintrust","optional":false},{"name":"CI/CD","optional":false},{"name":"Claude","optional":false},{"name":"Claude Code","optional":false},{"name":"Dagster","optional":false},{"name":"Databricks","optional":false},{"name":"dbt","optional":false},{"name":"Delta Lake","optional":false},{"name":"Embeddings","optional":false},{"name":"Fivetran","optional":false},{"name":"GitHub Actions","optional":false},{"name":"GraphRAG","optional":false},{"name":"Hybrid Search","optional":false},{"name":"Knowledge Graph","optional":false},{"name":"Langfuse","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Microsoft Fabric","optional":false},{"name":"MLFlow","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Neo4j","optional":false},{"name":"OCR","optional":false},{"name":"OpenAI Codex","optional":false},{"name":"OpenSearch","optional":false},{"name":"pgvector","optional":false},{"name":"Power BI","optional":false},{"name":"RAG","optional":false},{"name":"Reranking","optional":false},{"name":"Snowflake","optional":false},{"name":"SOC 2","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Time Series","optional":false},{"name":"AI Agents","optional":true},{"name":"Anthropic","optional":true},{"name":"Cortex","optional":true},{"name":"DSPy","optional":true},{"name":"Fine-tuning","optional":true},{"name":"LookML","optional":true},{"name":"LoRA","optional":true},{"name":"OpenAI","optional":true},{"name":"PEFT","optional":true},{"name":"PostgreSQL","optional":true},{"name":"Prometheus","optional":true},{"name":"pySpark","optional":true},{"name":"Python","optional":true}],"status":"live","first_seen_at":"2026-11-06T18:17:05Z","employer_posted_date":"2026-11-06","last_verified_at":"2026-11-15T01:03:36Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":8},"description":"About Cadre AI\nCadre AI is an AI strategy and integration firm that builds production AI systems for B2B companies in private equity, wholesale lending, real estate, and SaaS. We do not build decks about what AI could do. We ship systems that move revenue, compress costs, and automate the work that used to take entire teams.\nThe Role\nYou will build the data foundations that make our clients’ AI systems trustworthy: the lakehouses that unify their ERPs and SaaS tools, the retrieval systems that ground agents in their documents and records, the semantic layers that give every metric one definition, and the evaluation datasets that prove an answer is right. Every agent, dashboard, and workflow Cadre ships sits on top of the data layer you design.\nYour week moves between architecture and execution: mapping a manufacturer’s ERP tables with their finance team one day, building Databricks medallion layers or tuning a hybrid retrieval pipeline the next. You thrive here if you keep up with retrieval research, know why a random split leaks in a time series, and can explain a GraphRAG trade-off to a CFO in plain language.\nYou will work in client pods with AI engineers, solution architects, and AI strategists, and shape the reusable data and retrieval blocks every engagement starts from.\nWhat You’ll Do\nLakehouse and Pipeline Engineering\nDesign and build medallion (bronze, silver, gold) lakehouses on Databricks (Delta Lake, Unity Catalog, Lakeflow Spark Declarative Pipelines) and Microsoft Fabric (OneLake, Data Factory), and on Snowflake where a client already runs it.\nOwn ingestion from ERPs, CRMs, and SaaS platforms using platform-native connectors (Lakeflow Connect, Fabric Data Factory), Fivetran, or dlt, including change data capture and schema drift handling.\nModel and transform data with dbt, orchestrate it with Apache Airflow, Dagster, or Lakeflow Jobs, and ship it through CI/CD with GitHub Actions and Declarative Automation Bundles (formerly Databricks Asset Bundles).\nBuild data quality checks and data contracts into every pipeline so downstream agents never consume silent errors.\nRun agentic data engineering safely in client Databricks repositories: set up coding agents (Claude Code, Codex) with shared skills, policies, and guardrails, using Omnigent where it is available, and accelerate legacy-to-Databricks migrations with Genie Migrate.\nRetrieval Systems: RAG and GraphRAG\nBuild production RAG pipelines end to end: document parsing and OCR, chunking, embeddings, hybrid retrieval (BM25 plus dense vectors), and cross-encoder reranking.\nDesign GraphRAG systems that extract entities and relationships into a knowledge graph, using frameworks such as Microsoft GraphRAG or LightRAG on graph stores such as Neo4j, for questions that span documents, systems, and multiple entity hops.\nChoose the right retrieval architecture for each use case, from hybrid search with reranking to graph-based and agentic retrieval, and justify the cost and maintenance burden of each to the client.\nOperate vector, full-text, and hybrid search infrastructure (Databricks AI Search, formerly Vector Search; Azure AI Search; OpenSearch; pgvector), including incremental indexing, permission-aware retrieval, and citations back to the source.\nBuild and deploy retrieval-backed agents on Databricks Agent Bricks, using its code-first tooling (Agent Bricks CLI, Agent Runtime on Databricks Apps) and governing model access through Unity AI Gateway.\nSemantic Layers and Business Ontologies\nTranslate how a client defines a customer, an order, or an invoice into governed metric and entity definitions using dbt Semantic Layer (MetricFlow), Power BI semantic models, Cube, Databricks metric views, or Snowflake semantic views.\nLead data readiness assessments: inventory source systems, map fields and business logic, and document gaps and remediation steps before engineering starts.\nExpose governed data to Claude and other agents through MCP, so every agent, dashboard, and text-to-SQL query answers from the same definitions.\nUse AI to draft documentation and definitions, then own their correctness together with the business owner.\nEvaluation Data and Model Quality\nBuild golden datasets, output and label pairs, and backtesting splits that measure whether an agent, retriever, or model is right.\nInstrument retrieval and agent pipelines with dedicated tracing and evaluation tools (Langfuse, Arize Phoenix, Braintrust), or with MLflow 3 GenAI evaluation and LLM judges on Databricks, and track recall, precision, and groundedness.\nTrack experiments and version models and evaluation runs in the MLflow registry, so every change to a retriever or agent is traceable.\nDesign leakage-free, time-aware splits for forecasting, scoring, and risk ranking.\nMonitor drift, freshness, and data gaps that degrade agents in production, and fix them at the data layer.\nGovernance and Client Delivery\nImplement role-based access, row-level security, and masking in Unity Catalog, Microsoft Purview, or Snowflake roles, and carry those permissions into retrieval and agent context.\nEnsure every data path meets SOC 2 expectations and the client’s own AI governance policies, including the handling of PII, PHI, and payroll data.\nWork directly with client IT, finance, and operations leaders to scope work, secure access, and explain build-versus-buy trade-offs across platforms.\nRecord architecture decisions as ADRs in the client repository and contribute reusable pipeline, retrieval, and semantic-layer blocks to Cadre’s internal library.\nWhat You Need to Succeed\n5+ years of data engineering experience, including at least one production lakehouse you designed and operated on Databricks or Microsoft Fabric with a medallion architecture; experience on both is a plus, since our clients run both.\nA Master’s degree or PhD in Computer Science, Mathematics, Engineering, or a related quantitative field, with the theoretical grounding to read current research and apply it to client problems.\nHands-on experience shipping RAG pipelines to production (hybrid retrieval on Databricks AI Search or an equivalent engine, reranking, chunking strategy, and permission-aware indexing over messy enterprise documents), with command of the current state of the art: you follow the literature and can explain when hybrid, graph-based, or agentic retrieval fits a problem.\nExperience designing and running GraphRAG or knowledge-graph retrieval (Microsoft GraphRAG, LightRAG, or equivalent, on graph stores such as Neo4j), and the judgment to know when a graph earns its maintenance cost and when hybrid search is enough.\nProduction experience orchestrating data pipelines with Apache Airflow or Dagster, including dependencies, backfills, retries, and alerting; you understand the trade-off between task-based and asset-based orchestration and work comfortably in platform-native schedulers such as Lakeflow Jobs and Fabric Data Factory.\nExpert SQL and strong Python and PySpark, with production experience in dbt or platform-native pipelines and in ingestion tools (Fivetran, Lakeflow Connect, Azure or Fabric Data Factory, or dlt); you use coding agents such as Claude Code daily and review everything they generate.\nExperience authoring a semantic or metrics layer (dbt Semantic Layer, Power BI semantic models, Cube, LookML) and turning business definitions into governed metrics that people and agents both rely on.\nWorking ML fluency: you understand data leakage, time-based validation, class imbalance, and precision versus recall, and you have built datasets that trained or evaluated a model or retriever, evaluated with Langfuse, Arize Phoenix, Braintrust, or MLflow and versioned in the MLflow registry.\nHands-on experience building agents on Databricks (Agent Bricks or Mosaic AI Agent Framework) and running coding agents such as Claude Code or Codex under governed harnesses with skills, policies, and guardrails; familiarity with Databricks’ newer agentic tooling (Omnigent, Genie Migrate, Agent Bricks CLI) and with how agents consume data through RAG, text-to-SQL, and MCP.\nGovernance depth across Unity Catalog, Purview, or Snowflake RBAC; you treat access control as a design input for pipelines and retrieval alike.\nClient-facing consulting or forward-deployed experience and professional English fluency, because you will work daily with US executives and turn technical trade-offs into business decisions.\nStrong plus: Snowflake Cortex, Palantir Foundry ontologies, RDF/OWL knowledge graphs, DSPy prompt optimization, or LoRA fine-tuning of open-weight models.\nWhy Cadre AI\nAI-native culture. We do not just build AI for clients. We use it to run our own operations. You will work with people who are as obsessed with the tools as you are.\nAccess to the frontier. Through partnerships with Anthropic and OpenAI, you will be among the first to experiment with new models and capabilities.\nUpside. We are bootstrapped, profitable, and growing fast. Growth means new teams and new initiatives, and room to move into them.\n No bureaucracy. Small, scrappy team. Clear accountability. The best idea wins, regardless of who says it.\nCompensation\nThe base pay range for this role is Col$6,000 - Col$7,500 per year.","description_format":"text","description_chars":9224,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[{"language":"English","level":"Advanced (C1)","optional":false}]},"benefits":["Equity"],"hiring_locations":[{"name":"Colombia","iso":"CO","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Consulting & Integration"],"lifecycle":[{"event":"open","at":"2026-11-06T21:05:39Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":7,"expected_fill_days":30,"reasons":["seen:7","win:early"],"computed_at":"2026-11-14T05:58:10Z"},"pay":null,"html_url":"https://alion.io/job/cadre-ai-senior-ai-data-engineer-latam","json_url":"https://alion.io/job/cadre-ai-senior-ai-data-engineer-latam.json","meta":{"generated_at":"2026-11-15T05:23:35Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":626,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-16T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4423642},"rest":"https://alion.io/mcp/rest/get_company?id=4423642"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcadre-ai-senior-ai-data-engineer-latam"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcadre-ai-senior-ai-data-engineer-latam"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcadre-ai-senior-ai-data-engineer-latam"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/cadre-ai-senior-ai-data-engineer-latam\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcadre-ai-senior-ai-data-engineer-latam"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3890458}},{"tool":"search_jobs","arguments":{"company":"cadre.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3890458},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}