Document Intelligence Engineer

We're looking for a mid-level IDP Engineer to join Lentra's Document Intelligence team. You'll design, build, and maintain end-to-end document processing pipelines that power critical workflows in the BFSI space, including KYC verification, loan application processing, bank statement analysis, invoice extraction, and check processing. This is a hands-on, high-impact individual contributor role. You'll work across the full pipeline from document ingestion and OCR to layout understanding, extraction, validation, and downstream integration, and you'll be expected to take features from prototype to production independently. As the team grows, you'll also have the opportunity to guide junior engineers and contribute to the team's technical direction.

Responsibilities:

• Architect and own end-to-end IDP pipelines, document ingestion, classification, layout analysis, field extraction, validation, and structured output serving production traffic at scale.
• Develop, train, and deploy deep learning models using PyTorch and/or TensorFlow for tasks like document classification, key-value extraction, table detection, and signature/stamp recognition.
• Implement and fine-tune Vision Language Models (LayoutLMv2/v3 Donut, Pix2Struct, DiT, or similar) for complex document understanding handling semi-structured and unstructured formats across varying templates.
• Fine-tune LLMs and VLMs using parameter-efficient techniques (LoRA, QLoRA, PEFT) for domain-specific tasks including dataset curation, annotation strategy, hyperparameter optimization, evaluation framework design, and deployment of fine-tuned checkpoints.
• Build robust OCR pipelines with pre-processing (deskewing, binarization, noise removal, layout segmentation) and post-processing (confidence thresholding, spell correction, structured output formatting) using open-source engines.
• Design model-serving infrastructure batching strategies, latency optimization, GPU/CPU inference tradeoffs, model versioning, and A/B testing of model variants
• Handle BFSI document edge cases: poor scan quality, handwritten text, multilingual content, stamp/seal interference, and multi-page documents with complex layouts.
• Collaborate with product, backend, and data engineering teams to integrate IDP outputs into downstream business systems via APIs and event-driven architectures.

Requirements:

• 3-6 years of hands-on experience in ML, Computer Vision, or Document Processing (exceptional candidates with 2+ years will be considered).
• Deep Learning Frameworks: Production experience building, training, and deploying models in PyTorch and/or TensorFlow, including custom model architectures, training loops, data loaders, and mixed-precision training.
• Using scikit-learn, XGBoost, or calling pre-trained model APIs does not meet this requirement. IDP Systems: Proven track record of building end-to-end IDP pipelines deployed in production, not just PoCs. Comfortable with the full lifecycle: ingestion, pre-processing, model inference, post-processing, validation, and integration.
• General backend/data pipeline experience or working with clean, structured data is not equivalent. We mean pipelines that process real scanned/photographed documents.
• Vision Language Models: Hands-on experience applying VLMs (LayoutLM variants, Donut, Pix2Struct, LLaVA, or similar) to real document understanding tasks, layout analysis, table extraction, form parsing, or key-value extraction.
• General NLP model experience (BERT for text classification, GPT for chatbots) does not qualify. We need document layout model experience specifically.
• LLM/VLM Fine-tuning: Demonstrated experience fine-tuning large models, including dataset preparation and quality control, training with LoRA/QLoRA/PEFT, evaluation metric design, ablation studies, and deploying fine-tuned models to production.
• Prompt engineering, RAG setups, or using LLM APIs is not fine-tuning. We mean modifying model weights with your own training data.
• Open-Source OCR: Hands-on experience with OCR engines (Tesseract, PaddleOCR, EasyOCR, and TrOCR) in production settings, including custom training, language pack configuration, accuracy benchmarking, and pre/post-processing pipelines.
• Using Google Vision API, AWS Textract, or Azure Form Recognizer is not an open-source OCR experience.
• Strong Python skills with production-quality code practices (testing, logging, error handling, documentation).
• Familiarity with model evaluation methodologies, precision/recall at the field level, end-to-end accuracy metrics, and human-in-the-loop validation workflows.

Nice-to-Have:

• Prior work in Fintech, Banking, Insurance, or Financial Services; familiarity with document types like KYC forms, PAN/Aadhaar, loan applications, bank statements, NACH mandates, insurance claims, and checks.
• Experience with AWS (SageMaker, Lambda, S3 ECR, Step Functions) or equivalent cloud ML infrastructure.
• MLOps exposure model registry, experiment tracking (MLflow/W& B), CI/CD for ML, drift monitoring, and automated retraining.
• Understanding of data privacy and compliance requirements in BFSI (PII handling, data masking, and audit trails).
• Publications, open-source contributions, or patents in document AI, OCR, or related fields.

Skills

Donut, IDP, LLaVA, LayoutLM, OCR, Pix2Struct, PyTorch, TensorFlow, VLM, document