Sai Jahnavi Kolli Get in touch
Sai Jahnavi Kolli

Sai Jahnavi
Kolli

Data Engineer, open to roles across the United States

I turn complex, scattered data into reliable, analytics-ready platforms that teams trust to make decisions. I build ETL/ELT pipelines in Python, SQL, and PySpark, run them on Spark and Snowflake, and ship them across AWS and Azure.

Email me LinkedIn GitHub
How I move data. Select a stage to see what I built there.

Select any stage above to see the work behind it.

Work

Show where I've used

  1. Feb 2025 – Present Remote

    Data Engineer, GM Financial

    Own production pipelines for automotive finance data behind portfolio performance reporting, risk analysis, and customer insights.

    • Build ETL/ELT pipelines in Python, PySpark, and SQL, loading curated data into Snowflake with Snowpipe, Streams, and Tasks for incremental (CDC) processing.
    • Improved Spark processing performance by 35% through query restructuring and transformation tuning.
    • Built dimensional data models and optimized Snowflake storage and compute for reporting and business intelligence.
    • Run Git-based CI/CD and automated testing for pipeline releases, and handle production support and data-quality issues.
  2. Jan 2022 – Jun 2024 Chennai, India

    Data Engineer, Reliance Industries

    Built enterprise data workflows for manufacturing and supply chain analytics that fed KPI-driven decisions.

    • Built Python, SQL, and Spark-based ETL/ELT pipelines integrating operational and supply chain data.
    • Automated recurring data workflows with orchestration and scheduling tools, cutting manual effort by 50% and improving production reliability.
    • Designed data models and warehouse solutions for enterprise analytics, including data migration and governance standards.
    • Ran data validation, root-cause analysis, and production support to keep data accurate and reliable.

Projects

Snowflake incremental ELT (CDC)

Files land in S3 and Snowpipe loads them as they arrive. Streams capture exactly what changed, and a Task graph merges only those changes into current-state tables, SCD Type 2 history and a daily mart, with no full reloads. Tested end to end against late, duplicate and invalid data, including tasks that fail mid-run.

Snowflake (Snowpipe, Streams, Tasks), SQL, Python, AWS S3, GitHub Actions

Code on GitHub

Multi-cloud ingestion framework

One config-driven framework ingests REST APIs, databases and vendor file drops into S3 and ADLS Gen2, checks every batch against a data contract, and merges it into Snowflake. Survives crashes without duplicates; orchestrated by Azure Data Factory, with infrastructure in Terraform.

Python, AWS S3/IAM, Azure Data Factory, ADLS Gen2, Snowflake, Terraform, Docker

Code on GitHub

EV battery-health lakehouse on Apache Iceberg

Connected-vehicle telemetry flows into Iceberg tables that score the residual-value risk of leased EVs from battery health. Uses upserts, schema and partition evolution, time travel to past reports, and compaction and snapshot expiry to keep it fast.

Apache Iceberg, PyIceberg, DuckDB, Parquet, AWS Glue, Snowflake

Code on GitHub

RAG ingestion pipeline for an AI assistant

The data side of retrieval-augmented generation: syncs a changing policy library incrementally, redacts personal data, chunks and embeds only what changed, and serves hybrid vector and keyword search. A retrieval-accuracy check runs in CI on every change.

Python, embeddings, vector and BM25 search, DuckDB, pgvector

Code on GitHub

Real-time fraud detection

Card transactions stream through Kafka into Spark Structured Streaming, where windowed rules and stream-to-stream joins flag fraud in about four seconds, written exactly once to Delta Lake and tested by crashing the job mid-stream.

Kafka, Spark Structured Streaming, Delta Lake, Python, Docker

Code on GitHub

Auto loan risk analytics

Raw loan servicing data cleaned in PySpark and modeled with dbt into a tested star schema, with vintage, delinquency and dealer-risk marts on Snowflake, orchestrated by Airflow.

PySpark, dbt, Snowflake, Airflow, SQL

Code on GitHub

Medallion CDC pipeline

Change events from an operational database flow through bronze, silver and gold layers with idempotent loads, SCD Type 2 history and a quality gate, then reconcile with the source row for row.

Python, SQL, DuckDB, Airflow

Code on GitHub

DataWhisperer

A browser-based CSV profiler: drop in a file to see types, missing values, distributions and data-quality issues, then export generated dbt tests and Snowflake or Postgres DDL. Nothing leaves the browser.

TypeScript, DuckDB-WASM, SQL, Vite, Playwright

Code on GitHub Live demo

Toolkit

Grouped by where each tool sits in a pipeline.

Ingest & stream
Snowpipe, Azure Data Factory, Apache Kafka, data loading
Process
Python, PySpark, Apache Spark, Spark SQL, Azure Databricks, Bash
Model & store
Snowflake (Streams, Tasks, CDC), dimensional modeling, data warehousing, data migration, data governance, PostgreSQL, MySQL
Orchestrate & ship
Apache Airflow, Git, CI/CD, automated testing, data quality and validation
Cloud
AWS (S3, EC2, Redshift), Azure Synapse, Microsoft Fabric
Serve
Power BI, Tableau, KPI dashboards

Credentials

Certifications

  • AWS Certified Solutions ArchitectAmazon Web Services
  • Microsoft Certified: Azure Data FundamentalsMicrosoft
  • Google Data Analytics Professional CertificateGoogle

Education

  • M.S., Information TechnologyUniversity of Cincinnati
  • B.Tech., Computer Science and TechnologySpecialization in Computer Networking, SRM Institute of Science and Technology

Let's talk

Hiring for a data engineering role, or want to compare notes on pipelines? Send a message, or email me directly at kollijahnavi07@gmail.com.