Open to Opportunities

Somesh Ghaturle

Data & AI Engineer  ·  Databricks  ·  Snowflake  ·  Ontology  ·  Semantic Layers  ·  Lakehouse Federation

New York, NY

Data and AI engineer with roughly five years building lakehouse and warehouse platforms on Databricks and Snowflake and, since 2025, production AI systems on top of them: governed data products, ontologies, semantic layers, and LangGraph agents that reason over trusted data.

10TB+
Migrated to Snowflake
20+
Production Data & AI Services
15M+
Documents Processed

Who I Am

// somesh.config
role: "Data & AI Engineer"
company: "Pyramid Consulting"
platforms: "Databricks · Snowflake"
modeling: "Ontology · Semantic Layers"
governance: "Unity Catalog · Federation"
ai: "LangGraph · RAG · MCP"
cloud: "Azure · AWS"
degree: "MS Data Science, Pace (4.0)"

Governed Data, Trustworthy AI

I'm a data and AI engineer with roughly five years across consulting, enterprise, and founder roles, building lakehouse and warehouse platforms on Databricks and Snowflake and, since 2025, production AI systems on top of them.

I work directly with client and business stakeholders to turn ambiguous problems into governed data products and AI solutions. I design ontologies that model business entities and relationships, build semantic layers that give every team one governed definition of each metric, and use Lakehouse Federation to query external sources in place without copying data.

I pair Unity Catalog governance with LangGraph agents and RAG that reason over trusted, well-modeled data, and I've delivered across Azure and AWS, from a 10TB+ Hadoop-to-Snowflake migration to 20+ production AI services. My research project FAIR-Agent, presented at SARD 2025, improved faithfulness by 17.4%.

Lakehouse & Warehouse Platforms
Ontology & Semantic Layers
Unity Catalog Governance
AI on Trusted Data

Where I've Worked

Metric-driven impact across enterprise AI, data science, and data engineering.

Data & AI Engineer | Databricks, Unity Catalog, Semantic Layers
Pyramid Consulting, Inc.
Full-time  ·  Feb 2026 — Present
Remote, United States
Feb 2026 — Present
  • Lakehouse for Document Intelligence: Built the Databricks lakehouse behind an LLM document intelligence system on Azure Databricks: PySpark and SQL pipelines into governed Delta tables, refreshed with Databricks Workflows, processing 15M+ enterprise documents and reducing manual review 35%.
  • Lakehouse Federation: Configured Lakehouse Federation to query external operational databases and Snowflake directly from Databricks under Unity Catalog, removing duplicate copy pipelines and giving analysts and agents one governed access point.
  • Semantic Layer: Built a semantic layer with Unity Catalog metric views, defining shared business metrics, dimensions, and joins once so dashboards, SQL users, and AI agents all read the same definitions.
  • Ontology for Agents: Designed an ontology of business entities, attributes, and relationships that grounds LangGraph multi-agent workflows and RAG over Databricks Vector Search, so agents resolve terms consistently, cutting manual analyst effort about 40%.
  • Governance & Delivery: Governed the platform with Unity Catalog access control, lineage, and Lakehouse Monitoring, and shipped 20+ production data and AI services with MLflow and CI/CD, taking deployment cycles from two days to six hours.
  • Dataiku: Built Dataiku data preparation flows and recipes and governed LLM access through Dataiku LLM Mesh for analyst teams, and documented models, definitions, and pipelines for knowledge transfer.
Azure Databricks Unity Catalog Lakehouse Federation Metric Views Delta Lake PySpark Databricks Workflows Vector Search LangGraph MLflow Dataiku
AI & Data Analytics Co-op | Ontology, LLM Agents, AWS Glue
BMW Group
Co-op  ·  Jun 2025 — Dec 2025
Woodcliff Lake, New Jersey, United States
Jun 2025 — Dec 2025
  • Ontology for GAIA: Modeled an ontology of vehicle, workshop, and operations entities and their relationships to structure the enterprise data behind GAIA, BMW's AI platform, so LLM agents could answer business questions with consistent definitions, cutting manual query time about 40%.
  • Agents & Inference: Wrote the prompt templates and multi-agent handoff logic behind those agents, and hardened the Linux-based SageMaker inference path in AWS VPCs so new models shipped without bespoke setup.
  • Pipelines & Dashboards: Owned AWS Glue (PySpark) and S3 pipelines ingesting structured and unstructured data, cutting latency 35% with job bookmarking, partition pruning, and schema evolution handling, and built Tableau dashboards on the modeled data as part of a 7-person Agile squad.
Ontology LLM Agents SageMaker AWS VPC AWS Glue PySpark S3 Tableau
Data Engineer I | Snowflake, Azure Data Factory, Spark, Synapse
New Balance
Full-time  ·  Apr 2022 — Dec 2023
Pune, India
Apr 2022 — Dec 2023
  • Snowflake Warehouse: Rebuilt the data warehouse on Snowflake, migrating 10TB+ of legacy ETL from Hadoop with Azure Data Factory, Spark 3.0, and Azure Synapse, with CI/CD automation, cutting processing time roughly 40%.
  • Semantic Layer: Built a Snowflake semantic layer of dimensional models, conformed dimensions, and governed metric views so analytics teams worked from one definition of each KPI, improving SLA compliance for downstream reporting.
  • Performance & Quality: Tuned Snowflake performance and cost by rewriting the slowest SQL and calibrating event caching, cutting database load 26%, and built validation and cross-source reconciliation that halved errors.
  • Documentation: Documented data models and metric definitions for analysts, and onboarded new engineers.
Snowflake Azure Data Factory Spark 3.0 Azure Synapse Hadoop Dimensional Modeling CI/CD
Graduate Data Engineer | Python, SQL, Azure, PostgreSQL
Kagool Data Private Limited
Full-time  ·  Sep 2020 — Mar 2022
Hyderabad, India
Sep 2020 — Mar 2022
  • Healthcare ETL: Built Python, SQL, and Azure ETL pipelines on PostgreSQL and Hadoop, including HIPAA-compliant migration for a Walgreens healthcare project, and tuned queries with spatial, bitmap, and GIN indexes for roughly 30% faster fetches.
  • Data-Driven Action Plans: Turned analysis into data-driven action plans for business teams, contributing to a 15% reduction in operational costs.
Python SQL Azure PostgreSQL Hadoop HIPAA
Founder | Startup
Trust Control Center
Founder  ·  Jan 2019 — Aug 2020
Startup
Jan 2019 — Aug 2020
  • Product: Founded and built a Django-based platform that detects, visualizes, and resolves cross-source metric inconsistencies, the root cause of unreliable analytics and AI predictions across enterprise data sources.
  • Metric Logic: Designed the metric definitions and cross-source reconciliation logic at the product's core, and owned it end to end from customer discovery to build.
Django Python Metric Definitions Reconciliation

Featured Work

Complex systems that demonstrate the complete AI lifecycle — architecture, scale, and business impact.

Skills & Technologies

The modern stack for building production AI systems and enterprise data infrastructure.

Databricks

The governed lakehouse.

Unity Catalog Lineage Metric Views Lakehouse Federation Delta Lake Databricks SQL Workflows Vector Search Lakehouse Monitoring MLflow PySpark

Semantic & Knowledge Modeling

One definition of every entity and metric.

Ontology Design Knowledge Graphs Semantic Layers Metric Definitions Dimensional Modeling Conformed Dimensions Data Contracts

Snowflake & Warehousing

Warehouses and the pipelines that feed them.

Snowflake Data Modeling Tasks Stored Procedures Performance Tuning Cortex Search Azure Synapse PostgreSQL ETL/ELT Azure Data Factory AWS Glue Airflow Hadoop

AI & Engineering

AI that reasons over trusted data.

LangGraph LangChain RAG A2A MCP Python SQL FastAPI Docker Git CI/CD Terraform Tableau Power BI

Architecture Overview

How the autonomous multi-agent system is designed end-to-end.

📄

Data Ingestion

Upload PDFs, financial reports, legal docs. Chunk & embed with transformers.

PyPDF LangChain Transformers
🗄️

Vector Memory

Embedded chunks stored for fast similarity search and context retrieval.

Qdrant Pinecone HNSW
🧠

Agent Orchestrator

Manages cyclic agent workflows, state, routing, and tool calls.

LangGraph State Machine Tools
🔍

Researcher Agent

Queries vector DB, retrieves context, and pulls real-time web data.

RAG Web Search SQL
🛡️

Critique Agent

Validates for hallucinations, factual accuracy, and source attribution.

Fact Check Verify Score
✍️

Writer Agent

Formats findings into reports, emails, or code — then executes actions.

Reports Email API Code Gen

Education & Certifications

M.S. Data Science — Machine Learning & AI concentration, GPA 4.0
Pace University, New York, NY
Jan 2024 — Dec 2025
B.E. Computer Science
Nagpur University, Wardha, India
July 2019

Certifications

Databricks for Generative AI AWS Databricks Platform Architect Azure AZ-900 Azure DP-900 Azure AI-900 Azure PL-900

Let's Connect

Open to Data & AI Engineer roles across Databricks, Snowflake, and production AI. Feel free to reach out.