01
Flagship platform
Production data lakehouse on Dremio + Kubernetes
A production-critical service at 99.9% uptime that a dozen teams across the company query directly.
DremioApache IcebergKubernetesHelmHDFSSnowflakeGrafana
Read the write-upData Platform Engineer · SF Bay Area
Jordan Lewis
I build the infrastructure behind analytics.
I run a data lakehouse and a Kafka platform on Kubernetes, and build the tooling that keeps the data on them reliable and easy to find.
99.9%
Production lakehouse uptime
10+
Teams using the platform
30+
Data sources in one SQL engine
8
Platforms I run on Kubernetes
What I do
Platforms, end to end
I work across the whole stack that makes analytics possible: storage, query engines, streaming, identity, and governance.
Data lakehouse platforms
I run Dremio and an Apache Iceberg catalog on Kubernetes. One SQL engine queries HDFS, Hive, object storage, Snowflake, Postgres, and Mongo together, with a semantic layer, SSO, and query acceleration on top.
Reliability & operations
I own the platform end to end. Multi-environment CI/CD, on-call, incident response, HA/DR planning, and the dashboards that keep a production service at three nines.
Streaming & AI tooling
Kafka on Confluent for Kubernetes feeds the lakehouse. On top of it I build LLM and MCP tooling that documents a catalog and answers questions about it from your editor.
Selected work
Things I've built
- 02
Streaming
Kafka platform on Confluent for Kubernetes
Self-hosted streaming with end-to-end auth, running in staging. Topics land in the lakehouse as Iceberg tables.
Apache KafkaConfluentKRaftOAuthDebeziumApache IcebergRead the write-up - 03
AI / Platform
AI data steward + MCP server
Documents a data catalog on a daily run and answers catalog questions from an engineer's editor.
PythonLLMMCPApache IcebergSupersetPrometheusRead the write-up
Writing
Recent posts
Let's talk.
Happy to talk data platforms, lakehouses, or where AI actually earns its keep in infrastructure. LinkedIn is the fastest way to reach me.