01
Flagship platform
Production data lakehouse on Dremio + Kubernetes
A production-critical service at 99.9% uptime that a dozen teams across the company query directly.
DremioApache IcebergKubernetesHelmHDFSSnowflakeGrafana
View the projectData Platform Engineer · SF Bay Area
Jordan Lewis
I build the infrastructure behind analytics.
I run a data lakehouse and a Kafka platform on Kubernetes, and build the tooling that keeps the data on them reliable and easy to find.
99.9%
Production lakehouse uptime
10+
Teams using the platform
30+
Data sources in one SQL engine
8
Platforms I run on Kubernetes
What I do
Platforms, end to end
I work across the whole stack that makes analytics possible: storage, query engines, streaming, identity, and governance.
Data lakehouse platforms
I run Dremio and an Apache Iceberg catalog on Kubernetes. One SQL engine queries HDFS, Hive, object storage, Snowflake, Postgres, and Mongo together, with a semantic layer, SSO, and query acceleration on top.
Reliability & operations
I own the platform end to end. Multi-environment CI/CD, on-call, incident response, HA/DR planning, and the dashboards that keep a production service at three nines.
Streaming & AI tooling
Kafka on Confluent for Kubernetes feeds the lakehouse. On top of it I build LLM and MCP tooling that documents a catalog and answers questions about it from your editor.
Selected work
Things I've built
- 02
Streaming
Kafka platform on Confluent for Kubernetes
Self-hosted streaming with end-to-end auth, running in staging. Topics land in the lakehouse as Iceberg tables.
Apache KafkaConfluentKRaftOAuthDebeziumApache IcebergView the project - 03
AI / Platform
AI data steward + MCP server
Documents a data catalog on a daily run and answers catalog questions from an engineer's editor.
PythonLLMMCPApache IcebergSupersetPrometheusView the project
Writing
Recent posts
Let's talk.
Happy to talk data platforms, lakehouses, or where AI actually earns its keep in infrastructure. LinkedIn is the fastest way to reach me.