Data Platform Engineer · SF Bay Area
Jordan Allen Lewis
I take data platforms from proof of concept to production. Right now that's a data lakehouse on Kubernetes: one query engine over 40 data sources, backing a customer-facing product and about a dozen internal teams.
The platform I run, by the numbers
Selected work
Things I've built
- 01
Platform
Production data lakehouse on Dremio + Kubernetes
A production-critical service at 99.9% uptime. It powers a customer-facing SaaS product, and a dozen internal teams, data scientists, and engineers build on it directly, self-serve.
DremioApache IcebergKubernetesHelmHDFSSnowflakeGrafana - 02
Streaming
Kafka platform on Confluent for Kubernetes
Self-hosted, secured real-time data streaming, running in staging. The streamed data lands straight in the lakehouse, ready to query.
Apache KafkaConfluentKRaftOAuthDebeziumApache Iceberg
Writing
Latest thinking
Data Engineering Is Going Full-Stack
A Data Engineering Open Forum keynote by Airbnb's Jerry Wang argued that AI is collapsing the data stack into one role: the full-stack data engineer who owns the whole lifecycle. Looking at my own work, that shift already happened to me.
Read more
Get new posts by email
Occasional, only when I publish something. Data platforms, lakehouses, streaming, and where AI earns its keep in real pipelines.
Let's talk.
Happy to talk data platforms, lakehouses, or where AI actually earns its keep in infrastructure. LinkedIn is the fastest way to reach me.