Data Platform Engineer · SF Bay Area
Jordan Allen Lewis
Taking data platforms from proof of concept to production. Lately that's meant a federated lakehouse on Dremio, a Kafka streaming platform, and AI tooling, all on Kubernetes, backing a customer-facing product and about a dozen internal teams.
Selected work
Things I've built
- 01
Platform
Production data lakehouse on Dremio + Kubernetes
A production-critical service at 99.9% uptime. It powers a customer-facing SaaS product, and a dozen internal teams, data scientists, and engineers build on it directly, self-serve.
DremioApache IcebergKubernetesHelmHDFSSnowflakeGrafana - 02
Streaming
Kafka platform on Confluent for Kubernetes
Self-hosted, secured real-time data streaming, running in staging. The streamed data lands straight in the lakehouse, ready to query.
Apache KafkaConfluentKRaftOAuthDebeziumApache Iceberg
Writing
Latest thinking
I built my own data product: SteamBangers.com
SteamBangers is the first data product I own end to end, outside work: it scores every game on Steam 0 to 100 for value, the Bang Score. I built it for budget-conscious gamers, because I was one, and it runs on the same open-lakehouse instincts I bring to my day job.
Read more
Get new posts by email
Occasional, only when I publish something. Data platforms, lakehouses, streaming, and where AI earns its keep in real pipelines.
Let's talk.
Happy to talk data platforms, lakehouses, or where AI actually earns its keep in infrastructure. LinkedIn is the fastest way to reach me.