Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Preswald: Real-time DuckDB dashboards
This talk covers building real-time dashboards using DuckDB and Preswald’s Python API, focusing on in-memory SQL queries, data transformation, and integration with Pandas and Plotly.
This talk explores the technical mechanics of leveraging DuckDB, an in-memory OLAP database, to build real-time dashboards using Preswald, our open-source Python framework. Preswald combines DuckDB’s high-performance query engine with a declarative API, enabling developers to create fast, lightweight, code-first dashboards with full control over their analytics stack.
- In-memory query execution: DuckDB operates entirely in-memory, processing complex SQL queries directly on flat data files (CSV, Parquet, or raw streams) without the need for databases, data warehouses, or ETL pipelines.
- Real-time data transformation: Learn how DuckDB enables on-the-fly data transformations with SQL-like operations that run natively on your data files, avoiding unnecessary data movement or duplication.
- Design trade-offs: Preswald’s declarative Python API tightly integrates with tools like Pandas and Plotly, giving devs more control over data transformation, queries, schedules, and visualization workflows.
From an engineering standpoint, we decided to build Preswald around DuckDB because it let us rethink how analytics pipelines should work. Traditional setups rely on data warehouses, ETL pipelines, and distributed databases, which add a lot of complexity, latency, and maintenance overhead. These systems often require moving data around constantly, which slows things down and makes them harder to manage.
DuckDB stood out because of its in-memory, columnar database engine. It’s optimized for vectorized execution and can run complex SQL queries directly on flat files like CSVs and Parquet without needing a separate database server or moving data to another system. This drastically reduces I/O overhead and eliminates the need for extra infrastructure. We saw this as the foundation for building a faster, more lightweight approach to analytics.
To make it easier for developers to use DuckDB, we built Preswald with a declarative Python API. It lets developers define their data workflows without having to write raw SQL, while still taking full advantage of DuckDB’s performance. We designed the API to integrate with familiar tools like Pandas for data manipulation and Plotly for visualizations. Under the hood, Preswald uses DuckDB’s vectorized query engine to handle joins, aggregations, and transformations efficiently.
A big challenge was balancing performance and flexibility. DuckDB is incredibly fast in-memory, but it can also process data directly from disk for larger datasets. We built Preswald to handle both cases, allowing small, fast prototypes in memory and scaling up to larger workflows without adding extra dependencies or infrastructure.
Pyodide/DuckDB WASM packages Python data apps for offline browser execution.
Preswald converts Python/DuckDB scripts into reactive, zero-JavaScript, statically deployed web apps.
- DuckDBDuckDB is the high-performance, open-source, in-process analytical data management system (OLAP) that runs complex SQL queries directly on your data files.DuckDB is a fast, embedded analytical RDBMS: think SQLite, but optimized for OLAP workloads. It operates in-process—no separate server required—and boasts zero external dependencies, making deployment simple. The system uses a vectorized, column-oriented architecture for blazing-fast query execution on large datasets. It supports standard SQL, integrates seamlessly with languages like Python and R, and can query data directly from formats like Parquet, CSV, and JSON, eliminating ETL overhead. With over 25 million downloads per month and adoption by 20+ Fortune-100 companies, DuckDB is the go-to tool for local, high-speed data analysis.
- PreswaldPreswald is the Python framework that converts data scripts into fast, interactive web apps and dashboards with zero frontend code.Preswald is a powerful, open-source Python framework designed to bridge the gap between raw data analysis and production-ready applications. It allows data scientists and analysts to transform existing Python scripts into fully interactive data apps, dashboards, and internal tools without writing any JavaScript (JS). The core architecture utilizes a Directed Acyclic Graph (DAG) execution model for smart dependency tracking, ensuring only necessary components update for instant, reactive performance. It packages the full data stack—including Pyodide, DuckDB, and Plotly—to run entirely in the browser, enabling static-site deployment and offline functionality. Deployment is a one-command operation, making it efficient for teams to share sophisticated tools, like financial reports or custom analytics, instantly.
- PandasPandas is the core Python library for high-performance data manipulation and analysis: it introduces the DataFrame and Series structures for fast, flexible handling of labeled data.Pandas is your go-to, open-source Python library for data science, engineered for efficient data manipulation and analysis. It centers on two primary data structures: the two-dimensional DataFrame (like a spreadsheet or SQL table) and the one-dimensional Series. This toolkit handles everything from loading diverse file formats (CSV, Excel, JSON) to complex operations: data cleaning, transformation, statistical analysis, and time series processing. Developed by Wes McKinney in 2008, Pandas builds directly on NumPy and provides an intuitive API, making it the industry standard for turning raw data into actionable insights with minimal code.
- PlotlyThe leading open-source library for creating interactive, publication-quality data visualizations and web-based analytic applications.Plotly is a technical computing company delivering powerful, interactive data visualization tools built on its core open-source library, Plotly.js. The platform supports multiple languages (Python, R, MATLAB) through wrappers like Plotly.py and Plotly.R. Its standout product is Dash: a framework for building full-stack analytic web applications entirely in Python, R, or Julia. This allows data science teams to deploy complex visualizations—from simple scatter plots to advanced 3D charts—directly to production environments, moving beyond static reports to deliver scalable data apps for Fortune 500 clients and millions of users.
- PythonPython: The high-level, general-purpose language built for readability, powering everything from web backends to advanced machine learning models.Python is the high-level, general-purpose language prioritizing clear, readable syntax (via significant indentation), ensuring rapid development for any team . Its ecosystem is massive: use it for robust web development with frameworks like Django and Flask, or leverage its power in data science with libraries such as Pandas and NumPy . The Python Package Index (PyPI) provides thousands of community-contributed modules, offering immediate solutions for tasks from network programming to GUI creation . The language is actively maintained by the Python Software Foundation (PSF), with the stable release currently at Python 3.14.0 (as of November 2025) .
Related projects
Standd - Data Visualization for Deal Diligence
DC
We'll demonstrate how Standd's AI agent detects data visualization opportunities in deal diligence and automatically generates charts to…
The Bitter Lesson - Compute vs Human Knowledge in AI Systems
New York City
Exploring how increasing computational power consistently outperforms handcrafted AI knowledge, with case studies and implications for future AI…
Coach Amelia
New York City
Explore how an all‑in‑one AI wellness coach uses GPT‑4 vision to deliver personalized nutrition guidance, demonstrating practical real‑world…
SecretAIry
San Diego
See how an AI gatekeeper screens calls, remembers callers, and develops unique relationships to manage phone anxiety through…
AI Startup Scout
Seattle
A sub‑agent scans Hacker News, flags potential startup stories, adds founder data from external sources, and generates concise…
A smart file browser
Bengaluru
This talk covers a desktop file browser with smart folder triggers, AI file creation, and offline TTS/STT using…