Databricks announced FILE, a new native column type now in beta that stores unstructured data—documents, images, audio, and video—directly in Delta Lake tables alongside structured metadata. Unlike traditional approaches that store files in separate object storage and lose governance context, FILE columns use lightweight pointers to files while applying Unity Catalog fine-grained access controls to rows and columns. When a row containing a FILE is deleted, the binary in object storage is also deleted, making GDPR compliance automatic.
Queries can run directly on FILE columns using standard SQL and Python UDFs, triggering file processing only when needed. Example: an autonomous-vehicle company stores dashcam videos as FILE columns; a Python UDF samples frames into a new FILE column, an object-detection model runs on those frames, and SQL joins the extracted hazards to structured trip metadata—all without heavy binary encoding in the table. Performance scales because the engine pulls bytes only at the step that requests them, not for every intermediate operation.
Databricks is working with the community to build FILE support directly into Parquet, Delta Lake, Iceberg, and Spark, keeping multimodal data portable across tools. For data teams building agent-assisted document analysis, visual inspection, call analytics, and video understanding, FILE closes the gap between unstructured assets and queryable, governed data estates—reducing the friction of training, validation, and retrieval pipelines.