The previously created table in Feature Store can be deleted this way:
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": { "byteLimit": 2048000, "rowLimit": 10000 }, "inputWidgets": {}, "nuid": "b6db8d70-d3b1-452c-9761-381b397e50bf", "showTitle": false, "title": "" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2023/11/22 16:27:16 WARNING databricks.feature_store._compute_client._compute_client: Deleting a feature table can lead to unexpected failures in upstream producers and downstream consumers (models, endpoints, and scheduled jobs).\n" ] } ], "source": [ "fs.drop_table(name='seaborn_db.taxidataset')" ] }, { "cell_type": "markdown", "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": {}, "inputWidgets": {}, "nuid": "6bd6a1c4-6664-4dc9-bf3c-1d430241c1bf", "showTitle": false, "title": "" } }, "source": [ "Alternatively, an empty table structure can be created in the Feature store to be populated later. The command below only creates the structure of the table in Feature Store:
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": { "byteLimit": 2048000, "rowLimit": 10000 }, "inputWidgets": {}, "nuid": "971f5cc3-aed0-451e-bff0-bc670bd31b35", "showTitle": false, "title": "" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2023/11/22 16:27:21 INFO databricks.feature_store._compute_client._compute_client: Created feature table 'hive_metastore.seaborn_db.taxidataset'.\n" ] } ], "source": [ "customer_feature_table = fs.create_table(\n", " name='hive_metastore.seaborn_db.taxidataset', # format isNext step is to write data in it:
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": { "byteLimit": 2048000, "rowLimit": 10000 }, "inputWidgets": {}, "nuid": "4611f26d-5325-4178-8531-34cda6e03145", "showTitle": false, "title": "" } }, "outputs": [], "source": [ "fs.write_table(\n", " df=taxis_sdf,\n", " name='seaborn_db.taxidataset',\n", " mode='merge' # mode = 'overwrite' could also be used in this particular case\n", ")" ] }, { "cell_type": "markdown", "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": {}, "inputWidgets": {}, "nuid": "34d0abe2-65cc-466d-9b39-95a695272449", "showTitle": false, "title": "" } }, "source": [ "Finally, feature store table can be read this way:
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "application/vnd.databricks.v1+cell": { "cellMetadata": { "byteLimit": 2048000, "rowLimit": 10000 }, "inputWidgets": {}, "nuid": "76b1d93a-94a0-4cac-b546-57e00ee6ce51", "showTitle": false, "title": "" } }, "outputs": [ { "data": { "text/html": [ "| pickup | dropoff | passengers | distance | fare | tip | tolls | total | color | payment | pickup_zone | dropoff_zone | pickup_borough | dropoff_borough | id |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2019-03-17 13:16:13 | 2019-03-17 13:40:32 | 1 | 2.9 | 17.0 | 0.0 | 0.0 | 20.3 | yellow | cash | Clinton East | Lenox Hill East | Manhattan | Manhattan | 0 |
| 2019-03-22 08:43:39 | 2019-03-22 08:51:49 | 1 | 0.55 | 6.5 | 1.96 | 0.0 | 11.76 | yellow | credit card | Union Sq | Murray Hill | Manhattan | Manhattan | 1 |
| 2019-03-02 21:42:07 | 2019-03-02 22:01:15 | 1 | 2.5 | 13.5 | 3.45 | 0.0 | 20.75 | yellow | credit card | SoHo | Murray Hill | Manhattan | Manhattan | 2 |
| 2019-03-13 21:49:00 | 2019-03-13 22:02:04 | 1 | 2.68 | 11.0 | 1.0 | 0.0 | 15.8 | yellow | credit card | East Chelsea | Alphabet City | Manhattan | Manhattan | 3 |
| 2019-03-03 12:55:38 | 2019-03-03 13:01:05 | 1 | 0.85 | 5.5 | 0.0 | 0.0 | 8.8 | yellow | cash | Midtown Center | Murray Hill | Manhattan | Manhattan | 4 |
![](\"https://i.ibb.co/Yy56SQz/unity-catalog-1.png\"/)
"
]
}
],
"metadata": {
"application/vnd.databricks.v1+notebook": {
"dashboards": [],
"language": "python",
"notebookMetadata": {
"mostRecentlyExecutedCommandWithImplicitDF": {
"commandId": 886449356054794,
"dataframes": [
"_sqldf"
]
},
"pythonIndentUnit": 2
},
"notebookName": "Databricks-ML-professional-S01a-Data-management",
"widgets": {}
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
}
},
"nbformat": 4,
"nbformat_minor": 4
}