Steps to Browse Azure Databricks File System in Databricks Workspace
Step 1
Firstly, Open Azure Databricks Workspace then go to profile section then go to settings.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 2
Once setting is opened next go to advanced option and go down.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 3
By default it is disable make sure to enable DBFS file browser.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 4
Next go to new click on add or upload data.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 5
Go down you get to see upload files to DBFS click on it.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 6
Make sure first refresh workspace after it is refreshed you get to see Databricks File System
(DBFS).
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 7
Now go to workspace users folder right click on gmail id next go to create then go to
notebook to create a new notebook.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 8
Make a new notebook name select language python and connect it to cluster.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 9
In the first cell write the code [Link] (dbutils is python library to access data from
DBFS) fs (attribute is for file system) and ls (for list of files or folders) inside it mention /
and enclose it with a single quote when you execute the code you see the list of folders
like -
• dbfs:/FileStore/
• dbfs:/databricks-datasets/
• Dbfs:/databricks-results/
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 10
Either you can use display function to see the list of folders in tabular format when you
execute it you see the list of folders in the table.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 11
Similarly, you can see the list of files inside the folder dbfs:/databricks-datasets/ with writing
the given code.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Steps to Add Data to DBFS From File Browser
Step 1
Click on new then go to add or upload data.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 2
Go down click on upload files to DBFS.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 3
Next click on drop files to upload.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 4
Next upload any .csv file to DBFS.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 5
After file is uploaded successfully it looks like this.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 6
Now go to DBFS file system next go to root folder FileStore then go to tables you will be able
to see uploaded files in DBFS.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Steps to Add Data to DBFS From Notebook
Step 1
Go back to notebook next go to File menu then go to upload data to DBFS.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 2
Next click on drop files to upload.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 3
Next select any file to upload to DBFS.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 4
After file is uploaded click on next button under access files from notebook select pyspark
for given code copy it next click on done.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 5
Open a new cell and paste pyspark code here and execute it.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer
Step 6
Your file is loaded into dataframe df1 open a new cell use the display function and pass df1
dataframe when you execute it you will be able to see the data.
Master Azure Databricks | Follow Me Praveen Patel | Azure Data Engineer