Databricks magic commands

 Magic commands in databricks notebook 

We create a databricks notebook with a default language like SQL, SCALA or PYTHON and then we write codes in cells. Databricks notebooks allows us to write non executable instructions or also gives us ability to show charts or graphs for structured data. 

Databricks gives ability to change language of a specific cell or interact with the file system commands with the help of few commands and these are called magic commands. 

What are these magic commands in databricks ?

%fs 

This command allows us to write file system commands in a cell after writing the above command. Lets say we have created a notebook with python as default language but we can use the below code in a cell and execute file system command.

%fs

ls mnt/data

%Python

In case if you have selected default language other than python but you want to execute a specific python code then you can use %Python as first line in the cell and write down your python code below that.

%md

It is called markdown and specifically used to write comment or documentation inside the notebook to explain what kind of code we are writing.

example

%md

<a href="$./myNotebook">Link to notebook in same folder as current notebook</a>

<a href="$../myFolder">Link to folder in parent folder of current notebook</a>

<a href="$./myFolder2/myNotebook2">Link to nested notebook</a>

%sh

%sh is used as first line of the cell if we are planning to write some shell command.

%scala

similar to python you can write %scala and write the scala code.

%r

you can use R code in a cell with this magic command.

%sql

If you are using python/scala notebook and have a dataframe, you can create a temp view from the dataframe and use %sql command to access and query the view using SQL query