【DSW Gallery】Jupyter magic command usage skills

Tips for using Jupyter magic commands
In addition to being able to execute Python code, Jupyter Notebook also provides some magic commands (Magic Command) to facilitate users to solve various common problems in standard data analysis concisely. Magic commands come in two forms:
• line magic is denoted by a single % prefix and operates on a single line of input
• cell magic is denoted by a double %% prefix and operates on multi-line input
There are many kinds of magic commands, we can view all available magic commands through the %lsmagic command:
%ls magic
Available line magics:
%alias %alias_magic %autoawait %autocall %automagic %autosave %bookmark %cat %cd %clear %colors %conda %config %connect_info %cp %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history %killbgscripts %ldir %less %lf %lk %ll %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %lx %macro %magic %man %matplotlib %mkdir %more %mv %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %pip %popd %pprint %precision %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %rm %rmdir %run %save %sc %set_env %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode
Available cell magics:
%%! %%HTML %%SVG %%bash %%capture %%debug %%file %%html %%javascript %%js %%latex %%markdown %%perl %%prun %%pypy %%python %% python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile
Automagic is ON, % prefix IS NOT needed for line magics.
This article will introduce several common magic commands, and you can explore other magic commands by yourself
File operations using magic commands
You can export the contents of the cell to a file by adding the %%writefile command before the code
%%writefile script.py
import numpy as np
def random_nparray():
print(np. random. rand(3,2))
random_nparray()
Writing script.py
Use %pycat to display the contents of the specified file
%pycat script.py
import numpy as np
def random_nparray():
print(np. random. rand(3,2))
random_nparray()
Use the %run magic command to execute python files in Jupyter Notebook
%run script.py
[[0.40436084 0.71671436]
[0.40652354 0.14977982]
[0.40079174 0.74159821]]
variable management
Use the %who, %who_ls, %whos commands to view the variables and their information defined in the current Jupyter Notebook, the difference lies in the level of detail displayed
%who_ls
['np', 'random_nparray']
%who
np random_nparray
%whos
Variable Type Data/Info
-----------------------------------------
np module kages/numpy/__init__.py'>
random_nparray function
You can use %xdel to delete the specified variable
%xdel np
%who
random_nparray
Use %reset to clear all variables in the current environment
%reset
%who
Interactive namespace is empty.
Kernel environment package management
Use the %pip magic command to manage packages in the Python virtual environment corresponding to the currently running Kernel.
After using %pip install to install a new Python package, you need to restart the current Kernel to take effect.
%pip install tdqm
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting tdqm
Downloading https://mirrors.aliyun.com/pypi/packages/5a/38/58c9e22b95e98666fe29f35e217ab6126a03798dadf3f4f8f3e5f898510b/tdqm-0.0.1.tar.gz (1.4 kB)
Preparing metadata (setup.py) ... ?25ldone
?25hRequirement already satisfied: tqdm in /home/pai/lib/python3.6/site-packages (from tdqm) (4.64.0)
Requirement already satisfied: importlib-resources in /home/pai/lib/python3.6/site-packages (from tqdm->tdqm) (5.4.0)
Requirement already satisfied: zipp>=3.1.0 in /home/pai/lib/python3.6/site-packages (from importlib-resources->tqdm->tdqm) (3.6.0)
Building wheels for collected packages: tdqm
Building wheel for tdqm (setup.py) ... ?25ldone
?25h Created wheel for tdqm: filename=tdqm-0.0.1-py3-none-any.whl size=1319 sha256=a08573330ae5b0310db3fda329c82cbbff6807652542c76933111066cf215eeb
Stored in directory: /root/.cache/pip/wheels/64/bb/39/82c59ab975b19dc6cc2f99bb6034b69cbbae2f130ac112471d
Successfully built tdqm
Installing collected packages: tdqm
Successfully installed tdqm-0.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behavior with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Note: you may need to restart the kernel to use updated packages.
%pip show tdqm
Name: tdqm
Version: 0.0.1
Summary: Alias for typos of tqdm
Home-page: https://github.com/tqdm/tqdm
Author:
Author-email:
License: MPLv2.0
Location: /home/pai/lib/python3.6/site-packages
Requires: tqdm
Required-by:
Note: you may need to restart the kernel to use updated packages.
execution time
Use the %time command to record the execution time of a function
import numpy as np
%time np.random.normal(size=10)
CPU times: user 527 µs, sys: 135 µs, total: 662 µs
Wall time: 634 µs
array([ 0.15190477, -0.13989631, -1.94123265, 0.76655039, -1.70196434,
-0.21320566, -0.30997543, 1.34570534, -0.37875918, 0.30126323])
Use the %timeit command to evaluate the execution speed of the code through multiple runs, and generate the average + standard deviation of the execution time. Let us give an example:
import numpy as np
%timeit np.random.normal(size=1000)
42.3 µs ± 919 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
This command is useful when you want to determine the stability of code execution and looping processes.
%pinfo
In the code snippet below, we can initialize a simple object and then use %pinfo to get its details.
val = 100
%pinfo val
Type: int
String form: 100
Docstring:
int(x=0) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments
are given. If x is a number, return x.__int__(). For floating point
numbers, this truncates towards zero.
If x is not a number or if base is given,