Python3 Module

In the previous chapters, we basically used the Python interpreter for programming. If you exit the Python interpreter and enter again, all the methods and variables you defined will disappear.

To solve this, Python provides a way to store these definitions in a file for use by scripts or interactive interpreter instances. This file is called a module.

A module in Python is a file containing Python definitions and statements. The file name is the module name plus the.pysuffix.

A module can contain functions, classes, variables, and executable code. Through modules, we can organize code into reusable units for easier management and maintenance.

The Role of Modules

• Code reuse: Encapsulate common functionality into modules, which can be reused across multiple programs.

• Namespace management: Modules can avoid naming conflicts; functions or variables with the same name in different modules do not interfere with each other.

• Code organization: Divide code into different modules by function, making the program structure clearer.

The following is an example of using a module from the Python standard library.

The execution result is as follows:

$ python using_sys.py 参数1 参数2
命令行参数如下:
using_sys.py
参数1
参数2


Python 路径为： ['/root', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages'] 

• 1. import sys imports the sys.py module from the Python standard library; this is a way to import a module.
• 2. sys.argv is a list containing command line arguments.
• 3. sys.path contains a list of paths where the Python interpreter automatically looks for required modules.

---

import statement

To use a Python source file, simply execute an import statement in another source file. The syntax is as follows:

import module1[, module2[,... moduleN]

When the interpreter encounters an import statement, the module will be imported if it is in the current search path.

The search path is a list of all directories that the interpreter will search. If you want to import the support module, you need to put the command at the top of the script:

test.py imports the support module:

The output of the above example is:

$ python3 test.py 
Hello :  Example

Download code

A module is only imported once, no matter how many times you executeimport. This prevents the imported module from being executed over and over again.

When we use the import statement, how does the Python interpreter find the corresponding file?

This involves Python's search path. The search path consists of a series of directory names, and the Python interpreter looks for the imported module in these directories in turn.

Module Search Path

When importing a module, Python searches for the module in the following order:

1. The current directory.

2. Environment variablePYTHONPATHSpecified directory.

3. Python standard library directory.

4. .pthDirectory specified in the file.

The search path is determined at Python compile or installation time, and installing new libraries may also modify it. The search path is stored in the path variable in the sys module. Let's do a simple experiment: in the interactive interpreter, enter the following code:

>>> import sys
>>> sys.path
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>> 

The output of sys.path is a list, where the first item is an empty string '', representing the current directory (if printed from a script, you can more clearly see which directory it is), that is, the directory where we execute the Python interpreter (for scripts, it is the directory where the running script is located).

Therefore, if, as in my case, there is a file with the same name as the module to be imported in the current directory, it will shield the module to be imported.

After understanding the concept of the search path, you can modifysys.pathto import some modules that are not in the search path.

Now, create a file named fibo.py in the current directory of the interpreter or in a directory within sys.path. The code is as follows:

Then enter the Python interpreter and use the following command to import this module:

>>> import fibo

This does not write the function names directly defined in fibo into the current symbol table; it only writes the name of the module fibo there.

You can use the module name to access functions:

If you plan to use a function frequently, you can assign it to a local name:

>>> fib = fibo.fib
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

---

from ... import statement

Python's from statement allows you to import a specified part from a module into the current namespace. The syntax is as follows:

from modname import name1[, name2[, ... nameN]]

For example, to import the fib function from the module fibo, use the following statement:

>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This declaration does not import the entire fibo module into the current namespace; it only brings in the fib function from fibo.

Aliasing Modules

Use theaskeyword to alias a module or function:

import numpy as np  # 将 numpy 模块别名设置为 np
from math import sqrt as square_root  # 将 sqrt 函数别名设置为 square_root

---

from ... import * statement

Importing all contents of a module into the current namespace is also possible, just use the following declaration:

from modname import *

This provides a simple way to import all items from a module.

Not recommended, as it can easily cause naming conflicts.

---

Deep Dive into Modules

In addition to method definitions, a module can also include executable code. This code is generally used to initialize the module. It is executed only when the module is first imported.

Each module has its own independent symbol table, which is used as the global symbol table for all functions within the module.

Therefore, module authors can safely use these global variables within the module without worrying about confusing them with other users' global variables.

On the other hand, when you really know what you are doing, you can also access functions within the module using notation like modname.itemname.

Modules can import other modules. Use import at the top of a module (or script, or elsewhere) to import a module. Of course, this is just a convention, not mandatory. The name of the imported module will be placed into the symbol table of the module currently being operated on.

There is another way to import: you can use import to directly import names (functions, variables) from a module into the current operating module. For example:

>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This import method does not place the name of the imported module in the current symbol table (so in this example, the name fibo is not defined).

There is yet another method that can import all names (functions, variables) in a module into the current module's symbol table at once:

>>> from fibo import *
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This will import all names, but names starting with a single underscore (_) are not included. In most cases, Python programmers do not use this method, because names imported from other sources may override existing definitions.

---

__name__ Attribute

When a module is first introduced by another program, its main program will run.

If we want a certain block of code in the module not to execute when the module is imported, we can use the__name__attribute to make that block execute only when the module itself is run.

#!/usr/bin/python3
# Filename: using_name.py

if __name__ == '__main__':
   print('程序自身在运行')
else:
   print('我来自另一模块')

The runtime output is as follows:

$ python using_name.py
程序自身在运行

$ python
>>> import using_name
我来自另一模块
>>>

Explanation: Each module has a __name__ attribute.

• If the module is run directly,__name__its value is__main__。

• If the module is imported,__name__its value is the module name.

Note:__name__and__main__There are double underscores underneath,_ _is to remove the space in between.

---

dir() Function

The built-in function dir() can find all names defined within a module. It returns them as a list of strings:

>>> import fibo, sys
>>> dir(fibo)
['__name__', 'fib', 'fib2']
>>> dir(sys)  
['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__',
 '__package__', '__stderr__', '__stdin__', '__stdout__',
 '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe',
 '_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv',
 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder',
 'call_tracing', 'callstats', 'copyright', 'displayhook',
 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix',
 'executable', 'exit', 'flags', 'float_info', 'float_repr_style',
 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags',
 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit',
 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount',
 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info',
 'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path',
 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1',
 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit',
 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout',
 'thread_info', 'version', 'version_info', 'warnoptions']

If no argument is given, the dir() function lists all names currently defined:

>>> a = [1, 2, 3, 4, 5]
>>> import fibo
>>> fib = fibo.fib
>>> dir() # 得到一个当前模块中定义的属性列表
['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']
>>> a = 5 # 建立一个新的变量 'a'
>>> dir()
['__builtins__', '__doc__', '__name__', 'a', 'sys']
>>>
>>> del a # 删除变量名a
>>>
>>> dir()
['__builtins__', '__doc__', '__name__', 'sys']
>>>

---

Standard Modules

Python itself comes with some standard module libraries, which will be introduced in the Python Library Reference (that is, the "Library Reference" later).

Module name	Description
math	Mathematical operations (such as square root, trigonometric functions, etc.)
os	Operating system related functions (such as file and directory operations)
sys	System-related parameters and functions
random	Generate random numbers
datetime	Handle dates and times
json	Process JSON data
re	Regular expression operations
collections	Provide additional data structures (such as defaultdict, deque)
itertools	Provide iterator tools
functools	Higher-order function tools (such as reduce, lru_cache)

Some modules are built directly into the interpreter. Although these are not built-in features of the language, they can be used very efficiently, even for system-level calls.

These components are configured differently depending on the operating system. For example, the winreg module is only provided on Windows systems.

Note that there is a special module, sys, which is built into every Python interpreter. The variables sys.ps1 and sys.ps2 define the strings corresponding to the primary and secondary prompts:

>>> import sys
>>> sys.ps1
'>>> '
>>> sys.ps2
'... '
>>> sys.ps1 = 'C> '
C> print('Example!')
Example!
C> 

---

Package

Packages are a way of managing Python module namespaces, using "dotted module names".

For example, if a module's name is A.B, then it represents a submodule B in a package A.

Just as when using modules, you don't have to worry about global variables affecting each other between different modules, using dotted module names also means you don't have to worry about modules with the same name between different libraries.

In this way, different authors can provide NumPy modules, or Python graphics libraries.

Suppose you want to design a set of modules (or a "package") for uniformly handling sound files and data.

There are many different audio file formats (basically distinguished by their suffixes, for example: .wav, :file:.aiff, :file:.au,), so you need an ever-growing set of modules to convert between different formats.

And for this audio data, there are also many different operations (such as mixing, adding echo, adding equalizer functions, creating artificial stereo effects), so you also need an endless set of modules to handle these operations.

Here is a possible package structure (in a hierarchical file system):

sound/                          顶层包
      __init__.py               初始化 sound 包
      formats/                  文件格式转换子包
              __init__.py
              wavread.py
              wavwrite.py
              aiffread.py
              aiffwrite.py
              auread.py
              auwrite.py
              ...
      effects/                  声音效果子包
              __init__.py
              echo.py
              surround.py
              reverse.py
              ...
      filters/                  filters 子包
              __init__.py
              equalizer.py
              vocoder.py
              karaoke.py
              ...

When importing a package, Python searches the directories in sys.path to find the subdirectories contained in the package.

A directory is recognized as a package only if it contains a file called __init__.py. This is mainly to prevent some overly common names (such as string) from accidentally affecting valid modules in the search path.

In the simplest case, just place an empty :file:__init__.py file. Of course, this file can also contain some initialization code or assign a value to the __all__ variable (introduced later).

Users can import only a specific module from a package at a time, for example:

import sound.effects.echo

This will import the submodule: sound.effects.echo. It must be accessed using its full name:

sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)

There is another way to import a submodule:

from sound.effects import echo

This will also import the submodule: echo, and it does not require those lengthy prefixes, so it can be used like this:

echo.echofilter(input, output, delay=0.7, atten=4)

Another variation is to directly import a function or variable:

from sound.effects.echo import echofilter

Similarly, this method will import the submodule: echo, and you can directly use its echofilter() function:

echofilter(input, output, delay=0.7, atten=4)

Note that when usingfrom package import itemin this form, the corresponding item can be either a submodule (subpackage) in the package, or other names defined in the package, such as functions, classes, or variables.

The import statement first treats item as a name defined in the package. If not found, it then tries to import it as a module. If still not found, it raises a:exc:ImportErrorexception.

Conversely, if you use a form likeimport item.subitem.subsubitemin this import form, everything except the last item must be a package, and the last item can be a module or a package, but it cannot be the name of a class, function, or variable.

---

Importing * From a Package

If we usefrom sound.effects import *What will happen?

Python will enter the file system, find all submodules in the package, and import them all one by one.

But this method does not work very well on the Windows platform, because Windows is a case-insensitive system.

On Windows, we cannot determine whether a file called ECHO.py is imported as a module named echo, Echo, or ECHO.

To solve this problem, we only need to provide an explicit package index.

The import statement follows this rule: if the package definition file __init__.pycontains a list variable called__all__then when usingfrom package import *all names in this list are imported as the contents of the package.

As the package author, don't forget to make sure that after updating the package,__all__is also updated.

The following example contains the following code in file:sounds/effects/__init__.py:

__all__ = ["echo", "surround", "reverse"]

This means that when you use from sound.effects import *, you will only import these three submodules in the package.

If__all__is truly not defined, then usingfrom sound.effects import *with this syntax will not import any submodule from the package sound.effects. It only imports the package sound.effects and all the contents defined in it (possibly running the initialization code defined in __init__.py).

This will import all names defined in __init__.py. And it will not break any explicitly specified modules that we imported before this statement. Take a look at this part of the code:

import sound.effects.echo
import sound.effects.surround
from sound.effects import *

In this example, before executing from...import, the echo and surround modules in the package sound.effects have already been imported into the current namespace. (Of course, if __all__ is defined, there is no problem at all.)

Usually we do not recommend using*this method to import modules, because it often reduces the readability of the code. However, it can indeed save a lot of typing, and some modules are designed to be imported only through specific methods.

Remember, usingfrom Package import specific_submodulethis method is never wrong. In fact, it is also the recommended method. Unless the submodule you want to import might have the same name as a submodule in another package.

If the package is a subpackage in the structure (for example, in this case for the package sound), and you want to import a sibling package (a package at the same level), you must use an absolute path to import it. For example, if the module sound.filters.vocoder wants to use the module echo from the package sound.effects, you have to write from sound.effects import echo.

from . import echo
from .. import formats
from ..filters import equalizer

Both implicit and explicit relative imports start from the current module. The name of the main module is always "__main__". The main module of a Python application should always use absolute path references.

Packages also provide an additional attribute __path__. This is a list of directories, where each containing directory has an __init__.py that serves this package. You have to define it before other __init__.py files are executed. This variable can be modified to affect the modules and subpackages contained in the package.

This feature is not commonly used; it is generally used to extend the modules in a package.

Other extensions