How does open handle context management?

106
March 06, 2018, at 08:10 AM

The python built-ins open, and file work with context managers in a way that I don't quite understand.

It is to my understanding that open will create a file. file implements the context-manager methods __enter__ and __exit__. I would initially expect __enter__ to implement the actual opening of the file descriptor.

However, using open outside of a with block will return a file which is already open. So, it appears either file.__init__ or open is actually opening the file descriptor, and as far as I can tell file.__enter__ isn't doing anything. Or maybe file.__init__/open calls file.__enter__ directly?

First question:

What is the execution-flow of the open built-in? What does open handle, what does file.__init__ handle, and what does file.__enter__ handle? How does this work when re-using one file object for multiple cycles of opening/closing the file? How is this different from re-using other contextmanager objects for multiple context-cycles?

Second question:

Objects such as file objects have a setup steps and teardown steps. The setup occurs in __init__ , and the tear-down occurs in either close or __exit__.

Is this a good design pattern? Should this design pattern be implemented for custom functions/context managers?

Answer 1

If you look in _pyio.py (a pure-Python implementation of the io module) you find the following code in class IOBase:

### Context manager ###
def __enter__(self):  # That's a forward reference
    """Context management protocol.  Returns self (an instance of IOBase)."""
    self._checkClosed()
    return self
def __exit__(self, *args):
    """Context management protocol.  Calls close()"""
    self.close()

This contains the answers to most of your questions. The important thing to understand is that the context manager's function is to insure that you close the file when you are done with it. It does this simply by calling the close function, which saves you the trouble of doing so.

What does file.__enter__ handle? Nothing. It simply returns to you the file object that was the result of the call to the built-in function open().

How does this work when using one file object for multiple cycles of opening and closing the file? The context manager is not very useful for that purpose, since you must explicitly open the file each time.

Is this a good design pattern? Yes, because it reduces the amount of code you have to write, it's easy to read and understand.

Should this pattern be implemented for custom functions/context managers? Any time you have an object that needs to be cleaned up, or has usage that involves some type of open/close concept, you should consider this pattern. The standard library has many other examples.

Answer 2

For Question 1

In CPython, open() does nothing but creating a file object, which the underlying C type is PyFileObject; See source code in bltinmodule.c and fileobject.c

static PyObject *
builtin_open(PyObject *self, PyObject *args, PyObject *kwds)
{
    return PyObject_Call((PyObject*)&PyFile_Type, args, kwds);
}
  • file.__init__ would open the file

  • file.__enter__ indeed do nothing except doing empty check on field file.fp

  • file.__exit__ invoke close() method to close file

For Question 2

Why file design like this is due to a historical reason.

open and with are two different keywords introduced on different versions of CPython. with was introduced till Python 2.5 (see PEP 343). At that time, open has been used for a long time.

For our customized type, we could design like file or not, depends on the concrete application context.

For example, threading.Lock is a different design, its init and enter are separately.

READ ALSO
Tkinter - AttributeError: 'str' object has no attribute 'set' - Dice Roll Simulator

Tkinter - AttributeError: 'str' object has no attribute 'set' - Dice Roll Simulator

The code i'm trying run is belowI'm trying to get the user input through a widget box and then run the while loop to simulate the Dice roll

103
Adding together values in 2 different arrays to calculate an average python

Adding together values in 2 different arrays to calculate an average python

I have two arrays that need to be combined into a single array by adding the together the values from each array

133
Reading a csv file and performing formulations with certain rows

Reading a csv file and performing formulations with certain rows

I asked a question similar last night, but my professor gave some clarification on how she wants the question answered, and it's thrown me for a loop

78
Illegal instruction when import tensorflow in Python

Illegal instruction when import tensorflow in Python

I am trying to run python program in shell, CentOS 70 x64, on VPS

162