Tag Archives: twitter

Pythonic way of opening more than one file at a time

Posted on by
17

Some hours ago, someone asked on StackExchange about a more pythonic way to represent the following code:

PLAIN TEXT
PYTHON:
  1. with open(read_csv, 'r') as read_file:
  2.     with open(write_csv, 'r') as write_file:
  3.         reader = csv.reader(read_file)
  4.         writer = csv.writer(write_file)
  5.         for row in reader:
  6.             #Do some stuff to manipulate the fields from read_file and th

Unfortunately, the user deleted his question, before i could post my suggestion. The only evidence of the question remaining is this tweet from PythonQuestions. I'm posting my extended answer here for future reference and to see, if someone more versed in python might correct me.

Major update

Please have a look at the comments section below and some of the corrections posted there. Some suggested the use of contextlib, which is a part of the standard library i didn't know about yet.. Also, it seems python >2.7 can handle more than one context at a time so that my "solution" isn't really needed.

My try

Starting off, i do not think that the syntax used is too unpythonic. If it is only used once or twice, i wouldn't change anything about it, but if someone likes to open more than one file within a single with-statement often, then writing a proxy class that adheres to the with-statements syntax might be useful. For opening two files it would look like this:

PLAIN TEXT
PYTHON:
  1. class openTwoFiles():
  2.     def __init__(self, file1, file2):
  3.         self.fn1 = file1
  4.         self.fn2 = file2
  5.    
  6.     def __enter__(self):
  7.         self.f1 = open(self.fn1,'r')
  8.         self.f2 = open(self.fn2,'r')
  9.         return (self.f1, self.f2)
  10.    
  11.     def __exit__(self, type, value, traceback):
  12.         self.f1.close()
  13.         self.f2.close()

The class openTwoFiles implements the two basic methods __enter__ and __exit__ which the with-statement needs to execute. Using this class might look like this:

PLAIN TEXT
PYTHON:
  1. with openTwoFiles('test.txt','test2.txt') as (f1,f2):
  2.     print f1
  3.     print f2

Of course for the specific code in question the implementation of the csv-class and an iterator over the reader might also help, but i'm sticking with the general case, in which the opening of the files suffices. The class openTwoFiles might be extended to open a arbitrary number of files with custom flags:

PLAIN TEXT
PYTHON:
  1. class openFiles():
  2.     def __init__(self, files, flags):
  3.         if isinstance(files,basestring):
  4.             files = [files]
  5.         if isinstance(flags,basestring):
  6.             flags = [flags]
  7.         assert len(flags)==len(files)
  8.         self.files = files
  9.         self.flags = flags
  10.    
  11.     def __enter__(self):
  12.         self.fhs = []
  13.         for f, fl in zip(self.files, self.flags):
  14.             self.fhs.append(open(f,fl))
  15.         return self.fhs
  16.    
  17.     def __exit__(self, type, value, traceback):
  18.         for f in self.fhs:
  19.             f.close()
  20.  
  21. with openFiles(['test.txt','test2.txt'], ['r','r']) as ll:
  22.     print ll