Big picture
Variables lose their values when a program ends. Files store data permanently on disk so it can be used later. Python works with three kinds of files in the syllabus: text files (human-readable characters), binary files (data in the same format as in memory, used here to store Python objects with pickle) and CSV files (tables of comma-separated values that spreadsheets can open).
Every board paper has a file-handling program to write: count words or lines in a text file, search or update records in a binary file, or read and write a CSV file. This lesson gives the methods and the standard programs, all tested.
Learning goals
Distinguish text, binary and CSV files, and absolute and relative paths.
Open files in the correct mode and close them, using
with.Read and write text files using
read(),readline(),readlines(),write()andwritelines().Use
seek()andtell().Write programs that count characters, words, lines and specific items in text files.
Store and retrieve records in binary files using
pickle, including search and update.Read and write CSV files using the
csvmodule.
Core ideas
1. Types of files and paths
Type | Contents | Example |
|---|---|---|
Text file | Characters, organised in lines ending with an EOL (end-of-line) character |
|
Binary file | Raw bytes; not human-readable; no EOL translation | Images, |
CSV file | Plain text where each line is a record and values are separated by commas |
|
Absolute path: the full path from the root, e.g.
C:\Users\Asha\data\marks.txt.Relative path: relative to the current working directory, e.g.
data\marks.txtor..\notes.txt.
2. Opening and closing files
file_object = open('filename', 'mode'). After use, file_object.close() saves pending data and frees the file.
Mode | Meaning | If the file doesn't exist | If it exists |
|---|---|---|---|
| Read only (default) |
| Reads from the start |
| Write only | Creates it | Erases existing contents |
| Append | Creates it | Adds data at the end |
| Read and write |
| Pointer at the start; no truncation |
| Write and read | Creates it | Erases contents |
| Append and read | Creates it | Writes at the end |
Add 'b' for binary files: 'rb', 'wb', 'ab', 'rb+' and so on.
The `with` statement closes the file automatically, even if an error occurs:
with open('poem.txt', 'w') as f:
f.write('The sun is bright\nThe sky is blue\nWe love to learn\n')3. Reading text files
Method | Returns |
|---|---|
| The entire file as one string |
| The next n characters |
| The next line, including |
| A list of all lines, each ending with |
Using the file written above (tested):
with open('poem.txt') as f:
print(repr(f.read(7))) # 'The sun'
print(repr(f.readline())) # ' is bright\n' (rest of the first line)
print(f.tell()) # 18
with open('poem.txt') as f:
lines = f.readlines()
print(len(lines)) # 3
print(repr(lines[1])) # 'The sky is blue\n'A file object can also be looped over line by line: for line in f:.
4. Writing text files
write(string)writes a string (you must add\nyourself) and returns the number of characters written.writelines(list_of_strings)writes all strings in a list; it doesn't add newlines.Numbers must be converted to strings first:
f.write(str(25)).
5. seek() and tell()
tell()returns the current position of the file pointer (in bytes from the start).seek(offset)moves the pointer to the given position.seek(0)goes back to the start.
with open('poem.txt') as f:
f.seek(4)
print(f.read(3)) # sun6. Standard text-file programs
Count words, vowels and lines starting with a word (tested on poem.txt):
with open('poem.txt') as f:
data = f.read()
print(len(data.split())) # 12 words
print(sum(1 for c in data if c.lower() in 'aeiou')) # 14 vowels
count = 0
with open('poem.txt') as f:
for line in f:
if line.startswith('The'):
count += 1
print(count) # 2Other common tasks: count a particular word (data.split().count('the')), display lines longer than a given length, copy lines that contain a character into another file, and count uppercase letters (c.isupper()).
Appending: after open('poem.txt', 'a') and writing 'Done\n', the file has 4 lines; opening it with 'w' instead would have erased the original three.
7. Binary files with pickle
The pickle module converts Python objects (lists, dictionaries) to a byte stream (pickling or serialisation) and back (unpickling or deserialisation).
pickle.dump(obj, file)writes an object to a binary file opened in'wb'or'ab'.pickle.load(file)reads the next object; it raises `EOFError` at the end of the file, which is used to stop reading.
Write and search records (tested):
import pickle
records = [{'roll': 1, 'name': 'Asha', 'marks': 88},
{'roll': 2, 'name': 'Ravi', 'marks': 72}]
with open('stu.dat', 'wb') as f:
for r in records:
pickle.dump(r, f)
with open('stu.dat', 'rb') as f:
try:
while True:
r = pickle.load(f)
if r['marks'] > 80:
print(r['name']) # Asha
except EOFError:
passUpdating a record: read all records into a list, change the required one, and write the whole list back in 'wb' mode. (Alternatively, open in 'rb+', remember the position with tell() before each load, then seek() back and dump() the modified record.)
8. CSV files with the csv module
Open a CSV file for writing with
newline='', so that blank lines don't appear between rows on some systems.csv.writer(f)creates a writer;writerow(list)writes one row;writerows(list_of_lists)writes many.csv.reader(f)returns rows as lists of strings; convert numbers withint()orfloat().next(reader)skips the header row.
Write and read (tested):
import csv
with open('emp.csv', 'w', newline='') as f:
w = csv.writer(f)
w.writerow(['id', 'name', 'salary'])
w.writerows([[1, 'Neha', 50000], [2, 'Arun', 42000]])
with open('emp.csv') as f:
r = csv.reader(f)
next(r) # skip header
for row in r:
if int(row[2]) > 45000:
print(row[1]) # NehaThe file contains plain text: id,name,salary, then 1,Neha,50000, then 2,Arun,42000.
9. Text, binary and CSV files compared
Basis | Text file | Binary file | CSV file |
|---|---|---|---|
Readable in a text editor | Yes | No | Yes |
Stores | Characters | Bytes (pickled objects) | Comma-separated values |
Module needed | None |
|
|
Read method |
|
|
|
Write method |
|
|
|
End of file detected by | Empty string from |
| End of the reader loop |
A reliable routine for file programs
Choose the file type and mode:
'r'to read,'w'to create or overwrite,'a'to add; add'b'for pickle.Use
with open(...) as f:so the file closes automatically.For text files, decide whether to read the whole file, line by line, or word by word with
split().For binary files, loop on
pickle.load()insidetryand stop atEOFError.For CSV files, use
newline=''when writing, skip the header withnext(), and convert strings to numbers.
Connections that matter
Exception handling handles
FileNotFoundErrorandEOFError.Advanced Python revision: string methods like
split(),startswith()andisupper()are used in text processing.Python–SQL connectivity is another way of storing data permanently.
Common mistakes
Opening an existing file in `'w'` mode by mistake, which erases it.
Forgetting `\n` with `write()`, so all lines run together.
Using `pickle` with a file opened in text mode. Use
'rb'or'wb'.Not handling `EOFError` when reading pickled records in a loop.
Comparing CSV values as numbers without converting them.
csv.readerreturns strings.
Revision checklist
I can distinguish text, binary and CSV files and absolute and relative paths.
I can choose the right file mode and use
with.I can use every read and write method and
seek()/tell().I can write programs to count words, lines, vowels and specific items.
I can write, search and update records with
pickle.I can read and write CSV files with the
csvmodule.