11. Data Persistence

The modules described in this chapter support storing Python data in a persistent form on disk. The pickle and marshal modules can turn many Python data types into a stream of bytes and then recreate the objects from the bytes. The various DBM-related modules support a family of hash-based file formats that store a mapping of strings to other strings. The bsddb module also provides such disk-based string-to-string mappings based on hashing, and also supports B-Tree and record-based formats.

The list of modules described in this chapter is:

• 11.1. pickle — Python object serialization
  • 11.1.1. Relationship to other Python modules
  • 11.1.2. Data stream format
  • 11.1.3. Usage
  • 11.1.4. What can be pickled and unpickled?
  • 11.1.5. The pickle protocol
    • 11.1.5.1. Pickling and unpickling normal class instances
    • 11.1.5.2. Pickling and unpickling extension types
    • 11.1.5.3. Pickling and unpickling external objects
  • 11.1.6. Subclassing Unpicklers
  • 11.1.7. Example
• 11.2. cPickle — A faster pickle
• 11.3. copy_reg — Register pickle support functions
  • 11.3.1. Example
• 11.4. shelve — Python object persistence
  • 11.4.1. Restrictions
  • 11.4.2. Example
• 11.5. marshal — Internal Python object serialization
• 11.6. anydbm — Generic access to DBM-style databases
• 11.7. whichdb — Guess which DBM module created a database
• 11.8. dbm — Simple “database” interface
• 11.9. gdbm — GNU’s reinterpretation of dbm
• 11.10. dbhash — DBM-style interface to the BSD database library
  • 11.10.1. Database Objects
• 11.11. bsddb — Interface to Berkeley DB library
  • 11.11.1. Hash, BTree and Record Objects
• 11.12. dumbdbm — Portable DBM implementation
  • 11.12.1. Dumbdbm Objects
• 11.13. sqlite3 — DB-API 2.0 interface for SQLite databases
  • 11.13.1. Module functions and constants
  • 11.13.2. Connection Objects
  • 11.13.3. Cursor Objects
  • 11.13.4. Row Objects
  • 11.13.5. SQLite and Python types
    • 11.13.5.1. Introduction
    • 11.13.5.2. Using adapters to store additional Python types in SQLite databases
      • 11.13.5.2.1. Letting your object adapt itself
      • 11.13.5.2.2. Registering an adapter callable
    • 11.13.5.3. Converting SQLite values to custom Python types
    • 11.13.5.4. Default adapters and converters
  • 11.13.6. Controlling Transactions
  • 11.13.7. Using sqlite3 efficiently
    • 11.13.7.1. Using shortcut methods
    • 11.13.7.2. Accessing columns by name instead of by index
    • 11.13.7.3. Using the connection as a context manager
  • 11.13.8. Common issues
    • 11.13.8.1. Multithreading