Read and Write JSON Files Idiomatically
Reading and writing JSON in Python is two lines once you know the right defaults. The right defaults are `with open(... encoding='utf-8')`, `json.load` / `json.dump` (not `loads` / `dumps`), and `indent=2, ensure_ascii=False` for human-readable files. This entry covers the round trip, atomic write, and the common pitfalls (bytes vs text mode, default ASCII escaping).
773 views
10
import json
import tempfile
import os
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, 'config.json')
config = {
'name': 'CodeSnatch',
'version': 3,
'features': ['lessons', 'practice'],
'owner': {'team': 'platform', 'on_call': 'ana'},
}
# Always pair `with` + explicit encoding. UTF-8 is the JSON default.
with open(path, 'w', encoding='utf-8') as f:
json.dump(config, f, indent=2, ensure_ascii=False)
with open(path, 'r', encoding='utf-8') as f:
loaded = json.load(f)
print(loaded['features']) # ['lessons', 'practice']
print(loaded == config) # TrueThe standard recipe: open the file in text mode, hand the file object to json.dump (not dumps + manual write), and rely on with for cleanup. Always pass encoding='utf-8' explicitly; on Windows the platform default is not UTF-8 and you will hit mysterious decode errors otherwise. json.dump writes incrementally, so even multi-MB payloads do not need to build a giant string first. The tempfile.TemporaryDirectory() wrapper here is just for the runnable demo; in real code the path is whatever your config layer hands you.
import json
import tempfile
import os
user = {
'name': 'José Müller',
'note': 'café, naïve, façade',
'emoji': '\U0001F600',
}
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, 'user.json')
# Wrong defaults: ensure_ascii=True turns every non-ASCII char into \uXXXX.
bad = json.dumps(user, indent=2)
print('default (ASCII-escaped):')
print(bad)
# Right defaults for human-readable files: indent + UTF-8 raw text.
good = json.dumps(user, indent=2, ensure_ascii=False, sort_keys=True)
print()
print('readable:')
print(good)
with open(path, 'w', encoding='utf-8') as f:
f.write(good)
print('size on disk:', os.path.getsize(path), 'bytes')Two flags decide how friendly the output looks. indent=2 formats with newlines and two-space indent (use 4 if you prefer 4-space). ensure_ascii=False keeps real UTF-8 characters in the output instead of escaping every non-ASCII byte to \uXXXX, which halves the file size and makes diffs readable. sort_keys=True is the cherry on top for config files: deterministic order means stable diffs in code review. None of these matter for machine-to-machine JSON, but they matter a lot for files humans edit.
import json
import os
import tempfile
def atomic_write_json(path, data):
"""Write JSON to `path` atomically: a crash mid-write cannot corrupt it."""
directory = os.path.dirname(os.path.abspath(path))
os.makedirs(directory, exist_ok=True)
# NamedTemporaryFile in the same directory so os.replace stays on one filesystem.
fd, tmp_path = tempfile.mkstemp(prefix='.tmp-', dir=directory, suffix='.json')
try:
with os.fdopen(fd, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False, sort_keys=True)
f.flush()
os.fsync(f.fileno())
os.replace(tmp_path, path) # atomic on POSIX and Windows
except Exception:
# Cleanup the temp file if anything went wrong.
try:
os.unlink(tmp_path)
except FileNotFoundError:
pass
raise
with tempfile.TemporaryDirectory() as tmp:
target = os.path.join(tmp, 'config.json')
atomic_write_json(target, {'version': 1, 'name': 'demo'})
with open(target, encoding='utf-8') as f:
print(f.read())
print('done, file size:', os.path.getsize(target))Naive json.dump to the destination path is unsafe: if the process crashes mid-write, readers see a truncated, unparseable file. The fix is the classic 'write to temp, fsync, rename' sequence. os.replace is atomic on every modern OS as long as the temp file is on the same filesystem (so always create the temp file in the same directory). f.flush() followed by os.fsync(f.fileno()) is what guarantees the bytes hit the disk before the rename, not just the OS page cache. Ship this helper for any config writer that has to survive a power loss.
