Group Consecutive Items with groupby
`itertools.groupby` collapses runs of equal-keyed items into `(key, group_iterator)` pairs. The catch is that it only groups *consecutive* equal items, so the input must already be sorted by the key if you want full grouping. This snippet covers run-length encoding, the sort-first idiom for dict-like grouping, and the iterator gotcha that bites every newcomer.
769 views
5
from itertools import groupby
letters = 'aaabbcaaa'
encoded = [(key, len(list(group))) for key, group in groupby(letters)]
print(encoded)
# [('a', 3), ('b', 2), ('c', 1), ('a', 3)]
# Reconstruction:
decoded = ''.join(ch * n for ch, n in encoded)
print(decoded) # aaabbcaaagroupby(iterable) yields a (key, group_iter) pair every time the key changes between consecutive items. Without a key= argument it groups by element identity, which makes one-line run-length encoding trivial. The 'a' run at the end shows up as a separate group because the second 'a' run is not adjacent to the first. RLE is the canonical example, but the same shape pops up in 'longest streak of green builds', 'plateaus in a stock chart', and any other 'consecutive equal' question.
from itertools import groupby
from operator import itemgetter
orders = [
{'user': 'ana', 'amount': 30},
{'user': 'ben', 'amount': 10},
{'user': 'ana', 'amount': 70},
{'user': 'cleo', 'amount': 5},
{'user': 'ben', 'amount': 20},
]
# Sort by the same key you will group on, otherwise groups get fragmented.
orders.sort(key=itemgetter('user'))
totals = {}
for user, group in groupby(orders, key=itemgetter('user')):
totals[user] = sum(item['amount'] for item in group)
print(totals)
# {'ana': 100, 'ben': 30, 'cleo': 5}When you want a true 'group by user' (not just consecutive runs), sort by the same key first and groupby will give you exactly one group per distinct key. operator.itemgetter('user') is a tiny callable that pulls a dict field, faster than lambda x: x['user'] and clearer at the call site. Without the sort, the example would yield two 'ana' groups and two 'ben' groups. For most workloads collections.defaultdict(list) is simpler than 'sort then groupby', but groupby shines when you stream data or already have it sorted.
from itertools import groupby
data = 'aaabb'
for key, group in groupby(data):
# Wrong: peeking at len() exhausts the iterator before you can use it.
# print(key, len(list(group)), list(group)) # second list() is empty!
# Right: materialize once into a list, then read.
items = list(group)
print(key, len(items), items)
# a 3 ['a', 'a', 'a']
# b 2 ['b', 'b']
# Also: advancing to the next outer group invalidates the previous group iter.
grouped = list(groupby(data)) # only the keys are now safe
print([(k, list(g)) for k, g in grouped]) # all groups are exhausted now
# [('a', []), ('b', [])]Each group is a thin iterator into the same underlying stream as the outer groupby, not an independent list. Consuming it twice gives an empty result the second time. Worse, advancing the outer loop (or materializing it with list(groupby(...))) silently invalidates every previous group object. The safe pattern is: convert each group to a list immediately inside the loop, or aggregate it (sum, count, max) before moving on. This is one of the top three Python interview gotchas.