How to beginner · 3 min read

How to do fixed size chunking in Python

Quick answer

Use Python slicing or iterators to split data into fixed size chunks. For example, use a list comprehension with slicing like [data[i:i+chunk_size] for i in range(0, len(data), chunk_size)] to chunk lists or strings efficiently.

PREREQUISITES

Python 3.8+

Setup

No special setup is required beyond having Python 3.8 or newer installed. This chunking technique uses built-in Python features without external libraries.

Step by step

Use slicing in a list comprehension to split a list or string into fixed size chunks. This method is simple, efficient, and works for any iterable supporting slicing.

python

def fixed_size_chunks(data, chunk_size):
    """Yield successive fixed-size chunks from data."""
    for i in range(0, len(data), chunk_size):
        yield data[i:i + chunk_size]

# Example usage
if __name__ == "__main__":
    data_list = list(range(1, 21))
    chunk_size = 5
    chunks = list(fixed_size_chunks(data_list, chunk_size))
    print("Chunks of list:", chunks)

    data_str = "abcdefghijklmnopqrstuvwxyz"
    chunks_str = list(fixed_size_chunks(data_str, 6))
    print("Chunks of string:", chunks_str)

output

Chunks of list: [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20]]
Chunks of string: ['abcdef', 'ghijkl', 'mnopqr', 'stuvwx', 'yz']

Common variations

You can implement fixed size chunking using itertools.islice for iterators or use list comprehensions for eager evaluation. Async chunking is possible but uncommon. Adjust chunk size dynamically or use libraries like more-itertools for advanced chunking.

python

from itertools import islice

def chunk_iterator(iterable, chunk_size):
    """Yield chunks from any iterable using islice."""
    it = iter(iterable)
    while True:
        chunk = list(islice(it, chunk_size))
        if not chunk:
            break
        yield chunk

# Example usage
if __name__ == "__main__":
    data = range(1, 16)
    for c in chunk_iterator(data, 4):
        print(c)

output

[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11, 12]
[13, 14, 15]

Troubleshooting

If chunks are smaller than expected, verify your chunk_size is a positive integer.
For non-sliceable iterables, use the itertools.islice approach.
Empty input returns no chunks; ensure your data is not empty.

✅

Key Takeaways

Use slicing with range stepping for simple fixed size chunking of lists and strings.
For general iterables, use itertools.islice to chunk without loading all data into memory.
Always validate chunk_size to avoid unexpected behavior or errors.

Verified 2026-04