# frontmatter
date:2024-01-22
author:Vedant Ghodekar
type:architecture
status:COMPLETED
tags:[go, rust]
summary:Optimizing ingestion pipelines by avoiding GC pressure with custom unmarshalers.
# Zero-Allocation JSON Parsing
Optimizing ingestion pipelines by avoiding GC pressure with custom unmarshalers.
##Context
Our event ingestion pipeline was processing 100k events/second, but Go's standard encoding/json was causing significant GC pressure. The heap was growing uncontrollably during peak hours.
##Approach
We switched to a zero-allocation JSON parser using json-iterator with custom unmarshalers. By reusing buffers and avoiding intermediate allocations, we reduced memory churn significantly.
##Benchmarks
- Standard library: 850 MB/s, 45% GC time
- Custom parser: 1.2 GB/s, 5% GC time
- Memory usage: Reduced by 70%
##Trade-offs
The code is more verbose and harder to maintain. We only applied this optimization to the hot path (ingestion workers), keeping the standard library for less critical paths.
The custom parser reduced GC pauses by 90% in high-throughput scenarios.