# I Tried to Build a Real-time Sync Engine. I Was Wrong About the Tradeoffs.

##Goal

Build a real-time synchronization system for Cropnest that would allow farmers and dealers to work offline and sync when connectivity returned. The system needed to handle concurrent edits, resolve conflicts automatically, and maintain strong consistency.

##What I Assumed

• CRDTs (Conflict-free Replicated Data Types) would solve the conflict resolution problem elegantly
• Eventual consistency would be acceptable for agricultural trade data
• The complexity would be manageable with a small team

##What I Tried

###Phase 1: CRDT Exploration

Started with Yjs and Automerge libraries. Built a prototype that could sync simple documents between two clients. Worked well for text documents.

###Phase 2: Custom Implementation

Realized we needed domain-specific merge logic for trade data. Built a custom CRDT-like structure for:

• Quantity updates
• Price negotiations
• Deal state transitions

###Phase 3: Integration

Integrated with our PostgreSQL backend. Hit the first wall: how do you query CRDT state efficiently? Every read required reconstructing state from operation history.

##What Failed

###Conflict Resolution Complexity

Agricultural deals aren't commutative operations. If two dealers bid on the same harvest simultaneously, "last write wins" is wrong, but so is "merge both bids." The business logic for resolution was more complex than the sync engine itself.

###Consistency vs. Availability

Farmers often have no connectivity for days. The system needed to be available offline, but trade data requires strong consistency. This tension proved fundamental, not implementation-specific.

###Operational Complexity

The sync engine became a distributed system problem. We needed:

• Vector clocks for causality tracking
• Tombstones for deleted data
• Compaction strategies for growing operation logs
• Conflict resolution UI for unresolvable cases

##Why I Stopped

After 6 weeks, the sync engine was 40% of our codebase but 0% of our business value. The complexity was pulling focus from core marketplace features.

More importantly, I realized we were solving the wrong problem. Farmers don't need real-time sync—they need clear, asynchronous workflows with explicit handoffs.

##What I'd Do Differently

1. Start with async workflows, not sync engines

   • Explicit "submit for review" states
   • Clear ownership transitions
   • No concurrent editing of the same record

2. Use existing tools for simple cases

   • Firebase or Supabase for user presence
   • Optimistic UI with rollback on conflict
   • Manual refresh for critical data

3. Question the requirement

   • Why does this need to be real-time?
   • What problem does offline editing solve?
   • Can we design around the constraint instead of through it?

##Open Questions

• Is there a middle ground between "fully offline" and "always online" that doesn't require a custom sync engine?
• How do other agricultural platforms handle this? (Research needed)
• Would a simpler "draft mode" with explicit publish solve 80% of the use case?