Notibuy needed reliable product data out of a large retailer’s search pages. Retail sites are among the more aggressively defended targets on the web, and the usual approach — drive a headless browser and hope — fails quickly and expensively at any real volume.

Approach

The scraper is fully asynchronous and paginated, and it doesn’t try to read rendered HTML. Search pages ship their data as a JSON blob embedded in the document for the frontend framework to hydrate from, so the parser pulls that payload directly and reads structured product records out of it. That is both faster and far more stable than selector-matching against markup that changes whenever someone ships a redesign.

Fetching is isolated behind a single request layer. The parsing logic doesn’t know or care how a page was retrieved, so the anti-bot strategy can be replaced without touching anything downstream — which matters, because that layer is the part that goes stale.

Alongside this I built the full stack for their Veribuy platform.

Note

This was client work under NDA. The description here is limited to the general technical approach; specifics of the platform, data handling, and business logic are omitted.