Refactor async #17
Merged
Refactor async #17
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove GenericScraper class (use client.scrape_url() directly) - Fix async naming: method() is async, method_sync() is sync - products_async() → products() - wait_async() → wait() - fetch_async() → fetch() - Add _ensure_initialized() checks for context manager usage - Update CLI scrape command from 'generic' to 'url' - Update all tests and examples to use new method names - Simplify README documentation - Add comprehensive sync_client.py with full scraper coverage BREAKING CHANGE: GenericScraper removed, use client.scrape_url() instead 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Document GenericScraper removal - Document async naming convention change (_async → no suffix) - Document new SyncBrightDataClient with full coverage - Update v2.0.0 examples to use new method names
- Reduce README from 419 to ~160 lines - Remove platform-specific examples (LinkedIn, Instagram, Facebook) - Rename 'Amazon Scraping' to 'Web Scraper API' with general explanation - Add docs/sync_client.md with full sync client details
- Add non-blocking async mode using /unblocker/req and /unblocker/get_result endpoints - SERP async: ~3 seconds, Web Unlocker async: ~2 minutes (sync recommended for single URLs) - Fix SyncBrightDataClient: remove unused customer_id parameter - Fix Web Unlocker poll_timeout default: 30s → 180s - Simplify API: remove _async suffix, method() is now async by default - Add AsyncUnblockerClient for trigger/poll/fetch workflow - Add comprehensive async mode guide and update documentation - Delete unused demo files, update all unit tests
Add async mode for SERP and Web Unlocker APIs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding sync search & extract