Use Cases
Data Extraction
Smart scraping with AVM
Objective: Enable smart scraping and data extraction through LLM-generated scripts executed within AVM’s sandbox.
Data Extraction
Delegate web scraping logic to an LLM, execute safely on AVM nodes, and obtain structured CSV/JSON without local risk.
Use Cases
Web2: Shopify Product APIs
Extract product data, pricing, and inventory information from e-commerce platforms.
Web3: CoinGecko, Twitter, Dune
Scrape cryptocurrency data, social sentiment, and blockchain analytics for comprehensive market analysis.
Scenario: Bulk Scraping
Extract account balances pages from a DeFi dashboard in parallel.
Implementation: Two-Stage Extraction
- Fetch HTML
Retrieve page content locally. - Parse with LLM
Prompt the LLM to extract table data via BeautifulSoup. - Run in AVM
Execute parsing code with therunPython
tool. - Aggregate Results
Combine CSV outputs for all URLs.
Example (TypeScript)
Next Steps
- Parallelize jobs via MCP concurrency.
- Store results on IPFS.
- Add retry and throttling mechanisms.