Open3DLab accuses Roblox crawler of downloading 145GB of 3D assets
Open3DLab says a crawler with the user agent trellis2-data-sourcing/1.0 and a @roblox.com address scanned 18,926 projects in 34 hours and downloaded 145.5GB, including 1,027 files from 712 projects. The crawler ignored robots.txt, the site's terms and creator licenses barring non-commercial use; Roblox did not respond to a request for comment.
- Crawler made 26,429 requests and pulled 1,027 files from 712 projects
- 54.8GB came from SmutBase and 62.6GB from SFMLab
- Scraping was spotted on October 18 and blocked by the next morning
- User agent points to Roblox's 3D foundation model research
Read next
AI