Developer Tools
Scrapyd
Service daemon that deploys and schedules Scrapy spiders through an HTTP JSON API, with a simple web interface for monitoring crawl jobs.
Available on distributions:UbuntuDebian
Published onAzure Marketplace, AWS Marketplace
Scrapy on Azure, AWS & Google Cloud
Python framework for writing web crawlers and scraping pipelines, handling request scheduling, CSS and XPath selectors, and export to JSON or CSV. Built and maintained by WIEWAVE for Azure Marketplace, AWS Marketplace and Google Cloud Marketplace, on Ubuntu, Debian, Red Hat Enterprise Linux, AlmaLinux and Rocky Linux.
Scrapy at a glance
| Built & maintained by | WIEWAVE |
|---|---|
| Category | Developer Tools |
| Operating systems | Ubuntu, Debian, Red Hat Enterprise Linux, AlmaLinux, Rocky Linux |
| Marketplaces | Azure Marketplace, AWS Marketplace and Google Cloud Marketplace |
| Offer types | Public listing, with private offers on request |
Availability by marketplace
| Marketplace | Status |
|---|---|
| Azure Marketplace | Published |
| AWS Marketplace | Published |
| Google Cloud Marketplace | Published |
Need it on another marketplace, or as a private offer for your organisation? cloud@wiewave.com
Operating systems
Scrapy is built on 5 distributions
Each image follows its distribution's own provisioning model, package manager and security tooling — not one build relabelled several times.
Ubuntu
LTS and interim releases, Minimal and Pro variants, built to Canonical's cloud-image conventions.
Debian
Stable and oldstable, with backports where a workload needs a newer runtime than the release ships.
Red Hat Enterprise Linux
RHEL 8 and 9 images, including BYOS and pay-as-you-go licensing models on each marketplace.
AlmaLinux
The 1:1 RHEL-compatible rebuild, and the default landing spot for teams migrating off CentOS Linux.
Rocky Linux
Enterprise-grade RHEL compatibility with a community governance model, on 8 and 9 streams.
How it's built
How WIEWAVE builds and maintains this image
The same four steps behind every offer we've published, including the hardening and CIS Benchmark checks every build goes through.
- 01
Scoped
The distribution, licensing model and target marketplaces are agreed before anything is built.
- 02
Built & hardened as code
Packer templates, Ansible provisioning and a pinned package set — then hardened, scanned and checked against the CIS Benchmark for its distribution.
- 03
Certified & published
Taken through each cloud's own certification pipeline before it goes live on the marketplace.
- 04
Maintained
Rebuilt on the upstream security cadence and re-published, with old versions retired without breaking deployments.
FAQ
Scrapy: common questions
If yours isn't here, ask our marketplace team directly.
Scrapy is available on all three major cloud marketplaces: Azure Marketplace, AWS Marketplace and Google Cloud Marketplace.
The Scrapy image is built on Ubuntu, Debian, Red Hat Enterprise Linux, AlmaLinux and Rocky Linux. Each build follows that distribution's own provisioning model, package manager and security tooling, so it behaves correctly once it's deployed.
WIEWAVE rebuilds its marketplace images on the upstream security cadence, re-scans and re-certifies them, and retires old versions in an order that doesn't break customers mid-deployment.
Pricing is shown on each marketplace listing and billed through your existing cloud account. If you need negotiated terms or a listing scoped to your organisation only, we can publish Scrapy to you as a private offer.
Yes. We can build a customised Scrapy image — on a different distribution, with a hardened baseline or your own configuration preinstalled — and publish it to the marketplace you need. Email cloud@wiewave.com with what you need.
Developer Tools
More in Developer Tools
Language runtimes and full framework stacks, so a developer VM is productive the minute it boots.
Developer Tools
Scripting Shell
General-purpose shell scripting environment with common interpreters and command-line utilities preinstalled for writing and running automation scripts.
Available on distributions:UbuntuDebian
Published onAzure Marketplace
Developer Tools
Shapely
Python library for manipulating and analysing planar geometric objects, wrapping the GEOS engine for buffers, intersections, unions and spatial predicates.
Available on distributions:UbuntuDebianRHELAlmaLinuxRocky
Published onAzure Marketplace, AWS Marketplace
Developer Tools
SingleDispatch
Python environment providing single-dispatch generic function support, with the backport package, standard library tooling and usage examples preinstalled.
Available on distributions:UbuntuDebian
Published onAzure Marketplace
Developer Tools
smtp4dev
Fake SMTP server that captures messages sent by applications during development and shows them in a web interface instead of delivering them.
Available on distributions:UbuntuDebian
Published onAzure Marketplace, AWS Marketplace
Developer Tools
Sphinx Doc
Documentation generator turning reStructuredText or Markdown sources into HTML, PDF and man pages, with autodoc extraction for Python APIs.
Available on distributions:UbuntuDebianRHELAlmaLinuxRocky
Published onAzure Marketplace, AWS Marketplace
Need Scrapy customised, or on another cloud?
Tell us the distribution, the marketplace and the commercial model — we'll build, certify and publish it as a public listing or a private offer.