← 返回首页
fix: Make adaptive-crawler extra importable by vdusek · Pull Request #2016 · apify/crawlee-python · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search
Appearance settings
Resetting focus

fix: Make adaptive-crawler extra importable#2016

Merged
vdusek merged 1 commit into
masterfrom
fix/adaptive-crawler-extra
Jul 3, 2026
Merged

fix: Make adaptive-crawler extra importable#2016
vdusek merged 1 commit into
masterfrom
fix/adaptive-crawler-extra

Conversation

vdusek commented Jul 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Problem

pip install 'crawlee[adaptive-crawler]' couldn't import AdaptivePlaywrightCrawler, because the module imports bs4 and parsel at the top level while the extra didn't include them.

Changes

The adaptive-crawler extra now pulls in crawlee[beautifulsoup,parsel], using the same self-referencing pattern as the all extra.

Verification

Installed only the adaptive-crawler extra into a clean venv, then imported AdaptivePlaywrightCrawler and constructed both the BeautifulSoup and Parsel static-parser factories.

vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Jul 3, 2026
vdusek self-assigned this Jul 3, 2026
vdusek mentioned this pull request Jul 3, 2026
Draft
vdusek requested a review from Pijukatel July 3, 2026 07:58

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.39%. Comparing base (7780e78) to head (33177ab).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@ Coverage Diff @@ ## master #2016 +/- ## ========================================== + Coverage 93.35% 93.39% +0.04% ========================================== Files 179 179 Lines 12482 12482 ========================================== + Hits 11652 11658 +6 + Misses 830 824 -6
Flag Coverage Δ
unit 93.39% <ø> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

vdusek requested a review from Copilot July 3, 2026 08:05
Copilot AI reviewed Jul 3, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Pull request overview

This PR fixes packaging for the adaptive-crawler optional extra so that pip install 'crawlee[adaptive-crawler]' can successfully import AdaptivePlaywrightCrawler by ensuring the extra also brings in the BeautifulSoup (bs4/html5lib) and Parsel dependencies that are imported at module import time.

Changes:

  • Make adaptive-crawler depend on crawlee[beautifulsoup,parsel] (self-referencing extras pattern already used by all).
  • Regenerate/update uv.lock to reflect the new extra dependency graph and metadata.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File Description
pyproject.toml Updates adaptive-crawler extra to include crawlee[beautifulsoup,parsel], ensuring transitive installation of bs4/parsel requirements.
uv.lock Updates locked optional-dependency and requires-dist metadata so the lock reflects the new adaptive-crawler extra composition.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Hide details View details vdusek merged commit b850cd6 into master Jul 3, 2026
38 checks passed
vdusek deleted the fix/adaptive-crawler-extra branch July 3, 2026 10:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
Pijukatel Pijukatel approved these changes

Assignees

vdusek

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

Footer

© 2026 GitHub, Inc.