Skip to content

Add configurable User-Agent for VulnerableCode scrapers (Fixes #410)#2077

Open
warrior-hub wants to merge 2 commits intoaboutcode-org:mainfrom
warrior-hub:add-user-agent
Open

Add configurable User-Agent for VulnerableCode scrapers (Fixes #410)#2077
warrior-hub wants to merge 2 commits intoaboutcode-org:mainfrom
warrior-hub:add-user-agent

Conversation

@warrior-hub
Copy link

Problem:

  • Web scrapers did not declare a User-Agent.
  • Some websites block requests without a proper User-Agent.

Solution:

  • Added a default User-Agent: VulnerableCode
  • Configurable via environment variable: VULNERABLECODE_USER_AGENT
  • Created helper function http.get() to be used for all future requests

This addresses issue #410

@ziadhany
Copy link
Collaborator

ziadhany commented Jan 9, 2026

@warrior-hub, please fix the DCO and the CI (code style formatting)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants