Exclude URLs in the web scraper | Voters

Exclude URLs in the web scraper

complete

Bastille Dingo

The possibility to mark URL's that you don't want to be scraped and added to the knowledge of the Pulse-chatbot

Created by Fleur Nouwens

Charl Haas

marked this post as

complete

Charl Haas

marked this post as

planned

Alexander Wijninga

marked this post as

open

Alexander Wijninga

marked this post as

planned

Used Ostrich

An addition; maybe based on robots.txt or a list of "scraped urls" that you can mark as exclude using a button in the GUI?