Fetch URLs from a search engine — fetch

Web (general). Queries a search engine and returns result URLs. Use read_urls to get content from these URLs.

fetch_urls(query, n_pages = 1, date_filter = "w")

Arguments

query: Search query string.
n_pages: Number of search result pages to fetch (default 1). ~30 results per page.
date_filter: Recency filter: "d" (day), "w" (week), "m" (month), or "none" (default "w").

A data.table with columns search_engine, url, is_excluded, and optionally path_depth.

if (FALSE) { # \dontrun{
urls_dt <- fetch_urls("R programming nlp", n_pages = 1)
urls_dt$url
} # }