Crawler (bot, robot, spider)

« Back to Glossary Index

An automated program used by search engines to discover and analyze web pages by following links across the internet, enabling the engine to index content and rank it in search results.

« Back to Glossary Index