Crawlability and indexing
28 procedures covering robots.txt, sitemap.xml, canonical, noindex and nofollow, hreflang. Out of scope here: keyword research, content strategy, link building, rankings.
Indexing Directives
Canonical and Duplicates
Structured Data and Social
Robots and Crawling
Sitemaps
Rendering for Crawlers
Questions people ask about this area
- how to verify robots txt
- how to configure robots.txt to allow everything
- robots.txt allow root only disallow everything else
- listing both sitemaps and sitemap index files in robots.txt
- how to detect search engine bots
- how to validate compliant xml sitemap
- should a sitemap have every url
- google sitemap date format