Files & downloads
Non-HTML resources linked from a regular page. A crawler should classify these by content type.
- datasheet.pdf —
application/pdf— A real (tiny) PDF — a mapper should record the URL but not parse it as a page. - products.json —
application/json— The product catalogue as JSON. - feed.xml —
application/rss+xml— RSS 2.0 feed of the blog. - llms.txt —
text/plain— An llms.txt site summary. - robots.txt —
text/plain— Crawl directives. - sitemap.xml —
application/xml— The XML sitemap. - Download the datasheet (with
downloadattribute)