User-agent: * Allow: / # The map's data, which is served from this domain because project pages # inherit the organization's custom domain. It is a few hundred megabytes of # JSON that exists for one map to draw, it is rewritten every hour, and it is # not a service — see github.com/oceansensing/realtime-data-repo. Nothing is # gained by indexing it and a crawler would pay for all of it. Two paths: # the live pipeline, and its frozen predecessor. Disallow: /realtime-data-repo/ Disallow: /ocean-data-repo/ # The bloom photographs, for the same reason and one more: they are lab # observations with people and effort behind them, not stock imagery. The # page that presents them is indexable; the image store is not. Disallow: /hab-data-repo/ # The development visualizer: a sandbox build of the map, deliberately # unfinished and sometimes deliberately wrong. It reads the same live data as # the published map, so there is nothing here a crawler needs that # /visualization/ does not already offer, and indexing it would put an # experimental page in front of people looking for the real one. Disallow: /dev/ Sitemap: https://oceansensing.org/sitemap-index.xml