Next level plan for larger workloads with concurrent and recurring scheduled crawls.
For monitoring at scale
