Lumar Protect (formerly Automation Hub) helps prevent loss of traffic or revenue from technical SEO errors being introduced in code updates. It allows you to set tests that will automatically run when new code is released. These tests will warn you of errors in the new code and automatically stop the release if any critical tests fail.
Tests can be established for over 200 metrics in the Lumar platform, plus any custom extractions that you’ve set up. Smart Thresholds now help to reduce the amount of time spent managing these tests in Protect.
What are Smart Thresholds?
Once enabled, Smart Thresholds use the data from the previous five test runs to automatically adjust thresholds based on the worst performance from those tests.
For example, when you start running a test suite in Protect you need to either run it manually at least 5 times, or set it up on a schedule that runs at least 5 times. Smart Thresholds will then kick in after the 5th run. If the threshold was set at 200 broken pages and you fix some of those broken pages, bringing the total down to 120 Broken Pages, Smart Thresholds will change the threshold to 121 Broken Pages without any manual intervention on your part.
As your site continues to improve, we keep the threshold in line with it so you’re aware of any negative spikes on the site. If your Broken Pages remain over the set threshold, you will continue to be notified that this test Failed unless you manually set the threshold higher. This is to ensure that a critical alert doesn’t get missed.
Websites are obviously ever-changing, so Smart Thresholds can ensure your thresholds will be kept updated based on how your website is changing over time, without manual intervention.
When to use Smart Thresholds?
Obviously you can use Smart Thresholds whenever you like. Like any automation though, it’s important to make sure it’s used efficiently. Automation is a great time saver for repetitive tasks that don’t require complex thinking, but when some errors in your site can lead to a significant loss of traffic or even revenue, your expertise is a vital factor in deciding on appropriate thresholds. For this reason, you may want to retain manual settings for the most critical tests for your site.
Fortunately, our Smart Thresholds can be enabled on a per test basis, so you may choose to enable it for those tests that result in a ‘warning’, but keep ‘fails’ that block the build manually set.
Of course you can also switch Smart Thresholds on and off at any time. This means you can turn them on to get a baseline for the thresholds, and then switch them off once that is set.
How to Set Up Smart Thresholds
From the Protect dashboard or the Test Suites page, click ‘New test suite’ in the top right corner of the screen.
In step 1 of the wizard you can give your test suite a name and add in the domain/subdomain that you want to run the tests on. You can also add in authentication details, change whitelisting options or add a custom DNS pair. Enter in the relevant details and then click save at the bottom of the section.
In step 2 you’ll set the URL Crawl Settings, including URL source, whether JavaScript rendering is enabled, maximum crawl speed and any URL limits or custom extractions you need. You can also access some Advanced Settings from this step. Again, enter in the relevant details and click save.
Step 3 is where you’ll choose the tests to run in the test suite (up to 100). You can use the search and filter options to narrow down the list of available tests, and check the box next to the relevant test to add it to the suite.
Once you’ve added the required tests to the test suite, you can then click the ‘Use Smart Thresholds’ option under test settings.
By default, ‘apply to all tests’ will be selected. If you’d like to use Smart Thresholds for the entire suite then you’re good to go. The Lumar platform will use data from the previous five crawls to automatically adjust the threshold.
To enable Smart Thresholds for individual test suites, choose ‘Apply to individual tests’. You can then open the relevant tests to enable Smart Thresholds for only selected tests.
And if you change your mind, you can always disable Smart Thresholds whenever you like.
If you choose not to use Smart Thresholds, you can then finalize the relevant elements of each test:
- Test severity. Choose whether the threshold breach will result in a fail (which stops the build) or a warning (where the build continues).
- Failure threshold type. Choose between a percentage of URLs or an absolute number.
- Failure threshold. Choose whether the failure threshold is greater than or equal to, or less than a particular percentage or absolute number.
- Jira ticket creation. You can also integrate Protect with Jira, to automatically create a ticket when a Fail or Warning is triggered.
