We use GraphQL for our API, to simplify integrations and give you greater flexibility by allowing you to request the data you want, rather than relying on what we make available.

Yes, the interface can be white-labeled with your own logo.

Yes, the Lumar GraphQL API is available for all users.

Usage is under a fair usage policy, but if you have very specific requirements, feel free to run them by us at– support@lumar.io.

You can find our API documentation here: API Documentation.

Most sites never experience a site slowdown whilst using Lumar. The only time this may happen is if a site’s server capacity is not able to handle user demand, or there is an increase in user demand with Lumar running at the same time.

With Lumar, you can control the maximum speed of the crawler to prevent any site performance slow down. You can also optimize your crawl activity further, by increasing your crawl rate during known quiet periods e.g. 1am-5am. We always recommend checking with your WebOps team to ascertain the fastest crawl speed that your site can handle.

Yes, you can set your crawl to run at certain times and at certain speeds (URLs per second), and set up schedules for your crawls (e.g. hourly weekly, daily, weekly, monthly, etc.).

Lumar will always identify itself by including ‘deepcrawl’ (our previous name) or ‘lumar’ within the user agent string.

By default, our crawler requests come from one of two IP addresses (52.5.118.182 or 52.86.188.211). However, if the crawler is running in stealth mode or you have specifically asked the crawler to originate the request from a different region, the IP address might be different.

For whitelisting purposes, please ask your web team to whitelist both the IP addresses mentioned above. You can also use your own proxy service, and we can provide a private proxy for a small monthly fee if required.

Lumar offers a wide range of user agents to use for a crawl including all of the most common search engines, desktop browsers, and mobile devices. You can also add your own custom user agents.

By default, we crawl as Googlebot Smartphone and can be identified by the following string:

“Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +https://www.google.com/bot.html) https://www.lumar.io/spdr/”.

However, some existing customers will see Deepcrawl at the end, from before our rebrand to Lumar. In this case, it can be identified by this string:

“Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +https://www.google.com/bot.html) https://deepcrawl.com/bot”.

Yes, our sophisticated crawling technology allows you to run crawls, large and small, all at the same time. We’ll then send you a notification when they’re complete and ready for you to analyze.

All data is stored using Amazon Web Services which has been architected to be one of the most secure cloud computing environments available. The crawl data is stored in a database on EC2 servers until the crawl is archived or deleted. The report data and backups are archived in S3. We use a VPN and security groups to prevent unauthorized access to the data.

The majority of analytics packages such as Google Analytics, Webtrends or Omniture use a JavaScript tracking tag that runs inside the user’s browser. Lumar does not run these scripts and will not affect your analytics data.

Some older analytics packages use log file data stored on the web server. This data can be affected by any crawling activity, including Google or Bing and therefore Lumar too.

No, you don’t usually need to do anything under normal circumstances to crawl a public website. No additional tracking tags or authentication processes are normally required.

It’s important to advise the team responsible for hosting your website to avoid your crawl being blocked by a bot management solution.

If you want to crawl a private website on your own network—like e.g. a test website or staging environment—then you will need to allow access to Lumar by identifying the user agent or IP address and allowing access to your network, and/or any basic authentication or DNS configurations needed.

Lumar is 100% focused on website technical health, giving you insights across a range of technical elements such as technical SEO, site speed, accessibility, performance and more. 

Extensive dashboards and charts give you a layered approach to the data, helping you understand what’s happening and what you need to do, rather than overwhelming you with data. 

Lumar is the only platform that mitigates the risk of lost traffic with automated QA testing and comprehensive trend tracking. It’s also the only platform that provides a single-view dashboard for monitoring multiple domains or important parts of your site. 

Because Lumar is run as a cloud-based service, the size of crawls that can be run is much larger compared to software-based crawlers that run on your local computer. Our crawler is also the fastest available on the market today, with crawl speeds of up to 450 URLs/s. With Lumar, you can crawl as fast as your site will allow, getting you the insights you need as quickly as possible. 

Lumar also provides a high level of customization and control for more experienced users, allowing crawls to be tailored to suit a specific project.

Lumar is a website intelligence platform, giving you eye-opening insights into the technical health of your website, including technical SEO, site speed,  accessibility and custom metrics.

Our ultra-fast crawler gives maximum flexibility, and is the cornerstone of the Lumar platform, helping you:

• Analyze your site for technical improvements that help drive traffic and create great experiences for all. 
• Monitor site changes and trends to help prioritize effectively and fix issues fast. 
• Protect your site from traffic-draining technical errors or accessibility issues with automated QA testing.
• Show the Impact of technical website health improvements.
• Integrate with your tech stack to get even deeper insights.

You can use Lumar for: 

• Website technical health and accessibility audits
• Monitoring website technical health
• SEO and accessibility QA testing
• Site migrations
• Building organic search strategies

You can also get bespoke advice and strategies from our team of world-class technical SEO experts and accessibility partners.

Protect can run over 200 different tests on your domain. Each test can be set with a severity level of Fail or Warning. A Fail will stop your build and a Warning will send a notification of the issue via Email or Slack, but allow the build to continue.

You can apply a threshold against each test and can choose between a percentage or an absolute number.

With fast-moving, or large, complex websites, there’s a risk of new code introducing website health issues which can result in sudden losses of traffic, rankings, and revenue. 

Lumar’s Protect gives development teams the ability to test their code for website health impact before it goes live, with automated tests. Protect helps reduce the time and cost of rollbacks or fixing bugs in the live site, and allows better collaboration between development and SEO teams.

Yes, integration with CI/CD tools allows Protect to automatically stop code being pushed to production when tests fail.

Protect can connect to all major CI /CD tools, including pre-built integrations with GitHub, Jenkins, CircleCI, and Azure DevOps.

For other solutions, we provide two ways of integrating – either via the API or through a shell script. We provide full step- by- step instructions for both scenarios as well as comprehensive API documentation.

Notifications can be sent at the end of each test via Email or Slack. You can also integrate with your Jira backlog and create a customizable ticket for each failed test. 

As well as Password authentication, you can use custom DNS and / or whitelisted IP.

Yes, we provide a selection of options for users for authentication and white listing like a static IP address, custom user-agent and request headers.

We do not limit the number of different domains you can crawl. We do have limits on the number of ‘Active’ projects in your account, and provide a set number of URLs that can be crawled in the billing period. These limits will be set based on your needs and can be easily amended as your needs change.

Your data will be available in the Lumar platform until the account expires. You can export your data before the account expires, to continue using the data.

We price our solutions based on your individual needs. We’ll work with you to understand your requirements, and deliver a tailored proposal.

DeepRank is a measurement of internal link weight calculated in a similar way to Google’s basic PageRank algorithm. Lumar stores every internal link and starts by giving each link the same value. It then iterates through all the found links a number of times, to calculate the DeepRank for each page, which is the sum of all link values pointing to the page. With each iteration, the values move towards their final value.

It is a signal of authority, and can help to indicate the most important URLs in the current report, or within the entire crawl.

Learn more about how DeepRank in our dedicated product guide.

Every report is assigned a weight, to represent the importance of the issue and it’s potential impact. Reports are also given a sign, either positive, negative, or neutral. The list of issues is filtered to negative reports, and ordered by the number of items in the report, multiplied by the weight. This is why the issues are rarely displayed in numerical order. The changes are ordered by the number of added or removed issues in the report, multiplied by their weight.

In addition to calculating the URLs which are relevant to a report, we also calculate the changes in URLs between crawls. If a URL appears in a report and wasn’t in that report in the previous crawl, it will be included in the ‘Added’ column of the Changes report’. If the URL was included in the previous crawl, and is present in the current crawl, but is no longer in that specific report, then it is reported in the ‘Moved’ column. If the URL was in the previous crawl, but is not included in any report in the current crawl, it is included in the ‘Missing’ column (e.g. the URL may have been unlinked since we last crawled, or may now fall outside of the scope of the crawl).

When your account expires, it will fall dormant in case you wish to reactivate it at any time. If you wish to have all of your data permanently deleted, you can request this via our support team.

Crawl data, including all tables used to display reports, is backed up in Amazon S3 storage, which is Write Once Read Many and therefore highly reliable. All user and account data is backed up every hour.

Reports only become available once a crawl has been finalized. This is because the majority of the calculations Lumar performs, such as duplication detection and internal linking analysis, require a complete set of page URLs before they can begin. It also ensures you get a complete set of reports, which is essential for effective prioritization.

Yes, you can generate a private or public link to any of the reports in Lumar. Private links are only accessible to other Lumar users who are members of the account, whereas public links allow anyone to view the crawls and reports within the project. 

You can also set an expiration time frame when sharing the report, ranging from 24 hours to 6 months.