Because the API for this tool is no longer reliable the development for this tool is on hold..
On-Page SEO Audit
Use the SEO List Crawler to quickly perform a SEO check on a list of URLs. You can enter up to a whopping 1000 URLs at a time with a daily limit of 2000 URLs. After submitting your list the tool will automatically extract important On-Page SEO elements. The returned results can be used to perform a technical SEO audit or generate a SEO report by using the Excel export function.
SEO elements this tool will check:
- Status code
- Destination URL (redirect)
- Page Title / Title tag content
- Page Title length
- Meta description content
- Meta description length
- H1 heading content
- Number of H1 headings found
- Link Count
- Canonical URL
- Twitter Profile URL
It’s important to know if a page / URL returns the right status code. Some status codes you can expect when entering an URL:
- 200, The request has succeeded. This is the most common status code and in most cases the preferred status code. Because it means search engine spiders and browsers can view / request this page.
- 404, The server hasn’t found a match for the entered URL and returned a 404 error. Depending on the cause of this problem you need to fix this issue or check if you entered the right URL (without typos).
- 301 or 302, These status codes are used to redirect visitors and spiders from URL 1 to URL 2. Redirects can be used for multiple purposes including site / URL migrations or to solve 404 error pages.
Destination URL (redirect)
When the spider detects a 301 or 302 status code it will also detect the final destination URL (the URL where visitors and spiders are redirected to). The destination URL is only visible in the Excel export to save some valuable space inside the HTML data table.
Page Title / Title tag
The SEO List Crawler will first check if the Page Title can be found if this is the case it will extract the content and it will perform a character count to see it stays within the max. visible (55 – 65 character) limit.
The meta description is extracted to check if a description has been added to the page and to check the character length of the content. To make sure your description is completely visible in the search results limit it to 150 / 160 characters.
The H1 Heading tag is extracted and displayed when found. The spider will also check how many H1 are found on a single page.
The link count number represents the total links found on a page (internal + external links). If you need a specified report please use this tool to get more detailed report: http://www.seoreviewtools.com/internal-link-analyzer/
When a canonical tag is found on a page it will be returned in the report and you can see which page should be index by search engines based on the canonical location.
Twitter Profile URL
The final metric returned by this tool (when found), is the Twitter Profile URL. This isn’t a On-Page SEO element, but it’s just nice information to have when you are looking for ways to contact the web master or author of an article.
SEO List Crawler FAQ:
- The SEO List Crawler obeys Robots.txt and Meta robot directives.
- Crawl up to 1K URLs at a time
- Daily URL crawl limit 2K
- For now it seems like the export function is only working, like it should, in Firefox and Google Chrome.
- When nested H1 elements are found the tool will only try to extract them to a certain depth. When this level is exceeded the tool will return a link to the tool specifically build for this purpose, enabling you extract the H1.
- How does Google interpret 302 (temporary) redirects? – https://www.seroundtable.com/google-302-301-permanent-redirect-21045.html
- What does this status code mean? Check out the complete list https://en.wikipedia.org/wiki/List_of_HTTP_status_codes
9 thoughts on “SEO List Crawler”
The tool is not working. There is not input field. I’ve tried it on different browsers without any luck
Thanks for the noticing this issue. I’ll have a look to find out what is the problem.
Just updated the tool and it’s ready for input again ;-)
I crawled my website here. But it says that there was no Meta Description found for it. But my website has a meta description. Also, it has mentioned “Not Found” in H1, H1 count, link count columns.
What could be the problem?
Thanks and Regards,
I also did a quick check for your site and the results look fine. This problem was probably caused by a temporary issue.
the download of the .cvs file on the http://www.seoreviewtools.com/seo-list-crawler/# is not working properly. I did a search with 500 url and the online result came out great. but when I hit the CVS Export button the file doesn’t work.
What URLs did you use.. Please send me the list so I can replicate your results.
And can you specify “.. the file doesn’t work”, does this mean you couldn’t open the file with Excel or Openoffice spreadsheets?
I”ve been working hard on this tool but there are still some issues I like to address to improve the functionality and usability.
(please check if your feedback is not already listed.. thanks!)
To be improved: