Google Search Console
This is a basic tool for every web analyst and SEO specialist. It provides a lot of extensive and very useful information that allows for a comprehensive look at the structure of the service or linking on the page.
After creating an account on Google Search Console and integrating the tool with the page, we have access to an intuitive panel.
To check if your site is generating indexing errors:
log in to your account.
go to
“Condition” → “Excluded”
“Condition” → “Error”
click “Not Found” and you will see a list of 404 errors.
Not found (404) in Search Console
Not found (404) in Search Console
The submitted URL could not be found (404) - Search Console
The submitted URL could not be found (404) - Search Console
However, remember that the tool rarely shows all 404 errors. Some of them may no longer exist. When Google Search Console notices a sudden increase or too many 404 errors, it will inform you about it.
Search Console alert - Increase in number of pages with “404” errors
Search Console alert - Increase in number of pages with “404” errors
However, treat the messages from GSC rather as an alert that something is wrong and use a tool that provides more precise data and indicates all 404 errors on the page, and specifically...
Screaming Frog SEO Spider
After installing the free version, you can scan 500 resources within your site, for a more extensive website I recommend the paid version.
In the top window, type the website address and click “Start”. In the “Status Code” column, you will find the results, which you can easily sort by server response status.
Screaming Frog error 404 address list
Screaming Frog error 404 address list
If you also want to find external 404 errors that your site links to, go to the "External" tab and search for addresses with status 404 in the same way.
In the case of more complex services, it is worth using the function "Bulk Export" -> "Response Codes" -> "Client Error (4xx) links". It allows you to generate an .xls file containing all detected 404 errors on our website (internal and external) along with information on which subpage they occur. What more could you want?
client error links Screaming Frog
client error links Screaming Frog
Note that Screaming Frog will show a status of 200 for db to data for trusted database source company false 404 errors, so if you notice any strange links, manually check to see if they are actually 404 pages.
Manual analysis
If there are few URLs in the page structure, you can check all addresses manually in the Google search engine using the command "site:strona.pl". This will give you a list of all indexed subpages for a given domain.
site search operator:
site search operator:
Ahrefs
A tool worth considering when analyzing 404 errors is Ahrefs, which provides several interesting functionalities in this area.
Outbound Link Analysis
Following the path “Outbound Links” -> “Broken Links” will give you a list of pages on your site that link to non-existent subpages, along with the anchor and the page it links to. These links should be updated or removed.
broken outbound links ahrefs
broken outbound links ahrefs
Inbound Link Analysis
“Link Profile” -> “Broken” - here you will find a list of external links that lead to deleted links on your site.
broken backlinks in Ahrefs
broken backlinks in Ahrefs
You can also check which 404 error generating pages have the most external links - select "Pages" -> "Top by Links" and then filter for HTTP 404 errors.
404 subpages with the most external links
404 subpages with the most external links
In the case of valuable linking domains, it is worth contacting the owner of the external portal and asking for the link to be replaced with an existing one. Below, in the paragraph - Use the 404 error on other websites - we explain why it is worth analyzing links leading to broken websites on competitors.
Server Log Analysis
Server logs contain information about when requests were sent to the server, where they came from, what specific pages were called, response codes, and much more. Server logs are collected in hosting resources. It is worth using dedicated tools to analyze them. To isolate from all the addresses and resources for which requests were sent to the server, those that return a 404 response code for the Google robot, it is worth using SEO Log File Analyzer from Screaming Frog.
Does your website need
