One of the most invaluable tasks or tools you can utilize for SEO is tracking web changes. Not only do I recommend tracking changes on your own websites but also track your competitors’ sites. Unless you are the only person who can push changes to your production site then it’s impossible to know every single update on a site, and that can have SEO ramifications.
Why Is It Important To Track Website Changes?
Many years ago, I noticed a sudden drop in all rankings for one of my clients. Upon manual digging through the page source code, I noticed that all of the internal links had been no-followed. I can’t remember exactly why but it was clearly a mistake by their dev team. The site wasn’t a huge site and not too complex so I was able to identify the issue on my own then communicate the problem. However, if the site were larger or more complex then I really would have benefitted from a dedicated tool that can monitor changes like this.
Another type of change tracking is server log analysis. I’ve worked on a site that had massive de-indexation and overnight traffic loss. It turned out that Google Bot had been blocked by some internal security updates. As a user, no one could have noticed this as humans were not blocked but the server log tool made it easy to isolate the issue and get it fixed.
Tracking competitor website updates can be invaluable as well. Whether it’s update new features, changing the underlying code, or drastically changing the internal link count these are all things that can impact their rankings as well as yours. For example, I was monitoring a competitor site in the travel industry and had noticed that their rankings in SEMRush had started to tank and our website was improving as a result. When I checked my track changes tool it turned out that website had quadrupled the number of links in their sitewide navigation menu because we initially had a lot more internal links than they did. The problem with that is it diluted the link equity flowing to their most important pages. When rebuilding our nav menu I learned from this and took the opposite approach by reducing the number of links which ended up further helping our rankings to our top pages. This was also long overdue to improve user experience.
In larger organizations, I guarantee you that many updates will slip through the cracks unless you have a way to monitor web changes. When the topic of budget comes up, I always compare change monitoring tools to health insurance. You may hardly ever use it for the longest time but the one time you need it you will be grateful that you have coverage. Monitor for web changes is the same concept. It can save you a lot of time and money.
Recommended Website Update Monitoring Tools
My favorite tools for monitoring website updates are SEORadar and Jet Octopus. SEORadar comes with many out-of-the-box alerts that pertain to SEO but can also be customized to suit your needs. Here’s an example of what their dashboard looks like.
Another useful feature is comparing the page source difference between crawls which highlights the SEO elements but you can also compare the raw HTML as well. If you have Javascript-heavy sites then you can also compare the source vs rendered code to see if there are substantial differences between what a user sees vs search engines which may impact organic search performance.
A relatively newcomer to the SEO tool scene, Jet Octopus is a tool that I started using a few years ago. If you’re managing large sites then a cloud-based crawler is essential. If you have ever tried to crawl a six-figure or multi-million page site with a desktop crawler like Screaming Frog good luck. I have a maxed out Mac in my office and crawls like that will bring my computer down to its knees plus take a long time to finish. Jet Octopus also stores all of the crawls so it’s very easy to compare data sets over time. You can combine the standard crawl data with all sorts of analytics and server log data. By filtering and joining data sets, that’s an extremely powerful thing you can do to monitor SEO updates and diagnostics.
Rank Tracking & Algorithm Updates
It’s important to track rankings as a way to measure progress but also for diagnostic reasons. Search engine rankings are always in flux which most of the time you won’t have to worry about daily fluctuations but drastic changes over a longer period of time can be a sign of deeper issues. Keyword rank tracking tools like STAT Search Analytics, SEMRush and Advanced Web Ranking are all tools that I like. I would also recommend tracking the same keyword sets from your competitors.
Or sometimes industry-wide ranking changes might be happening due to major Google core updates so it helps to rule things out before taking action. In a situation like this where the travel industry has major volatility there is probably nothing you can do about this so best not to overreact if your site was negatively impacted. Keep in mind that you can also make things worse by making unnecessary changes to your site. SEMRush Sensor is a good tool to monitor these types of updates in addition to following industry blogs for commentary.
There are other tools and methods to monitor SEO changes but these are some of my go-to solutions. Since SEO can have a lot of volatility you really have to stay on top of things. Having tools and processes in place can help mitigate SEO and business issues in a timely maner.