Google AdWords disapproved site as "destination not working", but it is reachable

Question

Our ads got disapproved by Google and the gist of the why is below:

Ad Status: Disapproved
Ad Issue(s): Destination not working

What's confusing is that our final Destination IS working, in fact I invite you to take a look at it https://montessori.edokiacademy.com/fr/

It's accessible via Chrome, Firefox, Safari, Opera, Edge, Brave... works on IOS and android too.

It's also accessible from anywhere in the world according to online test sites.

I've had a little email chat with AdWords' customer support and they came back saying they couldn't crawl our website. Which would be odd, since our robot.txt file is incredibly simple and valid according to Google's robots.txt tester tool

User-agent: *
Disallow: 

More than that, the webmaster tools indicate we are getting crawled:

I pointed this out to customer support and they stopped replying.

Then it gets better...

We re-submitted our ads with this URL (notice the useless param at the end) https://montessori.edokiacademy.com/fr/?param=kampn and it got approved! ...For a week. This morning it got disapproved too.

The last suspects

Something MUST be wrong with our website (Google is spelled with a capital G after all). So here are my last hopes...

According to the search console, there have been three http500 errors and four http404.

The 404s are somewhat mysterious because I can't find those links anywhere on our website, but they aren't the destination of our ad anyway so it shouldn't be the problem we're looking for at the moment.

The 500 errors are surprising because our website is up and running, maybe a little downtime has occured and Google tried fetching at that exact moment? Would something like a 1 minute downtime of our website be enough for disapproval?

The only other problem I have found is Safari showing 404s for .js.map files, but those being .js.map files, they don't prevent normal usage of the website.

Thank you so much for making it all the way down here, any clue is much appreciated!

Solution

WE FOUND THE PROBLEM.

The thing was, our server tries to redirect users based on their locale, but bots don't have a locale, so the server was trying to parse a non-existant string and crashed. This was hard to find out, because it would only happen when a BOT was trying to access the site. Manual testing was seamless.

A simple if(locale == undefined){ go to /en} did the trick.

So, the moral of the story is: test your website as a search engine bot would (using online test sites) on top of how a user would.

Comments

[dan]

Using different online testing sites, I noticed some 500 errors too. There are a number of possibilities: Your server or network might be setup to reject and blacklist IP's performing multiple requests, or those missing expected user-agents (check for firewall rules, htaccess code, security modules, etc...); your server might be having intermittent hardware issues (from bottlenecks, failing drives, etc...); content being served externally might also suffer from the above issues... You should check your server error logs and work with your provider to track down what might be the issue.

[dan]

Not sure I remember which ones returned errors, but I do remember this one returned all 500 errors yesterday, but today is returning only 200's. Note that you can get 500 errors if you're reaching capacity for hosting plan resources (some calculate this daily too). Good luck.

[Félix Paradis]

WE FOUND THE PROBLEM. The thing is, our server tries to redirect users based on their locale, but bots don't have a locale, so the server was trying to parse a non-existant string and crashed. This was hard to find out, because it would only happen when a BOT was trying to access the site. Manual testing was seamless. Hopefully this can help someone in the future. Thanks again @dan!

[dan]

No problem, thanks for the update and glad you figured out the problem. Typically we place questions requiring troubleshooting on hold because it doesn't fit the format of the site very well, but I think this might be helpful to others so I'm reopening it (otherwise it will be automatically removed if it remains on hold). If you can add your comment above as an answer and accept it when you can, that would be helpful, and would add to your reputation on the site - thanks.

[Félix Paradis]

Cheers, I'll do that right away! (feel free to edit :p)