How can I crawl all reviews of an app on Google Play Store?

Question

When I crawl an app page on Google Play Store, I could only crawl the first few pages of an app review.

(The next page of app review is showed up when I click the arrow button using the web browser)

How can I crawl the others of reviews?

Solution

I checked out Telegram's Google Play page and it seems all the reviews are loaded together with the rest of the page.

Reviews visible without clicking arrow

After clicking the arrow, the reviews section increases in size and shows full-length reviews

Then, you can look and older and "less helpful" reviews by clicking the right arrow.

However, all these reviews have been loaded from the start:

From there, you'd have to parse the HTML using your web crawler to get the reviews.

EDIT

Okay, after clicking the arrow a few times, Google Play starts to load some older reviews by pinging http://play.google.com/store/getreviews with a token of sorts.

This is a snippet of the response I got:

<div class="single-review">  <a href="/store/people/details?id=113340797493751556969">...

It seems to contain all the loaded reviews, so all you'll have to do is to parse that, or try to figure out how the token works so you can fetch from getreviews yourself.

Comments

[ycyoon]

Thanks for your answer. However, If you click the arrow button several times until loading mark is shown up upon the button, you'll see that the reviews which are shown up after the loading mark are not loaded on the HTML from the start.

[ycyoon]

Got it, thanks! Now, I can crawl all reviews of apps!!

[Amit Tripathi]

ycyoon could you please tell me how to do it?

[Nachiket]

@ycyoon if your project is opensource, can you share your code for crawling all reviews of an app?

[Jerry YY Rain]

Does anyone know how to get token? or how does this token generate?

[raditya gumay]

@ycyoon how you do that?

[Sharpless512]

Found how he did it. The token is always the same. Look at the form Data from his screenshot. It has a value pageNum. So basicly you can test this in Postman. Sent the same form-data as in the screenshot and iterate trough the pageNum