Good becomes the enemy of better

I was wondering when was the last time I went on to the second page of google search results. More often than not I have changed my key words and tried to get the content I was looking for rather than going to the next page. And this seems to work for me almost all the times. Now, this is something that google may not necessarily like because if there is a good relevant search result that doesn't appear on the first page and all the users are like me, chances of someone hitting it would be nil which means the rank of the page will not necessarily benefit from users like me. Of course, it may be the case that there are dozens other things that would impact the rankings but I would still assume that user clicking a link would certainly be one of the important criteria in page ranking a website/page. Not much of a thought, but thought of just noting it down.