Twitter in order to get more expansion and visibility for Twitter's millions of pages in SERPs. Twitter recently updated it's robot.txt file to being crawled it's millions pages through all major search engines.But the doubtfully matter is it not sure that actually big search engines accepts Twitter offer for what Twitter wants to update it's Robot.txt.
Instructively twitter wants to tell Search engines Google, Yahoo, Bing. it's own search result pages and other meta searcher's bots to crawl more from Twitter updated pages specifically making a discover from Twitter Olympics and hash tag pages.
Twitter Make Change to Discover More.Twitter now bringing a change to it's robot.txt by saying that
Twitter instructively telling Google to crawl its tweet/hashtag search results, but stop to crawl the other two types; Twitter’s instructions to Bing and other search engines are the same.
Authorising there's value to Twitter for having it hashtag and Olympics pages show in Google, Bing, Yahoo, and other search engine's SERPs in case of having much domain authority to rank well for several queries.But the question is that the big search engnes wants to index more from Twitter's result pages.
In front of Google webmaster Guidelines Google has been adamant over the years that it doesn’t want to show search results pages in its own search results to specifically diversifying webmasters to following the Twitter actions as saying about that
Related:
Instructively twitter wants to tell Search engines Google, Yahoo, Bing. it's own search result pages and other meta searcher's bots to crawl more from Twitter updated pages specifically making a discover from Twitter Olympics and hash tag pages.
Twitter Make Change to Discover More.Twitter now bringing a change to it's robot.txt by saying that
This change will help people find popular and helpful Twitter pages, such as the #olympics hashtag page (https://twitter.com/search/?q=%23olympics)Here the main purpose to updated robot.txt is only to allow search engines to crawl only twitter.com/search pages but not to for index searches for users.Twitter crucially wants expanding it's search result pages through Google and Bing's SERPs i.e., tweets and hashtags — while a stop sign for search engines bots from crawling its search results for users and other published contents like videos and images. Tweet and hashtag search results show via the /search path; searches for Twitter users show via the/search/users path; and searches for photos and videos show via /search/(keyword)/grid.
Twitter instructively telling Google to crawl its tweet/hashtag search results, but stop to crawl the other two types; Twitter’s instructions to Bing and other search engines are the same.
Authorising there's value to Twitter for having it hashtag and Olympics pages show in Google, Bing, Yahoo, and other search engine's SERPs in case of having much domain authority to rank well for several queries.But the question is that the big search engnes wants to index more from Twitter's result pages.
In front of Google webmaster Guidelines Google has been adamant over the years that it doesn’t want to show search results pages in its own search results to specifically diversifying webmasters to following the Twitter actions as saying about that
Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.
Related:
0 comments
Post a Comment