ImageMoosedogCrawlState

AI Overview😉

  • The potential purpose of this module is to track and manage the crawling and indexing of images on the web. It seems to be responsible for determining whether an image should be crawled, indexed, or removed from the search results, and for keeping track of the reasons why an image may not be crawled or indexed.
  • This module could impact search results by influencing which images are included in the search results, and in what order they appear. For example, if an image is marked as "not crawled" or "url deleted", it may not appear in the search results at all. On the other hand, if an image is marked as "roboted" (i.e., crawled by a specific user agent), it may be given more weight in the search results. The "noIndexAfterTimestamp" field could also impact search results by causing images to be removed from the index after a certain period of time.
  • A website may change things to be more favorable for this function by ensuring that their images are easily crawlable and indexable. This could involve optimizing image file names and alt tags, using descriptive and concise URLs, and avoiding the use of robots.txt files or other mechanisms that might prevent Googlebot from crawling their images. Additionally, websites may want to ensure that their images are not marked as "url deleted" or "not crawled" unnecessarily, and that they are not using user agents that might be considered "roboted".

Interesting Module? Vote 👇

Voting helps other researchers find interesting modules.

Current Votes: 0

GoogleApi.ContentWarehouse.V1.Model.ImageMoosedogCrawlState (google_api_content_warehouse v0.4.0)

Attributes

  • code (type: String.t, default: nil) -
  • detailedReason (type: integer(), default: nil) - Each of the above not_crawled_reason will have a set of detailed reason defined in crawler/trawler/trawler_enums.proto.
  • internalStatus (type: GoogleApi.ContentWarehouse.V1.Model.UtilStatusProto.t, default: nil) - The status returned when RPCs are used to internally fetch the image (eg. from FIFE).
  • isTerminal (type: boolean(), default: nil) - Specifies if the current crawl state is terminal.
  • noIndexAfterTimestamp (type: String.t, default: nil) - Time in seconds since epoch after which this image should be considered unavailable.
  • notCrawledReason (type: String.t, default: nil) -
  • overrodeTerminalState (type: boolean(), default: nil) - When true, it means that a non-terminal state has overwrote a terminal one.
  • repid (type: String.t, default: nil) - The repid for the urls. This repid is the id given to the dupe cluster this url belongs to.
  • robotedAgents (type: String.t, default: nil) - A comma separated list of user agents for which this image should be considered roboted. All images are crawled using googlebot-images and this exists here purely for informative reasons.
  • url (type: String.t, default: nil) - The url at which we crawled this content. With us starting to use repids the crawl table key no longer is suggestive of the url. In addition this is used in Amarna to detect race conditions between a reference changing its crawl directive and the original crawl job finishing.
  • urlDeleted (type: boolean(), default: nil) - Set to true if the url is taken down by clients. This indicates that this crawl state is used to fast remove the crawl result of the url instead of waiting for Multiverse crawl results. For more information, please refer to go/amarna-url-deletion.

Summary

Types

t()

Functions

decode(value, options)

Unwrap a decoded JSON object into its complex fields.

Types

Link to this type

t()

@type t() :: %GoogleApi.ContentWarehouse.V1.Model.ImageMoosedogCrawlState{
  code: String.t() | nil,
  detailedReason: integer() | nil,
  internalStatus: GoogleApi.ContentWarehouse.V1.Model.UtilStatusProto.t() | nil,
  isTerminal: boolean() | nil,
  noIndexAfterTimestamp: String.t() | nil,
  notCrawledReason: String.t() | nil,
  overrodeTerminalState: boolean() | nil,
  repid: String.t() | nil,
  robotedAgents: String.t() | nil,
  url: String.t() | nil,
  urlDeleted: boolean() | nil
}

Functions

Link to this function

decode(value, options)

@spec decode(struct(), keyword()) :: struct()

Unwrap a decoded JSON object into its complex fields.