TrawlerHostBucketDataUrlList

AI Overview😉

  • The potential purpose of this module is to manage and monitor the crawling and indexing of URLs by Google's search engine. It appears to be tracking various metrics related to crawling, such as the number of active fetches, URLs in the queue, and resource usage.
  • This module could impact search results by influencing the order and frequency of URL crawls, which in turn affects the freshness and accuracy of search results. For example, if a website is deemed to be resource-intensive, Google may limit its crawl rate to prevent overload, which could impact its search engine ranking.
  • To be more favorable for this function, a website could optimize its resource usage, ensure efficient crawling, and provide clear crawling instructions (e.g., via robots.txt). Additionally, websites could consider partnering with Google to dedicate host load and increase their crawl capacity. By doing so, they may be able to increase their crawl rate and improve their search engine ranking.

Interesting Module? Vote 👇

Voting helps other researchers find interesting modules.

Current Votes: 0

GoogleApi.ContentWarehouse.V1.Model.TrawlerHostBucketDataUrlList (google_api_content_warehouse v0.4.0)

Per each list that wishes to schedule url we return one of these:

Attributes

  • ClientCanCrawl (type: boolean(), default: nil) - Is this client/requestorid allowed to crawl now? (based on resource use)
  • IsDefaultNode (type: boolean(), default: nil) - Is this the 'default' user's list
  • IsListForUrl (type: boolean(), default: nil) - Was this the list that a given request landed in?
  • NumCurrentFetches (type: integer(), default: nil) - # of current active fetches
  • NumUrls (type: integer(), default: nil) - # of urls currently in the queue
  • PartnerTotalCapacityQps (type: number(), default: nil) - Total qps for this partner dedicated hostload, for non partner it will be -1.0.
  • PartnerTotalUsedQps (type: number(), default: nil) - Currently used qps for this partner dedicated hostlaod, for non partner it will be -1.0.
  • RequestType (type: String.t, default: nil) - The type of the request (low latency vs. high throughput)
  • RequestorFp (type: String.t, default: nil) - The fp64 of the requestor string

Summary

Types

t()

Functions

decode(value, options)

Unwrap a decoded JSON object into its complex fields.

Types

Link to this type

t()

@type t() :: %GoogleApi.ContentWarehouse.V1.Model.TrawlerHostBucketDataUrlList{
  ClientCanCrawl: boolean() | nil,
  IsDefaultNode: boolean() | nil,
  IsListForUrl: boolean() | nil,
  NumCurrentFetches: integer() | nil,
  NumUrls: integer() | nil,
  PartnerTotalCapacityQps: number() | nil,
  PartnerTotalUsedQps: number() | nil,
  RequestType: String.t() | nil,
  RequestorFp: String.t() | nil
}

Functions

Link to this function

decode(value, options)

@spec decode(struct(), keyword()) :: struct()

Unwrap a decoded JSON object into its complex fields.