Page object API

Product

class zyte_common_items.BaseProductPage(**kwargs)

Bases: BasePage, DescriptionMixin, PriceMixin, Returns[Product], HasMetadata[ProductMetadata]

BasePage subclass for Product.

class zyte_common_items.ProductPage(**kwargs)

Bases: Page, DescriptionMixin, PriceMixin, Returns[Product], HasMetadata[ProductMetadata]

Page subclass for Product.

class zyte_common_items.AutoProductPage(**kwargs)

Bases: BaseProductPage

Product list

class zyte_common_items.BaseProductListPage(**kwargs)

Bases: BasePage, Returns[ProductList], HasMetadata[ProductListMetadata]

BasePage subclass for ProductList.

class zyte_common_items.ProductListPage(**kwargs)

Bases: Page, Returns[ProductList], HasMetadata[ProductListMetadata]

Page subclass for ProductList.

class zyte_common_items.AutoProductListPage(**kwargs)

Bases: BaseProductListPage

Product navigation

class zyte_common_items.BaseProductNavigationPage(**kwargs)

Bases: BasePage, Returns[ProductNavigation], HasMetadata[ProductNavigationMetadata]

BasePage subclass for ProductNavigation.

class zyte_common_items.ProductNavigationPage(**kwargs)

Bases: Page, Returns[ProductNavigation], HasMetadata[ProductNavigationMetadata]

Page subclass for ProductNavigation.

class zyte_common_items.AutoProductNavigationPage(**kwargs)

Bases: BaseProductNavigationPage

Article

class zyte_common_items.BaseArticlePage(**kwargs)

Bases: BasePage, Returns[Article], HasMetadata[ArticleMetadata]

BasePage subclass for Article.

class zyte_common_items.ArticlePage(**kwargs)

Bases: Page, Returns[Article], HasMetadata[ArticleMetadata]

Page subclass for Article.

class zyte_common_items.AutoArticlePage(**kwargs)

Bases: BaseArticlePage

Article list

class zyte_common_items.BaseArticleListPage(**kwargs)

Bases: BasePage, Returns[ArticleList], HasMetadata[ArticleListMetadata]

BasePage subclass for ArticleList.

class zyte_common_items.ArticleListPage(**kwargs)

Bases: Page, Returns[ArticleList], HasMetadata[ArticleListMetadata]

Page subclass for ArticleList.

class zyte_common_items.AutoArticleListPage(**kwargs)

Bases: BaseArticleListPage

Article navigation

class zyte_common_items.BaseArticleNavigationPage(**kwargs)

Bases: BasePage, Returns[ArticleNavigation], HasMetadata[ArticleNavigationMetadata]

BasePage subclass for ArticleNavigation.

class zyte_common_items.ArticleNavigationPage(**kwargs)

Bases: Page, Returns[ArticleNavigation], HasMetadata[ArticleNavigationMetadata]

Page subclass for ArticleNavigation.

class zyte_common_items.AutoArticleNavigationPage(**kwargs)

Bases: BaseArticleNavigationPage

Business place

class zyte_common_items.BaseBusinessPlacePage(**kwargs)

Bases: BasePage, Returns[BusinessPlace], HasMetadata[BusinessPlaceMetadata]

BasePage subclass for BusinessPlace.

class zyte_common_items.BusinessPlacePage(**kwargs)

Bases: Page, Returns[BusinessPlace], HasMetadata[BusinessPlaceMetadata]

Page subclass for BusinessPlace.

class zyte_common_items.AutoBusinessPlacePage(**kwargs)

Bases: BaseBusinessPlacePage

Real estate

class zyte_common_items.BaseRealEstatePage(**kwargs)

Bases: BasePage, Returns[RealEstate], HasMetadata[RealEstateMetadata]

BasePage subclass for RealEstate.

class zyte_common_items.RealEstatePage(**kwargs)

Bases: Page, Returns[RealEstate], HasMetadata[RealEstateMetadata]

Page subclass for RealEstate.

class zyte_common_items.AutoRealEstatePage(**kwargs)

Bases: BaseRealEstatePage

Job posting

class zyte_common_items.BaseJobPostingPage(**kwargs)

Bases: BasePage, DescriptionMixin, Returns[JobPosting], HasMetadata[JobPostingMetadata]

BasePage subclass for JobPosting.

class zyte_common_items.JobPostingPage(**kwargs)

Bases: Page, DescriptionMixin, Returns[JobPosting], HasMetadata[JobPostingMetadata]

Page subclass for JobPosting.

class zyte_common_items.AutoJobPostingPage(**kwargs)

Bases: BaseJobPostingPage

Social media post

class zyte_common_items.BaseSocialMediaPostPage(**kwargs)

Bases: BasePage, Returns[SocialMediaPost], HasMetadata[SocialMediaPostMetadata]

class zyte_common_items.SocialMediaPostPage(**kwargs)

Bases: Page, Returns[SocialMediaPost], HasMetadata[SocialMediaPostMetadata]

class zyte_common_items.AutoSocialMediaPostPage(**kwargs)

Bases: BaseSocialMediaPostPage

Request templates

class zyte_common_items.SearchRequestTemplatePage(**kwargs)

Bases: ItemPage[SearchRequestTemplate], HasMetadata[SearchRequestTemplateMetadata]

Mixins

class zyte_common_items.pages.DescriptionMixin

Provides description and descriptionHtml field implementations.

description: str

Plain-text description. The default implementation makes it from the descriptionHtml field if that is user-defined.

descriptionHtml: str

HTML description. The default implementation makes it from the description field if that is user-defined.

class zyte_common_items.pages.PriceMixin

Provides price-related field implementations.

currency: str

Price currency ISO 4217 alphabetic code (e.g. "USD"). The default implementation returns self.CURRENCY if this attribute is defined.

currencyRaw: str

Price currency as it appears on the webpage (no post-processing), e.g. "$". The default implementation uses the data extracted by price_processor() from the price field.

Custom page objects

Subclass Page to create your own page object classes that rely on HttpResponse.

If you do not want HttpResponse as input, you can inherit from BasePage instead.

Your subclasses should also inherit generic classes web_poet.pages.Returns and zyte_common_items.HasMetadata to indicate their item and metadata classes.

class zyte_common_items.pages.base._BasePage(**kwargs)

class zyte_common_items.BasePage(**kwargs)

Bases: _BasePage

Base class for page object classes that has RequestUrl as a dependency.

metadata

Data extraction process metadata.

dateDownloaded is set to the current UTC date and time.

probability is set to 1.0.

url: str

Main URL from which the data has been extracted.

no_item_found() → ItemT

Return an item with the current url and probability=0, indicating that the passed URL doesn’t contain the expected item.

Use it in your .validate_input implementation.

class zyte_common_items.Page(**kwargs)

Bases: _BasePage, WebPage

Base class for page object classes that has HttpResponse as a dependency.

metadata: zyte_common_items.Metadata

Data extraction process metadata.

dateDownloaded is set to the current UTC date and time.

probability is set to 1.0.

url: str

Main URL from which the data has been extracted.

no_item_found() → ItemT

Return an item with the current url and probability=0, indicating that the passed URL doesn’t contain the expected item.

Use it in your .validate_input implementation.

class zyte_common_items.HasMetadata

Inherit from this generic mixin to set the metadata class used by a page class.