Core Concepts
This section will walk through the different components of offchain
as well as how it works.
Please make sure to read this section first before diving into the rest of the documentation.
Key Components
- Pipeline: Orchestrates the metadata fetching and normalizing process for multiple tokens.
- Adapter: Parses the metadata url into an acceptable request format for the fetcher i.e. converting an IPFS Hash to a valid URL.
- Fetcher: Makes network requests to a given uri to fetch data.
- Parser: Parses raw data into a standardized metadata format
- ContractCaller: Makes RPC calls to NFT contracts to retrieve the uri if not provided.
Parser Types
CollectionParsers
: Used for specific NFT contract addresses that have unique metadata e.g. Autoglphys, Nouns, and ENS.SchemaParsers
: Used for general purpose formatting across many NFT collections e.g. Opensea Metadata Standard.
How Offchain Works
An overview of how offchain retrieves NFT metadata.
- The
Pipeline
is initialized with aContractCaller
, aFetcher
, a list ofAdapters
, and a list ofParsers
. - An array of
Tokens
is passed into the pipeline to fetch. - If no
uri
is provided for the tokens, theContractCaller
will fetch it by callingtokenURI(uint256)
on the contract. - The
Adapters
to attempt to format the uri into a valid url. - The
Fetcher
then makes a request to the url and attempts to get JSON metadata for the token. - The
Parsers
attempt to parse the token metadata into an acceptable format. - Each parser runs in the order they were passed in. By default, the ordering is
CollectionParsers
,SchemaParsers
, and thenCatchallParsers
. - The pipeline returns the result from the first parser that is successful, unless a
metadata_selector_fn
is specified. - However, the pipeline returns a
MetadataProcessingError
if no parser is successful for that specific token.
Token Interface
This interface is how an NFT is passed into the pipeline for fetching.
collection_address
: The token's contract address.token_id
: The unique identifier for a token within a collection.chain_identifier
: The network and chain for the token. Defaults to "ETHEREUM-MAINNET" if nothing is passed in.uri
: The url where the metadata lives. Defaults to fetching from the contract directly if nothing is passed in.
# Gets metadata for Azuki #40
from offchain.metadata.pipelines.metadata_pipeline import MetadataPipeline
from offchain.metadata.models.token import Token
pipeline = MetadataPipeline()
token = Token(
collection_address="0xED5AF388653567Af2F388E6224dC7C4b3241C544",
token_id=40,
chain_identifier="ETHEREUM-MAINNET",
uri="https://ikzttp.mypinata.cloud/ipfs/QmQFkLSQysj94s5GvTHPyzTxrawwtjgiiYS2TBLgrvw8CW/40"
)
metadata = pipeline.run([token])[0]
RPC Provider
By default, the pipeline uses https://cloudflare-eth.com
as the provider for the ContractCaller
.
This is a free Ethereum RPC provider, which means that it is very easy to exceed the rate-limit.
The code below illustrates how to use a custom RPC provider to prevent getting rate-limited if token uris need to be retireved from the contract:
from offchain import MetadataPipeline
from offchain.web3.contract_caller import ContractCaller
from offchain.web3.jsonrpc import EthereumJSONRPC
rpc = EthereumJSONRPC(provider_url=MY_PROVIDER_URL)
contract_caller = ContractCaller(rpc=rpc)
pipeline = MetadataPipeline(contract_caller=contract_caller)
Next, check out the Usage
section to see how to use offchain
.