In this post you will learn how to build an NFT rarity tool. The project we will build will use Django, celery for asynchronous tasks and web3.py to interact with the Ethereum blockchain.

Final Code

You can find the full code for this project on GitHub in the Django NFT Sniper project. Here's a preview of the full code in action:

Project Setup

We will use the well-known Cookiecutter-Django template to bootstrap a Django project. You can setup a project to either be run locally, or with Docker. Because we will be using celery, I will setup my project to use Docker.

Bootstrap the project with:

cookiecutter gh:cookiecutter/cookiecutter-django

Follow the prompts. I recommend to select Y when you are prompted to add celery to the project. We will not be using celery in this tutorial but for future usage it will be better to use celery.

Make sure to continue the installation process - you can read more in the Cookiecutter-Django documentation.

Create an App

Create a new app to hold all the rarity tool logic:

docker-compose -f local.yml run --rm django python manage.py startapp sniper

Models

Our project will have a few models to store the NFT project, NFT attributes and unique NFTs. Inside the new app, in models.py add the following models:

from django.db import models


class NFTProject(models.Model):
    contract_address = models.CharField(max_length=100)
    contract_abi = models.TextField()
    name = models.CharField(max_length=50)  # e.g BAYC
    number_of_nfts = models.PositiveIntegerField()

    def __str__(self):
        return self.name


class NFT(models.Model):
    project = models.ForeignKey(
        NFTProject, on_delete=models.CASCADE, related_name="nfts"
    )
    rarity_score = models.FloatField(null=True)
    nft_id = models.PositiveIntegerField()
    image_url = models.CharField(max_length=200)
    rank = models.PositiveIntegerField(null=True)
    
    def __str__(self):
        return f"{self.project.name}: {self.nft_id}"


class NFTAttribute(models.Model):
    project = models.ForeignKey(
        NFTProject, on_delete=models.CASCADE, related_name="attributes"
    )
    name = models.CharField(max_length=50)
    value = models.CharField(max_length=100)

    def __str__(self):
        return f"{self.name}: {self.value}"


class NFTTrait(models.Model):
    nft = models.ForeignKey(
        NFT, on_delete=models.CASCADE, related_name="nft_attributes"
    )
    attribute = models.ForeignKey(
        NFTAttribute, on_delete=models.CASCADE, related_name="traits"
    )
    rarity_score = models.FloatField(null=True)

    def __str__(self):
        return f"{self.attribute.name}: {self.attribute.value}"

Web3

One of the most popular Python packages for interacting with the Ethereum blockchain is web3.py. Using this package we can interact with existing smart contracts.

Start by installing the web3 package with pip install web3 and rebuild your Docker images.

Interacting with the Bored Ape Yacht Club NFTs

For this tutorial we will focus on one NFT project - the Bored Ape Yacht Club.

To find the smart contract for any project, search BAYC in Etherscan's ERC721 token list. You can view the BAYC token here. Take note of the Contract value listed under the Profile Summary.

The next step is to view the contract code, which you can do by clicking on the Contract menu item. That takes you to this page. You can then view all of the methods that are available on the smart contract. Most of the methods on the contract are used for reading data.

The method we are interested in is the tokenURI method, which you can find right at the bottom of the code. The method takes in tokenId as a parameter and looks up the information for that specific NFT ID. For example you could provide a value of 7575 and it would return the information for the NFT BAYC #7575:

Smart Contract Address and ABI

You need two things to interact with a smart contract:

The smart contract address
The Application Binary Interface (ABI)

Both of these things can be found on Etherscan on the BAYC contract address page.

The contract address is shown in the URL of the link above, as well as in the page heading. The contract address is 0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D.

To get the ABI, navigate to Contract and scroll down to the Contract ABI section. There you will see a text input with a long value of JSON data like this:

[{"inputs":[{"internalType":"string","name":"name","type":"string"},{"internalType":"string","name":"symbol","type":"string"},{"internalType":"uint256","name":"maxNftSupply","type":"uint256"},{"internalType":"uint256","name":"saleStart","type":"uint256"}],"stateMutability":"nonpayable","type":"constructor"},{"anonymous":false,"inputs": .....

This is not the full ABI - it is cropped for better readability

This long text is the value of the ABI. We will use the contract address and ABI in the code.

Making Requests to the Ethereum Blockchain

To make a request you will need to setup a web3 provider. Following from the web3.py documentation:

The provider is how web3 talks to the blockchain. Providers take JSON-RPC requests and return the response. This is normally done by submitting the request to an HTTP or IPC socket based server.

One option is to run your own Ethereum node, but this is out of the scope for this tutorial so we will use a service instead.

Infura

Infura provides tools to help with blockchain development. Create an account (it's free), navigate to your dashboard and create a new project. In your project's settings you will find the PROJECT_ID . You will use that value to connect to your own web3 provider.

Fetching Trait Data from NFTs

A cool thing about NFTs is that you can attach data to the NFT such as text and images. This data is called metadata.

We are now going to write some code using the web3 Python package to interact with the BAYC contract and fetch the metadata for the BAYC #7575.

Here is a Django management command that can be run with either python manage.py fetch_nfts or docker-compose -f local.yml run --rm django python manage.py fetch_nfts

from django.core.management.base import BaseCommand
from web3.main import Web3

INFURA_PROJECT_ID = "<your_project_id>"
INFURA_ENDPOINT = f"https://mainnet.infura.io/v3/{INFURA_PROJECT_ID}"


class Command(BaseCommand):
    def handle(self, *args, **options):
        self.fetch_nfts(7575)

    def fetch_nfts(self, token_id):

        contract_address = "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"
        contract_abi = '<paste the ABI text in here>'

        w3 = Web3(Web3.HTTPProvider(INFURA_ENDPOINT))
        contract_instance = w3.eth.contract(address=contract_address, abi=contract_abi)

        print(f"Fetching NFT #{token_id}")
        data = contract_instance.functions.tokenURI(token_id).call()
        print(data)

fetch_nfts.py

In this script we are configuring an INFURA_ENDPOINT value that points to our Infura project. Make sure to replace <your_project_id> with your own value from your Infura project dashboard. We then use the web3 Python package to setup a connection to the Ethereum blockchain via Infura.

With the contract address and ABI we can call w3.eth.contract to connect to the contract. Once connected we can execute methods available on the contract - like we saw in the Etherscan contract code.

Specifically we are calling the tokenURI method. The syntax to do this might look strange at first. The most important part of this script is the following line:

data = contract_instance.functions.tokenURI(token_id).call()

This line calls the tokenURI function. Notice we use .call() . This is because we are reading data from the contract. If we wanted to write data to the contract, we would use .transact() . You can read more about this in the web3 docs.

After running the management command you should see the following printed in the terminal:

ipfs://QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq/7575

Result from calling the tokenURI method

This value is an IPFS url. InterPlanetary File System (IPFS) is a distributed file system. To view IPFS data you can use a service like ipfs.io.

Navigate to https://ipfs.io/ipfs/QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq/7575 and you will get the following JSON response:

{"image":"ipfs://QmTE1TK15CcmETgc6wwSNDNwMgF7PvH714GGq33ShcWjR7","attributes":[{"trait_type":"Eyes","value":"Sleepy"},{"trait_type":"Mouth","value":"Bored Unshaven"},{"trait_type":"Background","value":"Orange"},{"trait_type":"Hat","value":"Prussian Helmet"},{"trait_type":"Fur","value":"Black"},{"trait_type":"Clothes","value":"Sleeveless T"}]}

Now you can see the JSON data includes an image URL and a list of traits. Some of the traits are the mouth, background, hat etc. You can see the value for each trait as well. These traits are what we will use to calculate the rarity of the NFT.

To view the image you will need to use the IPFS URL value. Again you can use ipfs.io and view the image here.

Storing NFT Data

Now that we understand what type of data exists in an NFT, we will store that data using our Django models.

Inside sniper/admin.py make sure to register all of the models:

from django.contrib import admin
from .models import NFTProject, NFT, NFTTrait, NFTAttribute


class NFTAdmin(admin.ModelAdmin):
    list_display = ["nft_id", "rank", "rarity_score"]
    search_fields = ["nft_id__exact"]


class NFTAttributeAdmin(admin.ModelAdmin):
    list_display = ["name", "value"]
    list_filter = ["name"]


admin.site.register(NFTProject)
admin.site.register(NFTTrait)
admin.site.register(NFT, NFTAdmin)
admin.site.register(NFTAttribute, NFTAttributeAdmin)

Go to the Django admin on http://127.0.0.1:8000/admin and create a new NFTProject with all of the BAYC data:

We're going to modify our Django script so that it creates all the NFTs and links them to the NFTProject we just created.

from django.core.management.base import BaseCommand
import requests
from web3.main import Web3
from djsniper.sniper.models import NFTProject, NFT, NFTAttribute, NFTTrait

INFURA_PROJECT_ID = "<your_project_id>"
INFURA_ENDPOINT = f"https://mainnet.infura.io/v3/{INFURA_PROJECT_ID}"


class Command(BaseCommand):
    def handle(self, *args, **options):
        self.fetch_nfts(1)

    def fetch_nfts(self, project_id):
        project = NFTProject.objects.get(id=project_id)

        w3 = Web3(Web3.HTTPProvider(INFURA_ENDPOINT))
        contract_instance = w3.eth.contract(
            address=project.contract_address, abi=project.contract_abi
        )

        # Hardcoding only 10 NFTs otherwise it takes long
        for i in range(0, 10):
            ipfs_uri = contract_instance.functions.tokenURI(i).call()
            data = requests.get(
                f"https://ipfs.io/ipfs/{ipfs_uri.split('ipfs://')[1]}"
            ).json()
            nft = NFT.objects.create(nft_id=i, project=project, image_url=data["image"].split('ipfs://')[1])
            attributes = data["attributes"]
            for attribute in attributes:
                nft_attribute, created = NFTAttribute.objects.get_or_create(
                    project=project,
                    name=attribute["trait_type"],
                    value=attribute["value"],
                )
                NFTTrait.objects.create(nft=nft, attribute=nft_attribute)

Adjusted fetch_nfts.py

After running the script you should see 10 NFTs inside the Django admin. You should also see 48 nft attributes which you can filter using the list filter on the side.

Calculating NFT Rarity

Rarity Tools was one of the first projects that you could use to calculate NFT rarity. We will be calculating rarity using the same formula as Rarity Tools. You can also read more about how the rarity score is calculated.

The formula to calculate the rarity is as follows:

[Rarity Score for a Trait Value] = 1 / ([Number of Items with that Trait Value] / [Total Number of Items in Collection])

Now we will write a second script to calculate the rarity and rank of each NFT:

from django.core.management.base import BaseCommand
from django.db.models import OuterRef, Func, Subquery
from djsniper.sniper.models import NFTProject, NFTAttribute, NFTTrait


class Command(BaseCommand):
    def handle(self, *args, **options):
        self.rank_nfts(1)

    def rank_nfts(self, project_id):
        project = NFTProject.objects.get(id=project_id)

        # calculate sum of NFT trait types
        trait_count_subquery = (
            NFTTrait.objects.filter(attribute=OuterRef("id"))
            .order_by()
            .annotate(count=Func("id", function="Count"))
            .values("count")
        )

        attributes = NFTAttribute.objects.all().annotate(
            trait_count=Subquery(trait_count_subquery)
        )

        # Group traits under each type
        trait_type_map = {}
        for i in attributes:
            if i.name in trait_type_map.keys():
                trait_type_map[i.name][i.value] = i.trait_count
            else:
                trait_type_map[i.name] = {i.value: i.trait_count}

        # Calculate rarity
        """
        [Rarity Score for a Trait Value] = 1 / ([Number of Items with that Trait Value] / [Total Number of Items in Collection])
        """

        for nft in project.nfts.all():
            # fetch all traits for NFT
            total_score = 0

            for nft_attribute in nft.nft_attributes.all():
                trait_name = nft_attribute.attribute.name
                trait_value = nft_attribute.attribute.value

                # Number of Items with that Trait Value
                trait_sum = trait_type_map[trait_name][trait_value]

                rarity_score = 1 / (trait_sum / project.number_of_nfts)

                nft_attribute.rarity_score = rarity_score
                nft_attribute.save()

                total_score += rarity_score

            nft.rarity_score = total_score
            nft.save()

        # Rank NFTs
        for index, nft in enumerate(project.nfts.all().order_by("-rarity_score")):
            nft.rank = index + 1
            nft.save()

rank_nfts.py

There are a few things happening in this script. The first thing we do is calculate the number of traits each attribute has. Here we use a Subquery so that we can annotate the count using a foreign-key lookup.

trait_count_subquery = (
    NFTTrait.objects.filter(attribute=OuterRef("id"))
	.order_by()
	.annotate(count=Func("id", function="Count"))
	.values("count")
)

attributes = NFTAttribute.objects.all().annotate(
    trait_count=Subquery(trait_count_subquery)
)

The data returned from this query looks like this:

('Earring', 'Silver Hoop', 1)
('Background', 'Orange', 2)
('Fur', 'Robot', 3)
('Clothes', 'Striped Tee', 1)
('Mouth', 'Discomfort', 1)
('Eyes', 'X Eyes', 2)
('Mouth', 'Grin', 1)
('Clothes', 'Vietnam Jacket', 1)
...

Then we group the data into each attribute category so that it's easier to work with:

trait_type_map = {}
for i in attributes:
    if i.name in trait_type_map.keys():
        trait_type_map[i.name][i.value] = i.trait_count
    else:
        trait_type_map[i.name] = {i.value: i.trait_count}

The trait_type_map looks like this:

{'Background': {'Aquamarine': 2,
                'Army Green': 1,
                'Blue': 1,
                'Gray': 1,
                'Orange': 2,
                'Purple': 2,
                'Yellow': 1},
 'Clothes': {'Bayc T Red': 1,
             'Bone Necklace': 1,
             'Navy Striped Tee': 1,
             'Striped Tee': 1,
             'Stunt Jacket': 1,
             'Tweed Suit': 1,
             'Vietnam Jacket': 1,
             'Wool Turtleneck': 1},
...

Now we can easily access each trait type (e.g Background) and trait value (e.g Blue).

We then calculate the rarity using the rarity formula and finally calculate the rank of each NFT by ordering the NFTs according to rarity_score.

With our 10 NFTs you should get the following rank and rarity score in the Django admin:

Conclusion

Congratulations, you now have a working NFT rarity calculator. At this point you can improve the project in the following ways:

Use Celery to fetch the NFT data because if you fetch all 10000 NFTs it's going to timeout if being executed synchronously.
Build a UI around the models - add some views and forms to make interacting with the project more user-friendly.

Sponsored Section

Web3

Build an NFT rarity tool with Django

Matthew Freire