NFT Data storage options

Where to store NFT data when issuing a Bitshares NFT?

If you're interested in issuing your own NFTs on the Bitshares platform then you're going to need to decide where you'll be storing the associated files for long term access.

IPFS

The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data in a distributed file system. IPFS uses content-addressing to uniquely identify each file in a global namespace connecting all computing devices. (source)

Many NFTs issued on Ethereum utilize IPFS (InterPlanetary File System) for storage, this is one of the cheapest long term file host solutions.

You'll need to pin the files in an IPFS node and continuously host it for it to be available long into the future.

If you don't pin the file properly then the file could eventually be lost, to the detriment of future asset owners.

Cost wise you can use a premium pinning service for around $10/month with extra bells & whistles, however there are plenty of free IPFS pinning platforms such as Web3.Storage.

Some pinning services like Web3.storage also backup your content to blockchains like filecoin, so if they go down in the future you'll have a recovery option.

Centralized image hosts

Some NFTs on Ethereum have recently received significant backlash for simply containing hotlinks to centralized image hosts, some of which later broke or 'rug pulled' so criticism of this option is not entirely unwarranted.

That said if you were to host the NFT contents on multiple centralized image hosts, the risk of the hot links later breaking would be reduced.

Any NFTs which store their data in this manner should not be displayed using these hotlinks, but rather be downloaded and cached so as to not tax the image host too significantly.

It's advisable to use an alternative data hosting solution than this so as to improve the quality of your issued Bitshares NFTs.

If you find yourself the owner of such an NFT on the Bitshares blockchain in the future, you should take it upon yourself to store the file contents yourself so as to not lose it.

As of the release of this blog post no such centralized NFT exists on the Bitshares blockchain.

External blockchains

You could store your NFT data on an external blockchain and reference their location within the Bitshares NFT spec; currently the NFTEA Gallery doesn't support retrieving data from external blockchains.

Some examples of blockchains which specialise in decentralized storage:

  • https://sia.tech/
  • https://www.filecoin.com/
  • https://safenetwork.tech/
  • https://www.storj.io/
  • https://oceanprotocol.com/
  • https://www.bittorrent.com/

If you go ahead with integrating one of the above storage solutions into your Bitshares NFT, do consider submitting a pull request to the Github repository so that other nextjs based Bitshares galleries can display your NFT. Similarly, inform others of your NFT data storage decisions in the Bitshares NFT Spec.

Bitshares blockchain

It's 100% possible to permanently store data on the Bitshares blockchain, if you're interested in doing so you need to be aware of how the current Bitshares blockchain configuration will affect such a decision.

First get a life-time membership, this will save you a significant amount of money when taking this route.

If your NFT data exceeds the max transaction/block size limit, then you will either need to shrink the image or store the data on-chain differently.

Smaller than max limit -> base64 encoded asset detail storage

If the NFT contents are small enough then the base64 encoded data can be stored within the NFT Spec during issuance.

Data stored in this manner is fully decentralized, has a low impact on the Bitshares network, and fairly cheap.

Check out "Bitshares NFT SPEC - Media item keys for type NFT/ART" to see where to store the base64 encoded content within the nft_object.

If you are interested in storing a different file type than those referenced in the above Bitshares NFT Spec, then please raise in issue/pull-request in that repo so that other galleries can work on supporting the new NFT content type.

Larger than max limits -> base64 encoded custom operations

You can create custom operations to store data on the Bitshares blockchain, however it is very expensive and has a larger impact on the Bitshares network.

If stored on the blockchain the data is permanent, you don't need to worry about future asset owners being deprived of the original data.

Do the math on how much it's going to cost, lookup the fee for 'Custom - Price per KByte Transaction Size' in the Bitshares fee schedule; for an LTM it's currently 0.09652 BTS per KByte which translates to 98.83648 BTS per MB (currently $2.96) so for example 'NFTEA.CARE' storing its 31 iterations would have been 103MB or $305.40. IPFS was chosen over on-chain storage for this reason.

Example NodeJS code for multi-block data storage of a large image:

var fs  = require('fs')
const { Apis } = require("bitsharesjs-ws");
const {
    TransactionBuilder,
    ChainStore,
    PrivateKey
} = require("bitsharesjs");


function base64_encode(file) {
    var bitmap = fs.readFileSync(file);
    return Buffer.from(bitmap).toString('base64');
}

var bitmap = base64_encode("./IMAGE_FILE.png");
let batchQuantity = Math.round(bitmap.length / 400);

let currentBatch = [];
let batches = [];
for (let i = 0; i < bitmap.length; i++) {
    let currentChar = bitmap[i].charCodeAt(0).toString(2);
    currentBatch.push(currentChar);
    if (currentBatch.length === 400000 || i === bitmap.length) {
      batches.push(
        currentBatch
      );
      currentBatch = [];
    }
}

const wifKey = "";
const pKey = PrivateKey.fromWif(wifKey);

(async () => {
  Apis.instance("wss://eu.nodes.bitshares.ws", true).init_promise.then(
      res => {
          console.log("connected to:", res[0].network_name, "network");

          ChainStore.init().then(() => {
            let transactions = [];
            for (let y = 0; y < batches.length; y++) {
              let tr = new TransactionBuilder();

              tr.add_type_operation(
                "custom",
                {
                  id: 12345, // RANDOM ID
                  payer: "1.2.0", // Change to your account ID
                  data: batches[y]
                }
              );

              tr.set_required_fees().then(() => {
                tr.add_signer(pKey, pKey.toPublicKey().toPublicKeyString());
                /*
                console.log(JSON.stringify(tr.serialize().operations));
                */
                transactions.push(tr);
              });

            }

            let errors = [];
            let verifiedTXS = [];
            for (let z = 0; z < transactions.length; z++) {
              if (errors) {
                return;
              }

              let currentTX = transactions[z];

              let txBroadcast;
              try {
                txBroadcast = await tr.broadcast(currentTX);
              } catch (error) {
                console.error(error);
                errors.push(currentTX);
              }

              if (txBroadcast) {
                verifiedTXS.push(txBroadcast)
              }
            }
          })
          .catch(error => {
            console.log(error)
          })
      }
  )
  .catch(error => {
    console.log(error)
  })

})();

NOTE: There are issues with the above code, notably:

  • You need to store the IDs of the operations you stored the data in, otherwise you'll need to parse your account history to peice together the file.
  • If the API server rate limits you mid script execution then there's no fallback nor tracking of data upload progress.

Use the above as inspiration not for production, perhaps at a later date I'll make this more full featured in its own repo.

NFTEA Gallery

An open source NFT gallery powered by the Bitshares blockchain!
All displayed NFTs are tradeable on the Bitshares decentralized exchange, get collecting!
Created using Next.js and running on Vercel