Open-source search API and indexing service for real-time and efficient tagging of objects and buckets in BNB Greenfield.
Greenfield Indexer is an open-source search API and indexing service designed to efficiently organise tags and metadata of objects and buckets on BNB Greenfield. It provides real-time indexing to an off-chain database, allowing easy access and searchability of metadata for storage buckets and objects through a REST and GraphQL API.
This project is a respond to the following community wishlist: Greenfield Tagging and Indexing Service
This challenge is proposed to incorporate a robust tagging indexing system, allowing for efficient data search by tags.
In addition to buckets and objects tags, we also index the following fields as part of the tags:
The Greenfield Indexer API offers several queries and endpoints for interacting with the indexed metadata.
For a full list of available endpoints and detailed usage instructions, please refer to the API documentation at api-documentation.md
Experience Layer
Collection Layer
Data Layer
Greenfield Blockchain
The Greenfield Indexer is designed with modern architectural principles to ensure high availability, scalability, and maintainability. Below are key technical considerations that have been taken into account:
Serverless Architecture: The service is designed for deployment on cloud platforms as a serverless function. This approach eliminates the need for managing server resources, allowing the service to scale automatically in response to request volumes and reducing operational overhead.
Stateless Design: The collection service is engineered to be stateless, enabling it to be invoked multiple times without risking data duplication or inconsistency. This statelessness ensures that each operation is independent, promoting scalability and reliability.
Resilience and Reliability: With the implementation of scheduled triggers, the service possesses built-in resilience. These triggers not only facilitate automatic retries in case of failures but also ensure that data collection can be seamlessly resumed from the last successful synchronization point, minimizing data gaps.
Asynchronous Operations: Designed for asynchronous execution, the collection service allows for a "fire and forget" model of operation. Once initiated, it proceeds with the data ingestion and indexing processes independently, allowing calling processes to continue without waiting for completion.
These considerations underscore our commitment to leveraging contemporary cloud-native technologies and practices to deliver a robust, efficient, and user-friendly service.
We are in the midst of migrating away from depending on the Storage API to only depending on the Blockchain API for indexing the storage buckets and objects. By listening to the MsgCreateObject
and MsgSetTag
transaction events, we can more confidently ensure data integrity and availability in real time.
We welcome contributions to the Greenfield Indexer! If you have suggestions for improvements or encounter any issues, please feel free to submit an issue or pull request on GitHub.