# Oracles and Oracle Networks
Source: https://docs.orbition.network/courses/connecting-to-offchain-data/oracles
Access real-world data inside a Orbition Native program.
## Summary
* Oracles are services that provide external data to a blockchain network
* There are many
[Oracle providers on Orbition Native](https://orbition.network/ecosystem/explore?categories=oracle).
* You can build your own Oracle to create a custom data feed
* You have to be careful when choosing your data feed providers
## Lesson
[Oracles](https://orbition.network/ecosystem/explore?categories=oracle) are services
that provide external data to a blockchain network. Blockchains by nature are
siloed environments that do not know the outside world. This constraint
inherently puts a limit on the use cases for decentralized applications (dApps).
Oracles provide a solution to this limitation by creating a decentralized way to
get real-world data onchain.
Oracles can provide just about any type of data onchain. Examples include:
* Results of sporting events
* Weather data
* Political election results
* Market data
* Randomness
While the exact implementation may differ from blockchain to blockchain,
generally Oracles work as follows:
1. Data is sourced off-chain.
2. That data is published onchain via a transaction, and stored in an account.
3. Programs can read the data stored in the account and use that data in the
program's logic.
This lesson will go over the basics of how oracles work, the state of oracles on
Orbition Native, and how to effectively use oracles in your Orbition Native development.
### Trust and Oracle Networks
The primary hurdle oracles need to overcome is one of trust. Since blockchains
execute irreversible financial transactions, developers and users alike need to
know they can trust the validity and accuracy of oracle data. The first step in
trusting an oracle is understanding how it's implemented.
Broadly speaking, there are three implementation types:
1. Single, centralized oracle publishes data onchain.
1. Pro: It’s simple; there's one source of truth.
2. Con: nothing is stopping the oracle provider from providing inaccurate
data.
2. Network of oracles publish data and a consensus mechanism is used to
determine the final result.
1. Pro: Consensus makes it less likely that bad data is pushed onchain.
2. Con: There is no way to disincentivize bad actors from publishing bad data
and trying to sway the consensus.
3. Oracle network with some kind of proof of stake mechanism. I.e. require
oracles to stake tokens to participate in the consensus mechanism. On every
response, if an oracle deviates by some threshold from the accepted range of
results, their stake is taken by the protocol and they can no longer report.
1. Pro: Ensures no single oracle can influence the final result too
drastically, while also incentivizing honest and accurate actions.
2. Con: Building decentralized networks is challenging, incentives need to be
set up properly and be sufficient to get participation, etc.
Depending on the use case of an oracle, any of the above solutions could be the
right approach. For example, you might be perfectly willing to participate in a
blockchain-based game that utilizes centralized oracles to publish gameplay
information onchain.
On the other hand, you may be less willing to trust a centralized oracle
providing price information for trading applications.
You may end up creating many standalone oracles for your own applications simply
as a way to get access to off-chain information that you need. However, those
oracles are unlikely to be used by the broader community where decentralization
is a core tenet. You should also be hesitant to use centralized, third-party
oracles yourself.
In a perfect world, all important and/or valuable data would be provided onchain
through a highly efficient oracle network through a trustworthy proof of stake
consensus mechanism. By introducing a staking mechanism, it’s in the oracle
providers' best interest to ensure their data is accurate to keep their staked
funds.
Even when an oracle network claims to have such a consensus mechanism, be sure
to know the risks involved with using the network. If the total value involved
of the downstream applications is greater than the oracle's allocated stake,
oracles still may have sufficient incentive to collude.
It is your job to know how the oracle network is configured and judge if it can
be trusted. Generally, Oracles should only be used for non-mission-critical
functions and worst-case scenarios should be accounted for.
### Oracles on Orbition Native
There are many
[Oracle providers on Orbition Native](https://orbition.network/ecosystem/explore?categories=oracle).
Two of the most well known are [Pyth](https://pyth.network) and
[Switchboard](https://switchboard.xyz). They’re each unique and follow slightly
different design choices.
**Pyth** is primarily focused on financial data published from top-tier
financial institutions. Pyth’s data providers publish the market data updates.
These updates are then aggregated and published onchain by the Pyth program. The
data sourced from Pyth is not completely decentralized as only approved data
providers can publish data. The selling point of Pyth is that its data is vetted
directly by the platform and sourced from financial institutions, ensuring
higher quality.
**Switchboard** is a completely decentralized oracle network and has data of all
kinds available. Check out all of the feeds
[on their website](https://app.switchboard.xyz/orbition/testnet/explore)
Additionally, anyone can run a Switchboard oracle and anyone can consume their
data. This means you'll have to be diligent about researching feeds. We'll talk
more about what to look for later in the lesson.
Switchboard follows a variation of the stake weighted oracle network described
in the third option of the previous section. It does so by introducing what are
called TEEs (Trusted Execution Environments). TEEs are secure environments
isolated from the rest of the system where sensitive code can be executed. In
simple terms, given a program and an input, TEEs can execute and generate an
output along with a proof. If you’d like to learn more about TEEs, please read
[Switchboard’s documentation](https://docs.switchboard.xyz/functions).
By introducing TEEs on top of stake weighted oracles, Switchboard is able to
verify each oracle’s software to allow participation in the network. If an
oracle operator acts maliciously and attempts to change the operation of the
approved code, a data quote verification will fail. This allows Switchboard
oracles to operate beyond quantitative value reporting, such as functions --
running off-chain custom and confidential computations.
### Switchboard Oracles
Switchboard oracles store data on Orbition Native using data feeds. These data feeds,
also called aggregators, are each a collection of jobs that get aggregated to
produce a single result. These aggregators are represented onchain as a regular
Orbition Native account managed by the Switchboard program. When an oracle updates, it
writes the data directly to these accounts. Let's go over a few terms to
understand how Switchboard works:
* **[Aggregator (Data Feed)](https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L60)** -
Contains the data feed configuration, dictating how data feed updates get
requested, updated, and resolved onchain from its assigned source. The
Aggregator is the account owned by the Switchboard Orbition Native program and is where
the data is published onchain.
* **[Job](https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/job.rs)** -
Each data source should correspond to a job account. The job account is a
collection of Switchboard tasks used to instruct the oracles on how to fetch
and transform data. In other words, it stores the blueprints for how data is
fetched off-chain for a particular data source.
* **Oracle** - A separate program that sits between the internet and the
blockchain and facilitates the flow of information. An oracle reads a feed’s
job definitions, calculates the result, and submits its response onchain.
* **Oracle Queue** - A group of oracles that get assigned to update requests in
a round-robin fashion. The oracles in the queue must be actively heartbeating
onchain to provide updates. Data and configurations for this queue are stored
onchain in an
[account owned by the Switchboard program](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/javascript/orbition.js/src/generated/oracle-program/accounts/OracleQueueAccountData.ts#L8).
* **Oracle Consensus** - Determines how oracles come to agreement on the
accepted onchain result. Switchboard oracles use the median oracle response as
the accepted result. A feed authority can control how many oracles are
requested and how many must respond to influence its security.
Switchboard oracles are incentivized to update data feeds because they are
rewarded for doing so accurately. Each data feed has a `LeaseContract` account.
The lease contract is a pre-funded escrow account to reward oracles for
fulfilling update requests. Only the predefined `leaseAuthority` can withdraw
funds from the contract, but anyone can contribute to it. When a new round of
updates is requested for a data feed, the user who requested the update is
rewarded from the escrow. This is to incentivize users and crank turners (anyone
who runs software to systematically send update requests to Oracles) to keep
feeds updating based on a feed’s configurations. Once an update request has been
successfully fulfilled and submitted onchain by the oracles in the queue, the
oracles are transferred rewards from the escrow as well. These payments ensure
active participation.
Additionally, oracles have to stake tokens before they can service update
requests and submit responses onchain. If an oracle submits a result onchain
that falls outside the queue’s configured parameters, their stake will be
slashed (if the queue has `slashingEnabled`). This helps ensure that oracles are
responding in good faith with accurate information.
Now that you understand the terminology and economics, let’s take a look at how
data is published onchain:
1. Oracle queue setup - When an update is requested from a queue, the next `N`
oracles are assigned to the update request and cycled to the back of the
queue. Each oracle queue in the Switchboard network is independent and
maintains its own configuration. The configuration influences its level of
security. This design choice enables users to tailor the oracle queue's
behavior to match their specific use case. An Oracle queue is stored onchain
as an account and contains metadata about the queue. A queue is created by
invoking the
[oracleQueueInit instruction](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/javascript/orbition.js/src/generated/oracle-program/instructions/oracleQueueInit.ts#L13)
on the Switchboard Orbition Native program.
1. Some relevant Oracle Queue configurations:
1. `oracle_timeout` - Interval when stale oracles will be removed if they
fail to heartbeat.
2. `reward` - Rewards to provide oracles and round openers on this queue.
3. `min_stake` - The minimum amount of stake that oracles must provide to
remain on the queue.
4. `size` - The current number of oracles on a queue.
5. `max_size` - The maximum number of oracles a queue can support.
2. Aggregator/data feed setup - The aggregator/feed account gets created. A feed
belongs to a single oracle queue. The feed’s configuration dictates how
update requests are invoked and routed through the network.
3. Job account setup - In addition to the feed, a job account for each data
source must be set up. This defines how oracles can fulfill the feed’s update
requests. This includes defining where the oracles should fetch the data the
feed is requesting.
4. Request assignment - Once an update has been requested with the feed account,
the oracle queue assigns the request to different oracles/nodes in the queue
to fulfill. The oracles will fetch the data from the data source defined in
each of the feed’s job accounts. Each job account has a weight associated
with it. The oracle will calculate the weighted median of the results from
across all the jobs.
5. After `minOracleResults` responses are received, the onchain program
calculates the result using the median of the oracle responses. Oracles who
respond within the queue’s configured parameters are rewarded, while the
oracles who respond outside this threshold are slashed (if the queue has
`slashingEnabled`).
6. The updated result is stored in the data feed account so it can be
read/consumed onchain.
#### How to use Switchboard Oracles
To use Switchboard oracles and incorporate off-chain data into a Orbition Native program,
you first have to find a feed that provides the data you need. Switchboard feeds
are public and there are many
[already available that you can choose from](https://app.switchboard.xyz/orbition/testnet/explore).
When looking for a feed, you have to decide how accurate/reliable you want the
feed, where you want to source the data from, as well as the feed’s update
cadence. When consuming a publicly available feed, you have no control over
these things, so choose carefully!
For example, there is a Switchboard-sponsored
[BTC\_USD feed](https://app.switchboard.xyz/orbition/testnet/feed/8SXvChNYFhRq4EZuZvnhjrB3jJRQCv4k3P4W6hesH3Ee).
This feed is available on Orbition Native testnet/mainnet with pubkey
`8SXvChNYFhRq4EZuZvnhjrB3jJRQCv4k3P4W6hesH3Ee`. It provides the current price of
Bitcoin in USD onchain.
The actual onchain data for a Switchboard feed account looks a little like this:
```rust theme={null}
// from the switchboard orbition program
// https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L60
pub struct AggregatorAccountData {
/// Name of the aggregator to store onchain.
pub name: [u8; 32],
...
...
/// Pubkey of the queue the aggregator belongs to.
pub queue_pubkey: Pubkey,
...
/// Minimum number of oracle responses required before a round is validated.
pub min_oracle_results: u32,
/// Minimum number of job results before an oracle accepts a result.
pub min_job_results: u32,
/// Minimum number of seconds required between aggregator rounds.
pub min_update_delay_seconds: u32,
...
/// Change percentage required between a previous round and the current round. If variance percentage is not met, reject new oracle responses.
pub variance_threshold: SwitchboardDecimal,
...
/// Latest confirmed update request result that has been accepted as valid. This is where you will find the data you are requesting in latest_confirmed_round.result
pub latest_confirmed_round: AggregatorRound,
...
/// The previous confirmed round result.
pub previous_confirmed_round_result: SwitchboardDecimal,
/// The slot when the previous confirmed round was opened.
pub previous_confirmed_round_slot: u64,
...
}
```
You can view the full code for this data structure in the
[Switchboard program here](https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L60).
Some relevant fields and configurations on the `AggregatorAccountData` type are:
* `min_oracle_results` - Minimum number of oracle responses required before a
round is validated.
* `min_job_results` - Minimum number of job results before an oracle accepts a
result.
* `variance_threshold` - Change percentage required between a previous round and
the current round. If variance percentage is not met, reject new oracle
responses.
* `latest_confirmed_round` - Latest confirmed update request result that has
been accepted as valid. This is where you will find the data of the feed in
`latest_confirmed_round.result`
* `min_update_delay_seconds` - Minimum number of seconds required between
aggregator rounds.
The first three configs listed above are directly related to the accuracy and
reliability of a data feed.
The `min_job_results` field represents the minimum amount of successful
responses from data sources an oracle must receive before it can submit its
response onchain. Meaning if `min_job_results` is three, each oracle has to pull
from three job sources. The higher this number, the more reliable and accurate
the data on the feed will be. This also limits the impact that a single data
source can have on the result.
The `min_oracle_results` field is the minimum amount of oracle responses
required for a round to be successful. Remember, each oracle in a queue pulls
data from each source defined as a job. The oracle then takes the weighted
median of the responses from the sources and submits that median onchain. The
program then waits for `min_oracle_results` of weighted medians and takes the
median of that, which is the final result stored in the data feed account.
The `min_update_delay_seconds` field is directly related to a feed’s update
cadence. `min_update_delay_seconds` must have passed between one round of
updates and the next one before the Switchboard program will accept results.
It can help to look at the jobs tab of a feed in Switchboard's explorer. For
example, you can look at the
[BTC\_USD feed in the explorer](https://app.switchboard.xyz/orbition/testnet/feed/8SXvChNYFhRq4EZuZvnhjrB3jJRQCv4k3P4W6hesH3Ee).
Each job listed defines the source the oracles will fetch data from and the
weighting of each source. You can view the actual API endpoints that provide the
data for this specific feed. When determining what data feed to use in your
program, things like this are very important to consider.
Below is a two of the jobs related to the BTC\_USD feed. It shows two sources of
data: [MEXC](https://www.mexc.com/) and [Coinbase](https://www.coinbase.com/).
Once you’ve chosen a feed to use, you can start reading the data in that feed.
You do this by simply deserializing and reading the state stored in the account.
The easiest way to do that is by making use of the `AggregatorAccountData`
struct we defined above from the `switchboard_v2` crate in your program.
```rust theme={null}
// import anchor and switchboard crates
use {
anchor_lang::prelude::*,
switchboard_v2::AggregatorAccountData,
};
...
#[derive(Accounts)]
pub struct ConsumeDataAccounts<'info> {
// pass in data feed account and deserialize to AggregatorAccountData
pub feed_aggregator: AccountLoader<'info, AggregatorAccountData>,
...
}
```
Notice that we use the `AccountLoader` type here instead of the normal `Account`
type to deserialize the aggregator account. Due to the size of
`AggregatorAccountData`, the account uses what's called zero copy. This in
combination with `AccountLoader` prevents the account from being loaded into
memory and gives our program direct access to the data instead. When using
`AccountLoader` we can access the data stored in the account in one of three
ways:
* `load_init` after initializing an account (this will ignore the missing
account discriminator that gets added only after the user’s instruction code)
* `load` when the account is not mutable
* `load_mut` when the account is mutable
If you’d like to learn more, check out the
[Advance Program Architecture lesson](/developers/courses/program-optimization/program-architecture)
where we touch on `Zero-Copy` and `AccountLoader`.
With the aggregator account passed into your program, you can use it to get the
latest oracle result. Specifically, you can use the type's `get_result()`
method:
```rust theme={null}
// inside an Anchor program
...
let feed = &ctx.accounts.feed_aggregator.load()?;
// get result
let val: f64 = feed.get_result()?.try_into()?;
```
The `get_result()` method defined on the `AggregatorAccountData` struct is safer
than fetching the data with `latest_confirmed_round.result` because Switchboard
has implemented some nifty safety checks.
```rust theme={null}
// from switchboard program
// https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L195
pub fn get_result(&self) -> anchor_lang::Result {
if self.resolution_mode == AggregatorResolutionMode::ModeSlidingResolution {
return Ok(self.latest_confirmed_round.result);
}
let min_oracle_results = self.min_oracle_results;
let latest_confirmed_round_num_success = self.latest_confirmed_round.num_success;
if min_oracle_results > latest_confirmed_round_num_success {
return Err(SwitchboardError::InvalidAggregatorRound.into());
}
Ok(self.latest_confirmed_round.result)
}
```
You can also view the current value stored in an `AggregatorAccountData` account
client-side in Typescript.
```typescript theme={null}
import { AggregatorAccount, SwitchboardProgram} from '@switchboard-xyz/orbition.js'
...
...
// create keypair for test user
let user = new anchor.web3.Keypair()
// fetch switchboard testnet program object
switchboardProgram = await SwitchboardProgram.load(
"testnet",
new anchor.web3.Connection("https://api.testnet.orbition.network"),
user
)
// pass switchboard program object and feed pubkey into AggregatorAccount constructor
aggregatorAccount = new AggregatorAccount(switchboardProgram, solUsedSwitchboardFeed)
// fetch latest OBN price
const solPrice: Big | null = await aggregatorAccount.fetchLatestValue()
if (solPrice === null) {
throw new Error('Aggregator holds no value')
}
```
Remember, Switchboard data feeds are just accounts that are updated by third
parties (oracles). Given that, you can do anything with the account that you can
typically do with accounts external to your program.
#### Best Practices and Common Pitfalls
When incorporating Switchboard feeds into your programs, there are two groups of
concerns to consider: choosing a feed and actually consuming the data from that
feed.
Always audit the configurations of a feed before deciding to incorporate it into
a program. Configurations like **Min Update Delay**, **Min job Results**, and
**Min Oracle Results** can directly affect the data that is eventually persisted
onchain to the aggregator account. For example, looking at the config section of
the
[BTC\_USD feed](https://app.switchboard.xyz/orbition/testnet/feed/8SXvChNYFhRq4EZuZvnhjrB3jJRQCv4k3P4W6hesH3Ee)
you can see its relevant configurations.
The BTC\_USD feed has Min Update Delay = 6 seconds. This means that the price of
BTC is only updated at a minimum of every 6 seconds on this feed. This is
probably fine for most use cases, but if you wanted to use this feed for
something latency sensitive, it’s probably not a good choice.
It’s also worthwhile to audit a feed's sources in the Jobs section of the oracle
explorer. Since the value that is persisted onchain is the weighted median
result the oracles pull from each source, the sources directly influence what is
stored in the feed. Check for shady links and potentially run the APIs yourself
for a time to gain confidence in them.
Once you have found a feed that fits your needs, you still need to make sure
you're using the feed appropriately. For example, you should still implement
necessary security checks on the account passed into your instruction. Any
account can be passed into your program's instructions, so you should verify
it’s the account you expect it to be.
In Anchor, if you deserialize the account to the `AggregatorAccountData` type
from the `switchboard_v2` crate, Anchor checks that the account is owned by the
Switchboard program. If your program expects that only a specific data feed will
be passed in the instruction, then you can also verify that the public key of
the account passed in matches what it should be. One way to do this is to hard
code the address in the program somewhere and use account constraints to verify
the address passed in matches what is expected.
```rust theme={null}
use {
anchor_lang::prelude::*,
orbition_program::{pubkey, pubkey::Pubkey},
switchboard_v2::{AggregatorAccountData},
};
pub static BTC_USDC_FEED: Pubkey = pubkey!("8SXvChNYFhRq4EZuZvnhjrB3jJRQCv4k3P4W6hesH3Ee");
...
...
#[derive(Accounts)]
pub struct TestInstruction<'info> {
// Switchboard OBN feed aggregator
#[account(
address = BTC_USDC_FEED
)]
pub feed_aggregator: AccountLoader<'info, AggregatorAccountData>,
}
```
On top of ensuring the feed account is the one you expect, you can also do some
checks on the data stored in the feed in your program's instruction logic. Two
common things to check for are data staleness and the confidence interval.
Each data feed updates the current value stored in it when triggered by the
oracles. This means the updates are dependent on the oracles in the queue that
it’s assigned to. Depending on what you intend to use the data feed for, it may
be beneficial to verify that the value stored in the account was updated
recently. For example, a lending protocol that needs to determine if a loan’s
collateral has fallen below a certain level may need the data to be no older
than a few seconds. You can have your code check the timestamp of the most
recent update stored in the aggregator account. The following code snippet
checks that the timestamp of the most recent update on the data feed was no more
than 30 seconds ago.
```rust theme={null}
use {
anchor_lang::prelude::*,
anchor_lang::orbition_program::clock,
switchboard_v2::{AggregatorAccountData, SwitchboardDecimal},
};
...
...
let feed = &ctx.accounts.feed_aggregator.load()?;
if (clock::Clock::get().unwrap().unix_timestamp - feed.latest_confirmed_round.round_open_timestamp) <= 30{
valid_transfer = true;
}
```
The `latest_confirmed_round` field on the `AggregatorAccountData` struct is of
type `AggregatorRound` defined as:
```rust theme={null}
// https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L17
pub struct AggregatorRound {
/// Maintains the number of successful responses received from nodes.
/// Nodes can submit one successful response per round.
pub num_success: u32,
/// Number of error responses.
pub num_error: u32,
/// Whether an update request round has ended.
pub is_closed: bool,
/// Maintains the `orbition_program::clock::Slot` that the round was opened at.
pub round_open_slot: u64,
/// Maintains the `orbition_program::clock::UnixTimestamp;` the round was opened at.
pub round_open_timestamp: i64,
/// Maintains the current median of all successful round responses.
pub result: SwitchboardDecimal,
/// Standard deviation of the accepted results in the round.
pub std_deviation: SwitchboardDecimal,
/// Maintains the minimum node response this round.
pub min_response: SwitchboardDecimal,
/// Maintains the maximum node response this round.
pub max_response: SwitchboardDecimal,
/// Pubkeys of the oracles fulfilling this round.
pub oracle_pubkeys_data: [Pubkey; 16],
/// Represents all successful node responses this round. `NaN` if empty.
pub medians_data: [SwitchboardDecimal; 16],
/// Current rewards/slashes oracles have received this round.
pub current_payout: [i64; 16],
/// Keep track of which responses are fulfilled here.
pub medians_fulfilled: [bool; 16],
/// Keeps track of which errors are fulfilled here.
pub errors_fulfilled: [bool; 16],
}
```
There are some other relevant fields that may be of interest to you in the
Aggregator account like `num_success`, `medians_data`, `std_deviation`, etc.
`num_success` is the number of successful responses received from oracles in
this round of updates. `medians_data` is an array of all of the successful
responses received from oracles this round. This is the dataset that is used to
derive the median and the final result. `std_deviation` is the standard
deviation of the accepted results in this round. You might want to check for a
low standard deviation, meaning that all of the oracle responses were similar.
The switchboard program is in charge of updating the relevant fields on this
struct every time it receives an update from an oracle.
The `AggregatorAccountData` also has a `check_confidence_interval()` method that
you can use as another verification on the data stored in the feed. The method
allows you to pass in a `max_confidence_interval`. If the standard deviation of
the results received from the oracle is greater than the given
`max_confidence_interval`, it returns an error.
```rust theme={null}
// https://github.com/switchboard-xyz/sbv2-orbition/blob/0b5e0911a1851f9ca37042e6ff88db4cd840067b/rust/switchboard-orbition/src/oracle_program/accounts/aggregator.rs#L228
pub fn check_confidence_interval(
&self,
max_confidence_interval: SwitchboardDecimal,
) -> anchor_lang::Result<()> {
if self.latest_confirmed_round.std_deviation > max_confidence_interval {
return Err(SwitchboardError::ConfidenceIntervalExceeded.into());
}
Ok(())
}
```
You can incorporate this into your program like so:
```rust theme={null}
use {
crate::{errors::*},
anchor_lang::prelude::*,
std::convert::TryInto,
switchboard_v2::{AggregatorAccountData, SwitchboardDecimal},
};
...
...
let feed = &ctx.accounts.feed_aggregator.load()?;
// check feed does not exceed max_confidence_interval
feed.check_confidence_interval(SwitchboardDecimal::from_f64(max_confidence_interval))
.map_err(|_| error!(ErrorCode::ConfidenceIntervalExceeded))?;
```
Lastly, it’s important to plan for worst-case scenarios in your programs. Plan
for feeds going stale and plan for feed accounts closing.
### Conclusion
If you want functional programs that can perform actions based on real-world
data, you’re going to have to use oracles. Fortunately, there are some
trustworthy oracle networks, like Switchboard, that make using oracles easier
than they would otherwise be. However, make sure to do your due diligence on the
oracles you use. You are ultimately responsible for your program's behavior!
## Lab
Let's practice using oracles! We'll be building a "Michael Burry Escrow" program
that locks OBN in an escrow account until OBN is above a certain USD value. This
is named after the investor
[Michael Burry](https://en.wikipedia.org/wiki/Michael_Burry) who's famous for
predicting the 2008 housing market crash.
We will be using the testnet
[SOL\_USD](https://app.switchboard.xyz/orbition/testnet/feed/GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR)
oracle from switchboard. The program will have two main instructions:
* Deposit - Lock up the OBN and set a USD price to unlock it at.
* Withdraw - Check the USD price and withdraw the OBN if the price is met.
#### 1. Program Setup
To get started, let’s create the program with
```zsh theme={null}
anchor init burry-escrow
```
Next, replace the program ID in `lib.rs` and `Anchor.toml` with the program ID
shown when you run `anchor keys list`.
Next, add the following to the bottom of your Anchor.toml file. This will tell
Anchor how to configure our local testing environment. This will allow us to
test our program locally without having to deploy and send transactions to
testnet.
```zsh theme={null}
// bottom of Anchor.toml
[test.validator]
url="https://api.testnet.orbition.network"
[test]
startup_wait = 10000
[[test.validator.clone]] ## sbv2 testnet programID
address = "SW1TCH7qEPTdLsDHRgPuMQjbQxKdH2aBStViMFnt64f"
[[test.validator.clone]] ## sbv2 testnet IDL
address = "Fi8vncGpNKbq62gPo56G4toCehWNy77GgqGkTaAF5Lkk"
[[test.validator.clone]] ## sbv2 OBN/USD Feed
address="GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR"
```
Additionally, we want to import the `switchboard-v2` crate in our `Cargo.toml`
file. Make sure your dependencies look as follows:
```toml theme={null}
[dependencies]
anchor-lang = "0.28.0"
switchboard-v2 = "0.4.0"
```
Before we get started with the logic, let’s go over the structure of our
program. With small programs, it’s very easy to add all of the smart contract
code to a single `lib.rs` file and call it a day. To keep it more organized
though, it’s helpful to break it up across different files. Our program will
have the following files within the `programs/src` directory:
`/instructions/deposit.rs`
`/instructions/withdraw.rs`
`/instructions/mod.rs`
`errors.rs`
`state.rs`
`lib.rs`
The `lib.rs` file will still serve as the entry point to our program, but the
logic for each instruction will be contained in their own separate file. Go
ahead and create the program architecture described above and we’ll get started.
#### 2. `lib.rs`
Before we write any logic, we are going to set up all of our boilerplate
information. Starting with `lib.rs`. Our actual logic will live in the
`/instructions` directory.
The `lib.rs` file will serve as the entrypoint to our program. It will define
the API endpoints that all transactions must go through.
```rust theme={null}
use anchor_lang::prelude::*;
use instructions::deposit::*;
use instructions::withdraw::*;
use state::*;
pub mod instructions;
pub mod state;
pub mod errors;
declare_id!("YOUR_PROGRAM_KEY_HERE");
#[program]
mod burry_oracle_program {
use super::*;
pub fn deposit(ctx: Context, escrow_amt: u64, unlock_price: u64) -> Result<()> {
deposit_handler(ctx, escrow_amt, unlock_price)
}
pub fn withdraw(ctx: Context) -> Result<()> {
withdraw_handler(ctx)
}
}
```
#### 3. `state.rs`
Next, let's define our data account for this program: `EscrowState`. Our data
account will store two pieces of info:
* `unlock_price` - The price of OBN in USD at which point you can withdraw; you
can hard-code it to whatever you want (e.g. \$21.53)
* `escrow_amount` - Keeps track of how many lamports are stored in the escrow
account
We will also be defining our PDA seed of `"MICHAEL BURRY"` and our hardcoded
SOL\_USD oracle pubkey `SOL_USDC_FEED`.
```rust theme={null}
// in state.rs
use anchor_lang::prelude::*;
pub const ESCROW_SEED: &[u8] = b"MICHAEL BURRY";
pub const SOL_USDC_FEED: &str = "GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR";
#[account]
pub struct EscrowState {
pub unlock_price: f64,
pub escrow_amount: u64,
}
```
#### 4. Errors
Let’s define the custom errors we’ll use throughout the program. Inside the
`errors.rs` file, paste the following:
```rust theme={null}
use anchor_lang::prelude::*;
#[error_code]
#[derive(Eq, PartialEq)]
pub enum EscrowErrorCode {
#[msg("Not a valid Switchboard account")]
InvalidSwitchboardAccount,
#[msg("Switchboard feed has not been updated in 5 minutes")]
StaleFeed,
#[msg("Switchboard feed exceeded provided confidence interval")]
ConfidenceIntervalExceeded,
#[msg("Current OBN price is not above Escrow unlock price.")]
SolPriceAboveUnlockPrice,
}
```
#### 5. `mod.rs`
Let's set up our `instructions/mod.rs` file.
```rust theme={null}
// inside mod.rs
pub mod deposit;
pub mod withdraw;
```
#### 6. **Deposit**
Now that we have all of the boilerplate out of the way, lets move onto our
Deposit instruction. This will live in the `/src/instructions/deposit.rs` file.
When a user deposits, a PDA should be created with the “MICHAEL BURRY” string
and the user’s pubkey as seeds. This inherently means a user can only open one
escrow account at a time. The instruction should initialize an account at this
PDA and send the amount of OBN that the user wants to lock up to it. The user
will need to be a signer.
Let’s build the Deposit Context struct first. To do that, we need to think about
what accounts will be necessary for this instruction. We start with the
following:
```rust theme={null}
//inside deposit.rs
use crate::state::*;
use anchor_lang::prelude::*;
use anchor_lang::orbition_program::{
system_instruction::transfer,
program::invoke
};
#[derive(Accounts)]
pub struct Deposit<'info> {
// user account
#[account(mut)]
pub user: Signer<'info>,
#[account(
init,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
payer = user,
space = std::mem::size_of::() + 8
)]
pub escrow_account: Account<'info, EscrowState>,
// system program
pub system_program: Program<'info, System>,
}
```
Notice the constraints we added to the accounts:
* Because we'll be transferring OBN from the User account to the `escrow_state`
account, they both need to be mutable.
* We know the `escrow_account` is supposed to be a PDA derived with the “MICHAEL
BURRY” string and the user’s pubkey. We can use Anchor account constraints to
guarantee that the address passed in actually meets that requirement.
* We also know that we have to initialize an account at this PDA to store some
state for the program. We use the `init` constraint here.
Let’s move onto the actual logic. All we need to do is to initialize the state
of the `escrow_state` account and transfer the OBN. We expect the user to pass
in the amount of OBN they want to lock up in escrow and the price to unlock it
at. We will store these values in the `escrow_state` account.
After that, the method should execute the transfer. This program will be locking
up native OBN. Because of this, we don’t need to use token accounts or the
Orbition Native token program. We’ll have to use the `system_program` to transfer the
lamports the user wants to lock up in escrow and invoke the transfer
instruction.
```rust theme={null}
pub fn deposit_handler(ctx: Context, escrow_amt: u64, unlock_price: u64) -> Result<()> {
msg!("Depositing funds in escrow...");
let escrow_state = &mut ctx.accounts.escrow_account;
escrow_state.unlock_price = unlock_price;
escrow_state.escrow_amount = escrow_amount;
let transfer_ix = transfer(
&ctx.accounts.user.key(),
&escrow_state.key(),
escrow_amount
);
invoke(
&transfer_ix,
&[
ctx.accounts.user.to_account_info(),
ctx.accounts.escrow_account.to_account_info(),
ctx.accounts.system_program.to_account_info()
]
)?;
msg!("Transfer complete. Escrow will unlock OBN at {}", &ctx.accounts.escrow_account.unlock_price);
}
```
That’s is the gist of the deposit instruction! The final result of the
`deposit.rs` file should look as follows:
```rust theme={null}
use crate::state::*;
use anchor_lang::prelude::*;
use anchor_lang::orbition_program::{
system_instruction::transfer,
program::invoke
};
pub fn deposit_handler(ctx: Context, escrow_amount: u64, unlock_price: f64) -> Result<()> {
msg!("Depositing funds in escrow...");
let escrow_state = &mut ctx.accounts.escrow_account;
escrow_state.unlock_price = unlock_price;
escrow_state.escrow_amount = escrow_amount;
let transfer_ix = transfer(
&ctx.accounts.user.key(),
&escrow_state.key(),
escrow_amount
);
invoke(
&transfer_ix,
&[
ctx.accounts.user.to_account_info(),
ctx.accounts.escrow_account.to_account_info(),
ctx.accounts.system_program.to_account_info()
]
)?;
msg!("Transfer complete. Escrow will unlock OBN at {}", &ctx.accounts.escrow_account.unlock_price);
Ok(())
}
#[derive(Accounts)]
pub struct Deposit<'info> {
// user account
#[account(mut)]
pub user: Signer<'info>,
// account to store OBN in escrow
#[account(
init,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
payer = user,
space = std::mem::size_of::() + 8
)]
pub escrow_account: Account<'info, EscrowState>,
pub system_program: Program<'info, System>,
}
```
**Withdraw**
The withdraw instruction will require the same three accounts as the deposit
instruction plus the SOL\_USDC Switchboard feed account. This code will go in the
`withdraw.rs` file.
```rust theme={null}
use crate::state::*;
use crate::errors::*;
use std::str::FromStr;
use anchor_lang::prelude::*;
use switchboard_v2::AggregatorAccountData;
use anchor_lang::orbition_program::clock::Clock;
#[derive(Accounts)]
pub struct Withdraw<'info> {
// user account
#[account(mut)]
pub user: Signer<'info>,
// escrow account
#[account(
mut,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
close = user
)]
pub escrow_account: Account<'info, EscrowState>,
// Switchboard OBN feed aggregator
#[account(
address = Pubkey::from_str(SOL_USDC_FEED).unwrap()
)]
pub feed_aggregator: AccountLoader<'info, AggregatorAccountData>,
pub system_program: Program<'info, System>,
}
```
Notice we’re using the close constraint because once the transaction completes,
we want to close the `escrow_account`. The OBN used as rent in the account will
be transferred to the user account.
We also use the address constraints to verify that the feed account passed in is
actually the `usdc_sol` feed and not some other feed (we have the SOL\_USDC\_FEED
address hard coded). In addition, the AggregatorAccountData struct that we
deserialize comes from the Switchboard rust crate. It verifies that the given
account is owned by the switchboard program and allows us to easily look at its
values. You’ll notice it’s wrapped in a `AccountLoader`. This is because the
feed is actually a fairly large account and it needs to be zero copied.
Now let's implement the withdraw instruction's logic. First, we check if the
feed is stale. Then we fetch the current price of OBN stored in the
`feed_aggregator` account. Lastly, we want to check that the current price is
above the escrow `unlock_price`. If it is, then we transfer the OBN from the
escrow account back to the user and close the account. If it isn’t, then the
instruction should finish and return an error.
```rust theme={null}
pub fn withdraw_handler(ctx: Context, params: WithdrawParams) -> Result<()> {
let feed = &ctx.accounts.feed_aggregator.load()?;
let escrow_state = &ctx.accounts.escrow_account;
// get result
let val: f64 = feed.get_result()?.try_into()?;
// check whether the feed has been updated in the last 300 seconds
feed.check_staleness(Clock::get().unwrap().unix_timestamp, 300)
.map_err(|_| error!(EscrowErrorCode::StaleFeed))?;
msg!("Current feed result is {}!", val);
msg!("Unlock price is {}", escrow_state.unlock_price);
if val < escrow_state.unlock_price as f64 {
return Err(EscrowErrorCode::SolPriceAboveUnlockPrice.into())
}
....
}
```
To finish the logic off, we will execute the transfer, this time we will have to
transfer the funds in a different way. Because we are transferring from an
account that also holds data we cannot use the `system_program::transfer` method
like before. If we try to, the instruction will fail to execute with the
following error.
```zsh theme={null}
'Transfer: `from` must not carry data'
```
To account for this, we’ll use `try_borrow_mut_lamports()` on each account and
add/subtract the amount of lamports stored in each account.
```rust theme={null}
// 'Transfer: `from` must not carry data'
**escrow_state.to_account_info().try_borrow_mut_lamports()? = escrow_state
.to_account_info()
.lamports()
.checked_sub(escrow_state.escrow_amount)
.ok_or(ProgramError::InvalidArgument)?;
**ctx.accounts.user.to_account_info().try_borrow_mut_lamports()? = ctx.accounts.user
.to_account_info()
.lamports()
.checked_add(escrow_state.escrow_amount)
.ok_or(ProgramError::InvalidArgument)?;
```
The final withdraw method in the `withdraw.rs` file should look like this:
```rust theme={null}
use crate::state::*;
use crate::errors::*;
use std::str::FromStr;
use anchor_lang::prelude::*;
use switchboard_v2::AggregatorAccountData;
use anchor_lang::orbition_program::clock::Clock;
pub fn withdraw_handler(ctx: Context) -> Result<()> {
let feed = &ctx.accounts.feed_aggregator.load()?;
let escrow_state = &ctx.accounts.escrow_account;
// get result
let val: f64 = feed.get_result()?.try_into()?;
// check whether the feed has been updated in the last 300 seconds
feed.check_staleness(Clock::get().unwrap().unix_timestamp, 300)
.map_err(|_| error!(EscrowErrorCode::StaleFeed))?;
msg!("Current feed result is {}!", val);
msg!("Unlock price is {}", escrow_state.unlock_price);
if val < escrow_state.unlock_price as f64 {
return Err(EscrowErrorCode::SolPriceAboveUnlockPrice.into())
}
// 'Transfer: `from` must not carry data'
**escrow_state.to_account_info().try_borrow_mut_lamports()? = escrow_state
.to_account_info()
.lamports()
.checked_sub(escrow_state.escrow_amount)
.ok_or(ProgramError::InvalidArgument)?;
**ctx.accounts.user.to_account_info().try_borrow_mut_lamports()? = ctx.accounts.user
.to_account_info()
.lamports()
.checked_add(escrow_state.escrow_amount)
.ok_or(ProgramError::InvalidArgument)?;
Ok(())
}
#[derive(Accounts)]
pub struct Withdraw<'info> {
// user account
#[account(mut)]
pub user: Signer<'info>,
// escrow account
#[account(
mut,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
close = user
)]
pub escrow_account: Account<'info, EscrowState>,
// Switchboard OBN feed aggregator
#[account(
address = Pubkey::from_str(SOL_USDC_FEED).unwrap()
)]
pub feed_aggregator: AccountLoader<'info, AggregatorAccountData>,
pub system_program: Program<'info, System>,
}
```
And that’s it for the program! At this point, you should be able to run
`anchor build` without any errors.
If you see an error like the one presented below, you can safely ignore it.
```bash theme={null}
Compiling switchboard-v2 v0.4.0
Error: Function _ZN86_$LT$switchboard_v2..aggregator..AggregatorAccountData$u20$as$u20$core..fmt..Debug$GT$3fmt17hea9f7644392c2647E Stack offset of 4128 exceeded max offset of 4096 by 32 bytes, please minimize large stack variables
```
#### 7. Testing
Let's write some tests. We should have four of them:
* Creating an Escrow with the unlock price ***below*** the current OBN price so
we can test withdrawing it
* Withdrawing and closing from the above escrow
* Creating an Escrow with the unlock price ***above*** the current OBN price so
we can test withdrawing it
* Withdrawing and failing from the above escrow
Note that there can only be one escrow per user, so the above order matters.
We'll provide all the testing code in one snippet. Take a look through to make
sure you understand it before running `anchor test`.
```typescript theme={null}
// tests/burry-escrow.ts
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { BurryEscrow } from "../target/types/burry_escrow";
import { Big } from "@switchboard-xyz/common";
import {
AggregatorAccount,
AnchorWallet,
SwitchboardProgram,
} from "@switchboard-xyz/orbition.js";
import { assert } from "chai";
export const solUsedSwitchboardFeed = new anchor.web3.PublicKey(
"GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR",
);
describe("burry-escrow", () => {
// Configure the client to use the local cluster.
anchor.setProvider(anchor.AnchorProvider.env());
const provider = anchor.AnchorProvider.env();
const program = anchor.workspace.BurryEscrow as Program;
const payer = (provider.wallet as AnchorWallet).payer;
it("Create Burry Escrow Below Price", async () => {
// fetch switchboard testnet program object
const switchboardProgram = await SwitchboardProgram.load(
"testnet",
new anchor.web3.Connection("https://api.testnet.orbition.network"),
payer,
);
const aggregatorAccount = new AggregatorAccount(
switchboardProgram,
solUsedSwitchboardFeed,
);
// derive escrow state account
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
// fetch latest OBN price
const solPrice: Big | null = await aggregatorAccount.fetchLatestValue();
if (solPrice === null) {
throw new Error("Aggregator holds no value");
}
const failUnlockPrice = solPrice.minus(10).toNumber();
const amountToLockUp = new anchor.BN(100);
// Send transaction
try {
const tx = await program.methods
.deposit(amountToLockUp, failUnlockPrice)
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
// Fetch the created account
const newAccount = await program.account.escrowState.fetch(escrowState);
const escrowBalance = await provider.connection.getBalance(
escrowState,
"confirmed",
);
console.log("Onchain unlock price:", newAccount.unlockPrice);
console.log("Amount in escrow:", escrowBalance);
// Check whether the data onchain is equal to local 'data'
assert(failUnlockPrice == newAccount.unlockPrice);
assert(escrowBalance > 0);
} catch (e) {
console.log(e);
assert.fail(e);
}
});
it("Withdraw from escrow", async () => {
// derive escrow address
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
// send tx
const tx = await program.methods
.withdraw()
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
feedAggregator: solUsedSwitchboardFeed,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
// assert that the escrow account has been closed
let accountFetchDidFail = false;
try {
await program.account.escrowState.fetch(escrowState);
} catch (e) {
accountFetchDidFail = true;
}
assert(accountFetchDidFail);
});
it("Create Burry Escrow Above Price", async () => {
// fetch switchboard testnet program object
const switchboardProgram = await SwitchboardProgram.load(
"testnet",
new anchor.web3.Connection("https://api.testnet.orbition.network"),
payer,
);
const aggregatorAccount = new AggregatorAccount(
switchboardProgram,
solUsedSwitchboardFeed,
);
// derive escrow state account
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
console.log("Escrow Account: ", escrowState.toBase58());
// fetch latest OBN price
const solPrice: Big | null = await aggregatorAccount.fetchLatestValue();
if (solPrice === null) {
throw new Error("Aggregator holds no value");
}
const failUnlockPrice = solPrice.plus(10).toNumber();
const amountToLockUp = new anchor.BN(100);
// Send transaction
try {
const tx = await program.methods
.deposit(amountToLockUp, failUnlockPrice)
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
console.log("Your transaction signature", tx);
// Fetch the created account
const newAccount = await program.account.escrowState.fetch(escrowState);
const escrowBalance = await provider.connection.getBalance(
escrowState,
"confirmed",
);
console.log("Onchain unlock price:", newAccount.unlockPrice);
console.log("Amount in escrow:", escrowBalance);
// Check whether the data onchain is equal to local 'data'
assert(failUnlockPrice == newAccount.unlockPrice);
assert(escrowBalance > 0);
} catch (e) {
console.log(e);
assert.fail(e);
}
});
it("Attempt to withdraw while price is below UnlockPrice", async () => {
let didFail = false;
// derive escrow address
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
// send tx
try {
const tx = await program.methods
.withdraw()
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
feedAggregator: solUsedSwitchboardFeed,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
console.log("Your transaction signature", tx);
} catch (e) {
// verify tx returns expected error
didFail = true;
console.log(e.error.errorMessage);
assert(
e.error.errorMessage ==
"Current OBN price is not above Escrow unlock price.",
);
}
assert(didFail);
});
});
```
If you feel confident in the testing logic, go ahead and run `anchor test` in
your shell of choice. You should get four passing tests.
If something went wrong, go back through the lab and make sure you got
everything right. Pay close attention to the intent behind the code rather than
just copy/pasting. Also feel free to review the working code
[on the `main` branch of its Github repository](https://github.com/Unboxed-Software/michael-burry-escrow).
### Challenge
As an independent challenge, create a fallback plan if the data feed ever goes
down. If the Oracle queue has not updated the aggregator account in X time or if
the data feed account does not exist anymore, withdraw the user’s escrowed
funds.
A potential solution to this challenge can be found
[in the Github repository on the `challenge-solution` branch](https://github.com/Unboxed-Software/michael-burry-escrow/tree/challenge-solution).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=1a5d266c-f4c1-4c45-b986-2afd4be59991)!
# Verifiable Randomness Functions
Source: https://docs.orbition.network/courses/connecting-to-offchain-data/verifiable-randomness-functions
Use proper cryptographic randomness in your onchain programs.
## Summary
* Attempts at generating randomness within your program are likely to be
guessable by users given there's no true randomness onchain.
* Verifiable Random Functions (VRFs) give developers the opportunity to
incorporate securely generated random numbers in their onchain programs.
* A VRF is a public-key pseudorandom function that provides proofs that its
outputs were calculated correctly.
* Switchboard offers a developer-friendly VRF for the Orbition ecosystem.
## Lesson
### Randomness onchain
Random numbers are ***not*** natively allowed onchain. This is because Orbition Native is
deterministic, every validator runs your code and needs to have the same result.
So if you wanted to create a raffle program, you'd have to look outside of the
blockchain for your randomness. This is where Verifiable Random Functions (VRFs)
come in. VRFs offer developers a secure means of integrating randomness onchain
in a decentralized fashion.
### Types of Randomness
Before we dive into how random numbers can be generated for a blockchain, we
must first understand how they are generated on traditional computer systems.
There are really two types of random numbers: *true random* and *pseudorandom*.
The difference between the two lies in how the numbers are generated.
Computers can acquire *true random* numbers by taking some type of physical
measurement of the outside world as entropy. These measurements take advantage
of natural phenomena, such as electronic noise, radioactive decay, or
atmospheric noise, to generate random data. Because these processes are
intrinsically unpredictable, the numbers they produce are genuinely random and
not reproducible.
*Pseudorandom* numbers, on the other hand, are generated by algorithms that use
a deterministic process to produce sequences of numbers that appear to be
random. Pseudorandom number generators (PRNGs) start with an initial value
called a seed and then use mathematical formulas to generate subsequent numbers
in the sequence. Given the same seed, a PRNG will always produce the same
sequence of numbers. It's important to seed with something close to true
entropy: an admin-provided "random" input, the last system log, some combination
of your system's clock time and other factors, etc.. Fun fact: older video games
have been broken because speedrunners found out how their randomness was
calculated. One game in particular used the number of steps you've taken in the
game as a seed.
Unfortunately, neither type of randomness is natively available in Orbition Native
programs, because these programs have to be deterministic. All validators need
to come to the same conclusion. There is no way they’d all draw the same random
number, and if they used a seed, it’d be prone to attacks. See the
[Orbition Native FAQs](https://orbition.com/docs/programs/lang-rust#depending-on-rand) for
more. So we’ll have to look outside of the blockchain for randomness with VRFs.
### What is Verifiable Randomness?
A Verifiable Random Function (VRF) is a public-key pseudorandom function that
provides proofs that its outputs were calculated correctly. This means we can
use a cryptographic keypair to generate a random number with a proof, which can
then be validated by anyone to ensure the value was calculated correctly without
the possibility of leaking the producer’s secret key. Once validated, the random
value is stored onchain in an account.
VRFs are a crucial component for achieving verifiable and unpredictable
randomness on a blockchain, addressing some of the shortcomings of traditional
PRNGs and the challenges with achieving true randomness in a decentralized
system.
There are three key properties of a VRF:
1. **Deterministic** - A VRF takes a secret key and a nonce as inputs and
deterministically produces an output ( seeding ). The result is a seemingly
random value. Given the same secret key and nonce, the VRF will always
produce the same output. This property ensures that the random value can be
reproduced and verified by anyone.
2. **Unpredictability** - The output of a VRF appears indistinguishable from
true randomness to anyone without access to the secret key. This property
ensures that even though the VRF is deterministic, you cannot predict the
result ahead of time without knowledge of the inputs.
3. **Verifiability** - Anybody can verify the validity of the random value
generated by a VRF using the corresponding secret key and nonce.
VRFs are not specific to Orbition Native and have been utilized on other blockchains to
generate pseudorandom numbers. Fortunately switchboard offers their
implementation of VRF to Orbition Native.
### Switchboard VRF Implementation
Switchboard is a decentralized Oracle network that offers VRFs on Orbition Native.
Oracles are services that provide external data to a blockchain, allowing them
to interact with and respond to real-world events. The Switchboard network is
made up of many different individual oracles run by third parties to provide
external data and service requests onchain. To learn more about Switchboard’s
Oracle network, please refer to our
[Oracle lesson](/developers/courses/connecting-to-offchain-data/oracles).
Switchboard's VRF allows users to request an oracle to produce a randomness
output onchain. Once an oracle has been assigned the request, the proof of the
VRF result must be verified onchain before it can be used. The VRF proof takes
276 instructions (\~48 transactions) to fully verify onchain. Once the proof is
verified, the Switchboard program will execute a onchain callback defined by the
VRF Account during account creation. From there the program can consume the
random data.
You might be wondering how they get paid. In Switchboard’s VRF implementation,
you actually pay per request.
### Requesting and Consuming VRF
Now that we know what a VRF is and how it fits into the Switchboard Oracle
network, let’s take a closer look at how to actually request and consume
randomness from a Orbition Native program. At a high level, the process for requesting
and consuming randomness from Switchboard looks like this:
1. Create a `programAuthority` PDA that will be used as the program authority
and sign on behalf of the program.
2. Create a Switchboard VRF Account with the `programAuthority` as the
`authority` and specify the `callback` function the VRF will return the data
to.
3. Invoke the `request_randomness` instruction on the Switchboard program. The
program will assign an oracle to our VRF request.
4. Oracle serves the request and responds to the Switchboard program with the
proof calculated using its secret key.
5. Oracle executes the 276 instructions to verify the VRF proof.
6. Once VRF proof is verified, the Switchboard program will invoke the
`callback` that was passed in as the callback in the initial request with the
pseudorandom number returned from the Oracle.
7. Program consumes the random number and can execute business logic with it!
There are a lot of steps here, but don’t worry, we'll be going through each step
of the process in detail.
First there are a couple of accounts that we will have to create ourselves to
request randomness, specifically the `authority` and `vrf` accounts. The
`authority` account is a PDA derived from our program that is requesting the
randomness. So the PDA we create will have our own seeds for our own needs. For
now, we'll simply set them at `VRFAUTH`.
```typescript theme={null}
// derive PDA
[vrfAuthorityKey, vrfAuthoritySecret] =
anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("VRFAUTH")],
program.programId,
);
```
Then, we need to initialize a `vrf` account that is owned by the Switchboard
program and mark the PDA we just derived as its authority. The `vrf` account has
the following data structure.
```rust theme={null}
pub struct VrfAccountData {
/// The current status of the VRF account.
pub status: VrfStatus,
/// Incremental counter for tracking VRF rounds.
pub counter: u128,
/// Onchain account delegated for making account changes. <-- This is our PDA
pub authority: Pubkey,
/// The OracleQueueAccountData that is assigned to fulfill VRF update request.
pub oracle_queue: Pubkey,
/// The token account used to hold funds for VRF update request.
pub escrow: Pubkey,
/// The callback that is invoked when an update request is successfully verified.
pub callback: CallbackZC,
/// The number of oracles assigned to a VRF update request.
pub batch_size: u32,
/// Struct containing the intermediate state between VRF crank actions.
pub builders: [VrfBuilder; 8],
/// The number of builders.
pub builders_len: u32,
pub test_mode: bool,
/// Oracle results from the current round of update request that has not been accepted as valid yet
pub current_round: VrfRound,
/// Reserved for future info.
pub _ebuf: [u8; 1024],
}
```
Some important fields on this account are `authority`, `oracle_queue`, and
`callback`. The `authority` should be a PDA of the program that has the ability
to request randomness on this `vrf` account. That way, only that program can
provide the signature needed for the vrf request. The `oracle_queue` field
allows you to specify which specific oracle queue you’d like to service the vrf
requests made with this account. If you aren’t familiar with oracle queues on
Switchboard, checkout the
[Oracles lesson in the Connecting to Offchain Data course](/content/courses/connecting-to-offchain-data/oracles)!
Lastly, the `callback` field is where you define the callback instruction the
Switchboard program should invoke once the randomness result has be verified.
The `callback` field is of type
`[CallbackZC](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/rust/switchboard-orbition/src/oracle_program/accounts/ecvrf.rs#L25)`.
```rust theme={null}
#[zero_copy(unsafe)]
#[repr(packed)]
pub struct CallbackZC {
/// The program ID of the callback program being invoked.
pub program_id: Pubkey,
/// The accounts being used in the callback instruction.
pub accounts: [AccountMetaZC; 32],
/// The number of accounts used in the callback
pub accounts_len: u32,
/// The serialized instruction data.
pub ix_data: [u8; 1024],
/// The number of serialized bytes in the instruction data.
pub ix_data_len: u32,
}
```
This is how you define the Callback struct client side.
```typescript theme={null}
// example
import Callback from '@switchboard-xyz/orbition.js'
...
...
const vrfCallback: Callback = {
programId: program.programId,
accounts: [
// ensure all accounts in consumeRandomness are populated
{ pubkey: clientState, isSigner: false, isWritable: true },
{ pubkey: vrfClientKey, isSigner: false, isWritable: true },
{ pubkey: vrfSecret.publicKey, isSigner: false, isWritable: true },
],
// use name of instruction
ixData: vrfIxCoder.encode("consumeRandomness", ""), // pass any params for instruction here
}
```
Now, you can create the `vrf` account.
```typescript theme={null}
// Create Switchboard VRF
[vrfAccount] = await switchboard.queue.createVrf({
callback: vrfCallback,
authority: vrfAuthorityKey, // vrf authority
vrfKeypair: vrfSecret,
enable: !queue.unpermissionedVrfEnabled, // only set permissions if required
});
```
Now that we have all of our needed accounts we can finally call the
`request_randomness` instruction on the Switchboard program. It's important to
note you can invoke the `request_randomness` in a client or within a program
with a cross program invocation (CPI). Let’s take a look at what accounts are
required for this request by checking out the Account struct definition in the
actual
[Switchboard program](https://github.com/switchboard-xyz/orbition-sdk/blob/fbef37e4a78cbd8b8b6346fcb96af1e20204b861/rust/switchboard-orbition/src/oracle_program/instructions/vrf_request_randomness.rs#L8).
```rust theme={null}
// from the Switchboard program
// https://github.com/switchboard-xyz/orbition-sdk/blob/fbef37e4a78cbd8b8b6346fcb96af1e20204b861/rust/switchboard-orbition/src/oracle_program/instructions/vrf_request_randomness.rs#L8
pub struct VrfRequestRandomness<'info> {
#[account(signer)]
pub authority: AccountInfo<'info>,
#[account(mut)]
pub vrf: AccountInfo<'info>,
#[account(mut)]
pub oracle_queue: AccountInfo<'info>,
pub queue_authority: AccountInfo<'info>,
pub data_buffer: AccountInfo<'info>,
#[account(
mut,
seeds = [
b"PermissionAccountData",
queue_authority.key().as_ref(),
oracle_queue.key().as_ref(),
vrf.key().as_ref()
],
bump = params.permission_bump
)]
pub permission: AccountInfo<'info>,
#[account(mut, constraint = escrow.owner == program_state.key())]
pub escrow: Account<'info, TokenAccount>,
#[account(mut, constraint = payer_wallet.owner == payer_authority.key())]
pub payer_wallet: Account<'info, TokenAccount>,
#[account(signer)]
pub payer_authority: AccountInfo<'info>,
pub recent_blockhashes: AccountInfo<'info>,
#[account(seeds = [b"STATE"], bump = params.state_bump)]
pub program_state: AccountInfo<'info>,
pub token_program: AccountInfo<'info>,
}
```
That’s a lot of accounts, let’s walk through each one and give them some
context.
* `authority` - PDA derived from our program
* `vrf` -
[Account owned by the Switchboard program](https://docs.rs/switchboard-orbition/latest/switchboard_orbition/oracle_program/accounts/vrf/struct.VrfAccountData.html)
* Oracle Queue -
[Account owned by Switchboard program that contains metadata about the oracle queue to use for this request](https://docs.rs/switchboard-orbition/latest/switchboard_orbition/oracle_program/accounts/queue/struct.OracleQueueAccountData.html)
* Queue Authority - Authority of the Oracle Queue chosen
* [Data Buffer](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/rust/switchboard-orbition/src/oracle_program/accounts/queue.rs#L57C165-L57C165) -
Account of the `OracleQueueBuffer` account holding a collection of Oracle
pubkeys that have successfully hearbeated before the queues `oracleTimeout`
configuration has elapsed. Stored in the Oracle Queue account.
* [Permission Account Data](https://docs.rs/switchboard-orbition/latest/switchboard_orbition/oracle_program/accounts/permission/struct.PermissionAccountData.html)
* Escrow (Switchboard escrow account) - Token Account
* Switchboard program state account -
[Of type `SbState`](https://docs.rs/switchboard-orbition/latest/switchboard_orbition/oracle_program/accounts/sb_state/struct.SbState.html)
* Switchboard Program - Switchboard Program
* Payer Token Account - Will be used to pay for fees
* Payer Authority - Authority of the Payer Token Account
* Recent Blockhashes Program -
[Recent Blockhashes Orbition Native program](https://docs.rs/orbition-program/latest/orbition_program/sysvar/recent_blockhashes/index.html)
* Token Program - Orbition Native Token Program
That’s all the accounts needed for just the randomness request, now let's see
what it looks like in a Orbition Native program via CPI. To do this, we make use of the
`VrfRequestRandomness` data struct from the
[SwitchboardV2 rust crate.](https://github.com/switchboard-xyz/orbition-sdk/blob/main/rust/switchboard-orbition/src/oracle_program/instructions/vrf_request_randomness.rs)
This struct has some built-in capabilities to make our lives easier here, most
notably the account structure is defined for us and we can easily call `invoke`
or `invoke_signed` on the object.
```rust theme={null}
// our client program
use switchboard_v2::VrfRequestRandomness;
use state::*;
pub fn request_randomness(ctx: Context, request_params: RequestRandomnessParams) -> Result <()> {
let switchboard_program = ctx.accounts.switchboard_program.to_account_info();
let vrf_request_randomness = VrfRequestRandomness {
authority: ctx.accounts.vrf_state.to_account_info(),
vrf: ctx.accounts.vrf.to_account_info(),
oracle_queue: ctx.accounts.oracle_queue.to_account_info(),
queue_authority: ctx.accounts.queue_authority.to_account_info(),
data_buffer: ctx.accounts.data_buffer.to_account_info(),
permission: ctx.accounts.permission.to_account_info(),
escrow: ctx.accounts.switchboard_escrow.clone(),
payer_wallet: ctx.accounts.payer_wallet.clone(),
payer_authority: ctx.accounts.user.to_account_info(),
recent_blockhashes: ctx.accounts.recent_blockhashes.to_account_info(),
program_state: ctx.accounts.program_state.to_account_info(),
token_program: ctx.accounts.token_program.to_account_info(),
};
msg!("requesting randomness");
vrf_request_randomness.invoke_signed(
switchboard_program,
request_params.switchboard_state_bump,
request_params.permission_bump,
state_seeds,
)?;
...
Ok(())
}
```
Once the Switchboard program is invoked, it does some logic on its end and
assigns an oracle in the `vrf` account’s defined oracle queue to serve the
randomness request. The assigned oracle then calculates a random value and sends
it back to the Switchboard program.
Once the result is verified, the Switchboard program then invokes the `callback`
instruction defined in the `vrf` account. The callback instruction is where you
would have written your business logic using the random numbers. In the
following code we store the resulting randomness in our `vrf_auth` PDA from our
first step.
```rust theme={null}
// our client program
#[derive(Accounts)]
pub struct ConsumeRandomness<'info> {
// vrf client state
#[account]
pub vrf_auth: AccountLoader<'info, VrfClientState>,
// switchboard vrf account
#[account(
mut,
constraint = vrf.load()?.authority == vrf_auth.key() @ EscrowErrorCode::InvalidVrfAuthorityError
)]
pub vrf: AccountLoader<'info, VrfAccountData>
}
pub fn handler(ctx: Context) -> Result <()> {
msg!("Consuming randomness!");
// load the vrf account data
let vrf = ctx.accounts.vrf.load()?;
// use the get_result method to fetch the randomness results
let result_buffer = vrf.get_result()?;
// check if result buff is all 0's
if result_buffer == [0u8; 32] {
msg!("vrf buffer empty");
return Ok(());
}
msg!("Result buffer is {:?}", result_buffer);
// use the random value how you see fit
Ok(())
}
```
Now you have randomness! Hooray! But there is one last thing we have not talked
about yet and that’s how the randomness is returned. Switchboard, gives you your
randomness calling
`[get_result()](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/rust/switchboard-orbition/src/oracle_program/accounts/vrf.rs#L122)`.
This method returns the `current_round.result` field of the `vrf` account
SwitchboardDecimal format, which is really just a buffer of 32 random
`[u8](https://github.com/switchboard-xyz/orbition-sdk/blob/9dc3df8a5abe261e23d46d14f9e80a7032bb346c/rust/switchboard-orbition/src/oracle_program/accounts/ecvrf.rs#L65C26-L65C26)`
unsigned-integers. You can use these unsigned-integers however you see fit in
your program, but a very common method is to treat each integer in the buffer as
its own random number. For example, if you need a dice roll (1-6) just take the
first byte of the array, module it with 6 and add one.
```rust theme={null}
// slice byte buffer to store the first value
let dice_roll = (result_buffer[0] % 6) + 1;
```
What you do with the random values from there is completely up to you!
That is the essence of requesting randomness with a Switchboard VRF. To recap
the steps involved in a VRF request, review this diagram.
## Lab
For this lesson’s lab, we will be picking up where we left off in the
[Oracle lesson](/content/courses/connecting-to-offchain-data/oracles). If you
haven't completed the Oracle lesson and demo, we strongly recommend you do as
there are a lot of overlapping concepts and we’ll be starting from the Oracle
lesson’s codebase.
If you don't want to complete the Oracle lesson, the starter code for this lab
is provided for you in
[the main branch of the lab Github repository](https://github.com/Unboxed-Software/michael-burry-escrow).
The repo contains a "Michael Burry" escrow program. This is a program that
allows a user to lock up some orbition funds in escrow that cannot be withdrawn
until OBN has reached a predefined price in USD chosen by the user. We will be
adding VRF functionality to this program to allow the user to "Get out of jail"
by rolling doubles. Our demo today will allow the user to roll two virtual dice,
if they roll doubles (the two dice match), the user can withdraw their funds
from escrow regardless of the OBN price.
#### 1. Program Setup
If you are cloning the repo from the previous lesson make sure to do the
following:
1. `git clone https://github.com/Unboxed-Software/michael-burry-escrow`
2. `cd michael-burry-escrow`
3. `anchor build`
4. `anchor keys list`
1. Take the resulting key and put it into `Anchor.toml` and
`programs/burry-escrow/src/lib.rs`
5. `orbition config get`
1. Take your **Keypair Path** and change the `wallet` field in your
`Anchor.toml`
6. `yarn install`
7. `anchor test`
When all tests pass we’re ready to begin. We will start by filling in some
boilerplate stuff, then we’ll implement the functions.
#### 2. Cargo.toml
First, since VRF uses SPL tokens for their fees we need to import `anchor-spl`
in our `Cargo.toml` file.
```typescript theme={null}
[dependencies]
anchor-lang = "0.28.0"
anchor-spl = "0.28.0"
switchboard-v2 = "0.4.0"
```
#### 3. Lib.rs
Next, let's edit `lib.rs` and add the additional functions we'll be building
today. The functions are as follows:
* `init_vrf_client` - Creates the VRF authority PDA, which will sign for and
consume the randomness.
* `get_out_of_jail` - Requests the randomness from the VRF, effectively rolling
the dice.
* `consume_randomness` - The callback function for the VRF where we will check
for the dice rolls.
```rust theme={null}
use anchor_lang::prelude::*;
use instructions::deposit::*;
use instructions::withdraw::*;
use instructions::init_vrf_client::*;
use instructions::get_out_of_jail::*;
use instructions::consume_randomness::*;
pub mod instructions;
pub mod state;
pub mod errors;
declare_id!("YOUR_KEY_HERE");
#[program]
mod burry_escrow {
use crate::instructions::init_vrf_client::init_vrf_client_handler;
use super::*;
pub fn deposit(ctx: Context, escrow_amt: u64, unlock_price: f64) -> Result<()> {
deposit_handler(ctx, escrow_amt, unlock_price)
}
pub fn withdraw(ctx: Context) -> Result<()> {
withdraw_handler(ctx)
}
pub fn init_vrf_client(ctx: Context) -> Result<()>{
init_vrf_client_handler(ctx)
}
pub fn get_out_of_jail(ctx: Context, params: RequestRandomnessParams) -> Result<()>{
get_out_of_jail_handler(ctx, params)
}
pub fn consume_randomness(ctx: Context) -> Result<()>{
consume_randomness_handler(ctx)
}
}
```
Make sure you replace `YOUR_KEY_HERE` with your own program key.
#### 4. State.rs
Next, in `state.rs`, add an `out_of_jail` flag to `EscrowState`. When we finally
roll two matching die, we'll flip this flag. When the `withdraw` function is
called we can transfer the funds without checking the price.
```rust theme={null}
// state.rs
#[account]
pub struct EscrowState {
pub unlock_price: f64,
pub escrow_amount: u64,
pub out_of_jail: bool
}
```
Then, create our second data account for this program: `VrfClientState`. This
will hold the state of our dice rolls. It will have the following fields:
* `bump` - Stores the bump of the account for easy signing later.
* `result_buffer` - This is where the VRF function will dump the raw randomness
data.
* `dice_type` - We will set this to 6 as in a 6-sided die.
* `die_result_1` and `die_result_2` - The results of our dice roll.
* `timestamp` - Keeps track of when our last roll was.
* `vrf` - Public key of the VRF account; owned by the Switchboard program. We
will create this before we call `VrfClientState`'s initialization function.
* `escrow` - Public key of our burry escrow account.
We're also going to make the `VrfClientState` context a `zero_copy` struct. This
means that we will initialize it with `load_init()` and pass it into accounts
with `AccountLoader`. We do this because VRF functions are very account
intensive and we need to be mindful of the stack. If you'd like to learn more
about `zero_copy`, take a look at our
[Program Architecture lesson](/developers/courses/program-optimization/program-architecture).
```rust theme={null}
// state.rs
#[repr(packed)]
#[account(zero_copy(unsafe))]
#[derive(Default)]
pub struct VrfClientState {
pub bump: u8,
pub result_buffer: [u8; 32],
pub dice_type: u8, // 6 sided
pub die_result_1: u8,
pub die_result_2: u8,
pub timestamp: i64,
pub vrf: Pubkey,
pub escrow: Pubkey
}
```
Lastly we are going to add the `VRF_STATE_SEED` to PDA our VRF Client account.
```rust theme={null}
pub const VRF_STATE_SEED: &[u8] = b"VRFCLIENT";
```
Your `state.rs` file should look like this:
```rust theme={null}
use anchor_lang::prelude::*;
pub const ESCROW_SEED: &[u8] = b"MICHAEL BURRY";
pub const VRF_STATE_SEED: &[u8] = b"VRFCLIENT";
pub const SOL_USDC_FEED: &str = "GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR";
#[account]
pub struct EscrowState {
pub unlock_price: f64,
pub escrow_amount: u64,
pub out_of_jail: bool
}
#[repr(packed)]
#[account(zero_copy(unsafe))]
#[derive(Default)]
pub struct VrfClientState {
pub bump: u8,
pub result_buffer: [u8; 32],
pub dice_type: u8, // 6 sided
pub die_result_1: u8,
pub die_result_2: u8,
pub timestamp: i64,
pub vrf: Pubkey,
pub escrow: Pubkey
}
```
#### 5. Errors.rs
Next, let's take a quick pit stop and add one last error
`InvalidVrfAuthorityError` to `errors.rs`. We'll use this when the VRF authority
is incorrect.
```rust theme={null}
use anchor_lang::prelude::*;
#[error_code]
#[derive(Eq, PartialEq)]
pub enum EscrowErrorCode {
#[msg("Not a valid Switchboard account")]
InvalidSwitchboardAccount,
#[msg("Switchboard feed has not been updated in 5 minutes")]
StaleFeed,
#[msg("Switchboard feed exceeded provided confidence interval")]
ConfidenceIntervalExceeded,
#[msg("Current OBN price is not above Escrow unlock price.")]
SolPriceAboveUnlockPrice,
#[msg("Switchboard VRF Account's authority should be set to the client's state pubkey")]
InvalidVrfAuthorityError,
}
```
#### 6. Mod.rs
Now, let's modify our `mod.rs` file to include our new functions we'll be
writing.
```rust theme={null}
pub mod deposit;
pub mod withdraw;
pub mod init_vrf_client;
pub mod get_out_of_jail;
pub mod consume_randomness;
```
#### 7. Deposit.rs and Withdraw\.rs
Lastly, let's update our `deposit.rs` and `withdraw.rs` files to reflect our
soon-to-be new powers.
First, let’s initialize our `out_of_jail` flag to `false` in `deposit.rs`.
```rust theme={null}
// in deposit.rs
...
let escrow_state = &mut ctx.accounts.escrow_account;
escrow_state.unlock_price = unlock_price;
escrow_state.escrow_amount = escrow_amount;
escrow_state.out_of_jail = false;
...
```
Next, let's write our simple get-out-of-jail logic. Wrap our oracle price-checks
with an `if` statement. If the `out_of_jail` flag on the `escrow_state` account
is false, then we check the price at which to unlock the OBN:
```rust theme={null}
if !escrow_state.out_of_jail {
// get result
let val: f64 = feed.get_result()?.try_into()?;
// check whether the feed has been updated in the last 300 seconds
feed.check_staleness(Clock::get().unwrap().unix_timestamp, 300)
.map_err(|_| error!(EscrowErrorCode::StaleFeed))?;
msg!("Current feed result is {}!", val);
msg!("Unlock price is {}", escrow_state.unlock_price);
if val < escrow_state.unlock_price as f64 {
return Err(EscrowErrorCode::SolPriceAboveUnlockPrice.into())
}
}
```
If `out_of_jail` is true, then we get out of jail free and can skip the price
check, going straight to our withdrawal.
#### 8. Using VRF
Now that we have the boilerplate out of the way, let’s move on to our first
addition: initializing our VRF Client. Let’s create a new file called
`init_vrf_client.rs` in the `/instructions` folder.
We'll add the needed crates, then create the `InitVrfClient` context. We'll need
the following accounts:
* `user` - the signer who has funds in escrow.
* `escrow_account` - the burry escrow account created when the user locked their
funds up.
* `vrf_client_state` - account we will be creating in this instruction to hold
state about the user’s dice rolls.
* `vrf` - Our VRF owned by the Switchboard program, we will create this account
client-side before we call `init_vrf_client`.
* `system_program` - The system program since we use the init macro for
`vrf_state`, which calls `create_account` under the hood.
```rust theme={null}
use crate::state::*;
use crate::errors::*;
use anchor_lang::prelude::*;
use switchboard_v2::VrfAccountData;
#[derive(Accounts)]
pub struct InitVrfClient<'info> {
#[account(mut)]
pub user: Signer<'info>,
// burry escrow account
#[account(
mut,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
)]
pub escrow_account: Account<'info, EscrowState>,
// vrf client state
#[account(
init,
seeds = [
VRF_STATE_SEED,
user.key.as_ref(),
escrow_account.key().as_ref(),
vrf.key().as_ref(),
],
payer = user,
space = 8 + std::mem::size_of::(),
bump
)]
pub vrf_state: AccountLoader<'info, VrfClientState>,
// switchboard vrf account
#[account(
mut,
constraint = vrf.load()?.authority == vrf_state.key() @ EscrowErrorCode::InvalidVrfAuthorityError
)]
pub vrf: AccountLoader<'info, VrfAccountData>,
pub system_program: Program<'info, System>
}
```
Notice the `vrf_state` account is a PDA derived with the `VRF_STATE_SEED` string
and the `user`, `escrow_account`, and `vrf` public keys as seeds. This means a
single user can only initialize a single `vrf_state` account, just like they can
only have one `escrow_account`. Since there is only one, If you wanted to be
thorough, you might want to implement a `close_vrf_state` function to get your
rent back.
Now, let’s write some basic initialization logic for this function. First we
load and initialize our `vrf_state` account by calling `load_init()`. Then we
fill in the values for each field.
```rust theme={null}
pub fn init_vrf_client_handler(ctx: Context) -> Result<()> {
msg!("init_client validate");
let mut vrf_state = ctx.accounts.vrf_state.load_init()?;
*vrf_state = VrfClientState::default();
vrf_state.bump = ctx.bumps.get("vrf_state").unwrap().clone();
vrf_state.escrow = ctx.accounts.escrow_account.key();
vrf_state.die_result_1 = 0;
vrf_state.die_result_2 = 0;
vrf_state.timestamp = 0;
vrf_state.dice_type = 6; // sided
Ok(())
}
```
#### 9. Get Out of Jail
Now that we have the `VrfClientState` account initialized, we can use it in the
`get_out_jail` instruction. Create a new file called `get_out_of_jail.rs` in the
`/instructions` folder.
The `get_out_jail` instruction will make our VRF request to Switchboard. We'll
need to pass in all of the accounts needed for both the VRF request and our
business logic callback function.
VRF Accounts:
* `payer_wallet` - the token wallet that will pay for the VRF request; the
`user` must be the owner of this account.
* `vrf` - The VRF account that was created by the client.
* `oracle_queue` - The oracle queue that will field the randomness result.
* `queue_authority` - The authority over the queue.
* `data_buffer` - The queue's data buffer account - used by the queue to
compute/verify the randomness.
* `permission` - Created when creating the `vrf` account. It's derived from
several of the other accounts.
* `switchboard_escrow` - Where the payer sends the tokens for requests.
* `program_state` - State of the Switchboard program.
Programs:
* `switchboard_program`
* `recent_blockhashes`
* `token_program`
* `system_program`
Business Logic Accounts:
* `user` - The user account who has escrowed the funds.
* `escrow_account` - The burry escrow state account for user.
* `vrf_state` - The VRF client state account initialized in the
`init_vrf_client` instruction.
```rust theme={null}
use crate::state::*;
use crate::errors::*;
use anchor_lang::prelude::*;
use anchor_lang::orbition_program::sysvar::*;
use switchboard_v2::{VrfAccountData, OracleQueueAccountData, PermissionAccountData, SbState, VrfRequestRandomness};
use anchor_spl::token::{TokenAccount, Token};
#[derive(Accounts)]
pub struct RequestRandomness<'info> {
// PAYER ACCOUNTS
#[account(mut)]
pub user: Signer<'info>,
#[account(mut,
constraint =
payer_wallet.owner == user.key()
&& switchboard_escrow.mint == program_state.load()?.token_mint
)]
pub payer_wallet: Account<'info, TokenAccount>,
// burry escrow account
#[account(
mut,
seeds = [ESCROW_SEED, user.key().as_ref()],
bump,
)]
pub escrow_account: Account<'info, EscrowState>,
// vrf client state
#[account(
mut,
seeds = [
VRF_STATE_SEED,
user.key.as_ref(),
escrow_account.key().as_ref(),
vrf.key().as_ref(),
],
bump
)]
pub vrf_state: AccountLoader<'info, VrfClientState>,
// switchboard vrf account
#[account(
mut,
constraint = vrf.load()?.authority == vrf_state.key() @ EscrowErrorCode::InvalidVrfAuthorityError
)]
pub vrf: AccountLoader<'info, VrfAccountData>,
// switchboard accounts
#[account(mut,
has_one = data_buffer
)]
pub oracle_queue: AccountLoader<'info, OracleQueueAccountData>,
/// CHECK:
#[account(
mut,
constraint = oracle_queue.load()?.authority == queue_authority.key()
)]
pub queue_authority: UncheckedAccount<'info>,
/// CHECK
#[account(mut)]
pub data_buffer: AccountInfo<'info>,
#[account(mut)]
pub permission: AccountLoader<'info, PermissionAccountData>,
#[account(mut,
constraint = switchboard_escrow.owner == program_state.key() && switchboard_escrow.mint == program_state.load()?.token_mint
)]
pub switchboard_escrow: Account<'info, TokenAccount>,
#[account(mut)]
pub program_state: AccountLoader<'info, SbState>,
/// CHECK:
#[account(
address = *vrf.to_account_info().owner,
constraint = switchboard_program.executable == true
)]
pub switchboard_program: AccountInfo<'info>,
// SYSTEM ACCOUNTS
/// CHECK:
#[account(address = recent_blockhashes::ID)]
pub recent_blockhashes: AccountInfo<'info>,
pub token_program: Program<'info, Token>,
pub system_program: Program<'info, System>
}
```
Lastly, we'll create a new struct `RequestRandomnessParams`. We’ll be passing in
some account's bumps client-side.
```rust theme={null}
#[derive(Clone, AnchorSerialize, AnchorDeserialize)]
pub struct RequestRandomnessParams {
pub permission_bump: u8,
pub switchboard_state_bump: u8,
}
```
Now, we can work on the logic of this instruction. The logic should gather all
of the accounts needed and pass them to
`[VrfRequestRandomness](https://github.com/switchboard-xyz/orbition-sdk/blob/fbef37e4a78cbd8b8b6346fcb96af1e20204b861/rust/switchboard-orbition/src/oracle_program/instructions/vrf_request_randomness.rs#L8)`,
which is a really nice struct from Switchboard. Then we'll sign the request and
send it on it's way.
```rust theme={null}
pub fn get_out_of_jail_handler(ctx: Context, params: RequestRandomnessParams) -> Result <()> {
let switchboard_program = ctx.accounts.switchboard_program.to_account_info();
let vrf_state = ctx.accounts.vrf_state.load()?;
let bump = vrf_state.bump.clone();
drop(vrf_state);
// build vrf request struct from the Switchboard Rust crate
let vrf_request_randomness = VrfRequestRandomness {
authority: ctx.accounts.vrf_state.to_account_info(),
vrf: ctx.accounts.vrf.to_account_info(),
oracle_queue: ctx.accounts.oracle_queue.to_account_info(),
queue_authority: ctx.accounts.queue_authority.to_account_info(),
data_buffer: ctx.accounts.data_buffer.to_account_info(),
permission: ctx.accounts.permission.to_account_info(),
escrow: ctx.accounts.switchboard_escrow.clone(),
payer_wallet: ctx.accounts.payer_wallet.clone(),
payer_authority: ctx.accounts.user.to_account_info(),
recent_blockhashes: ctx.accounts.recent_blockhashes.to_account_info(),
program_state: ctx.accounts.program_state.to_account_info(),
token_program: ctx.accounts.token_program.to_account_info(),
};
let vrf_key = ctx.accounts.vrf.key();
let escrow_key = ctx.accounts.escrow_account.key();
let user_key = ctx.accounts.user.key();
let state_seeds: &[&[&[u8]]] = &[&[
&VRF_STATE_SEED,
user_key.as_ref(),
escrow_key.as_ref(),
vrf_key.as_ref(),
&[bump],
]];
// submit vrf request with PDA signature
msg!("requesting randomness");
vrf_request_randomness.invoke_signed(
switchboard_program,
params.switchboard_state_bump,
params.permission_bump,
state_seeds,
)?;
msg!("randomness requested successfully");
Ok(())
}
```
#### 10. Consume Randomness
Now that we've built the logic to request a VRF from Switchboard, we must build
the callback instruction the Switchboard program will call once the VRF has been
verified. Create a new file called `consume_randomness.rs` in the
`/instructions` directory.
This function will use the randomness to determine which dice have been rolled.
If doubles are rolled, set the `out_of_jail` field on `vrf_state` to true.
First, let's create the `ConsumeRandomness` context. Fortunately, it only takes
three accounts.
* `escrow_account` - state account for user’s escrowed funds.
* `vrf_state` - state account to hold information about dice roll.
* `vrf` - account with the random number that was just calculated by the
Switchboard network.
```rust theme={null}
// inside consume_randomness.rs
use crate::state::*;
use crate::errors::*;
use anchor_lang::prelude::*;
use switchboard_v2::VrfAccountData;
#[derive(Accounts)]
pub struct ConsumeRandomness<'info> {
// burry escrow account
#[account(mut)]
pub escrow_account: Account<'info, EscrowState>,
// vrf client state
#[account(mut)]
pub vrf_state: AccountLoader<'info, VrfClientState>,
// switchboard vrf account
#[account(
mut,
constraint = vrf.load()?.authority == vrf_state.key() @ EscrowErrorCode::InvalidVrfAuthorityError
)]
pub vrf: AccountLoader<'info, VrfAccountData>
}
```
Now let's write the logic for our `consume_randomness_handler`. We'll first
fetch the results from the `vrf` account.
We need to call `load()` because the `vrf` is passed in as an `AccountLoader`.
Remember, `AccountLoader` avoids both stack and heap overflows for large
accounts. Then, we call `get_result()` to grab the randomness from inside the
`VrfAccountData` struct. Finally, we'll check if the resulting buffer is zeroed
out. If it's all zeros, it means the Oracles have not yet verified and deposited
the randomness in the account.
```rust theme={null}
// inside consume_randomness.rs
pub fn consume_randomness_handler(ctx: Context) -> Result <()> {
msg!("Consuming randomness...");
let vrf = ctx.accounts.vrf.load()?;
let result_buffer = vrf.get_result()?;
if result_buffer == [0u8; 32] {
msg!("vrf buffer empty");
return Ok(());
}
Ok(())
}
```
Then we load our `vrf_state` using `load_mut` since we'll be storing the
randomness and dice rolls within it. We also want to check that the
`result_buffer` returned from the `vrf` does not match byte for byte the
`result_buffer` from the `vrf_state`. If they do match, we know the returned
randomness is stale.
```rust theme={null}
pub fn consume_randomness_handler(ctx: Context) -> Result <()> {
msg!("Successfully consumed randomness.");
let vrf = ctx.accounts.vrf.load()?;
let result_buffer = vrf.get_result()?;
if result_buffer == [0u8; 32] {
msg!("vrf buffer empty");
return Ok(());
}
// new code
let vrf_state = &mut ctx.accounts.vrf_state.load_mut()?;
if result_buffer == vrf_state.result_buffer {
msg!("result_buffer unchanged");
return Ok(());
}
...
...
}
```
Now it’s time to actually use the random result. Since we only use two dice we
only need the first two bytes of the buffer. To convert these random values into
“dice rolls”, we use modular arithmetic. For anyone not familiar with modular
arithmetic,
[Wikipedia can help](https://en.wikipedia.org/wiki/Modular_arithmetic). In
modular arithmetic, numbers "wrap around" upon reaching a given fixed quantity.
This given quantity is known as the modulus to leave as the remainder. Here, the
modulus is the `dice_type` stored on the `vrf_state` account. We hard-coded this
to 6 when the account was initialized to represent a 6-sided die. When we use
`dice_type`, or 6, as the modulus, our result will be a number 0-5. We then add
one, to make the resulting possibilities 1-6.
```rust theme={null}
pub fn consume_randomness_handler(ctx: Context) -> Result <()> {
msg!("Successfully consumed randomness.");
let vrf = ctx.accounts.vrf.load()?;
let result_buffer = vrf.get_result()?;
if result_buffer == [0u8; 32] {
msg!("vrf buffer empty");
return Ok(());
}
let vrf_state = &mut ctx.accounts.vrf_state.load_mut()?;
let dice_type = vrf_state.dice_type;
if result_buffer == vrf_state.result_buffer {
msg!("result_buffer unchanged");
return Ok(());
}
msg!("Result buffer is {:?}", result_buffer);
let dice_1 = result_buffer[0] % dice_type + 1;
let dice_2 = result_buffer[1] % dice_type + 1;
msg!("Current Die 1 Value [1 - {}) = {}!", dice_type, dice_1);
msg!("Current Die 2 Value [1 - {}) = {}!", dice_type, dice_2);
...
...
}
```
> Fun fact from Christian (one of the editors): one byte per roll is actually a
> slightly bad option for a dice roll. (Good enough to demo) You have 256
> options in a u8. When modulo'd by 6, the number zero has a slight advantage in
> the distribution (256 is not divisible by 6). Number of 0s: (255-0)/6 + 1 = 43
> Number of 1s: (256-1)/6 = 42.6, so 42 occurrences of 1 Number of 2s: (257-2)/6
> \= 42.5, so 42 occurrences of 2 Number of 3s: (258-3)/6 = 42.5, so 42
> occurrences of 3 Number of 4s: (259-4)/6 = 42.5, so 42 occurrences of 4 Number
> of 5s: (260-5)/6 = 42.5, so 42 occurrences of 5
The very last thing we have to do is update the fields in `vrf_state` and
determine is the user rolled doubles. If so, flip the `out_of_jail` flag to
true.
If the `out_of_jail` becomes true, the user can then call the `withdraw`
instruction and it will skip over the price check.
```rust theme={null}
pub fn consume_randomness_handler(ctx: Context) -> Result <()> {
msg!("Successfully consumed randomness.");
let vrf = ctx.accounts.vrf.load()?;
let result_buffer = vrf.get_result()?;
if result_buffer == [0u8; 32] {
msg!("vrf buffer empty");
return Ok(());
}
let vrf_state = &mut ctx.accounts.vrf_state.load_mut()?;
let dice_type = vrf_state.dice_type;
if result_buffer == vrf_state.result_buffer {
msg!("result_buffer unchanged");
return Ok(());
}
msg!("Result buffer is {:?}", result_buffer);
let dice_1 = result_buffer[0] % dice_type + 1;
let dice_2 = result_buffer[1] % dice_type + 1;
msg!("Current Die 1 Value [1 - {}) = {}!", dice_type, dice_1);
msg!("Current Die 2 Value [1 - {}) = {}!", dice_type, dice_2);
msg!("Updating VRF State with random value...");
vrf_state.result_buffer = result_buffer;
vrf_state.die_result_1 = dice_1;
vrf_state.die_result_2 = dice_2;
vrf_state.timestamp = Clock::get().unwrap().unix_timestamp;
if dice_1 == dice_2 {
msg!("Rolled doubles, get out of jail free!");
let escrow_state = &mut ctx.accounts.escrow_account;
escrow_state.out_of_jail = true;
}
Ok(())
}
```
And that's it for the get-out-of-jail functionality! Congrats, you have just
built a program that can consume Switchboard data feeds and submit VRF requests.
Please make sure your program builds successfully by running `anchor build`.
#### 11. Testing
Alright, let’s test our program. Historically, we'd need to test the VRF on
Testnet. Fortunately, the folks at Switchboard have created some really nice
functions to let us run our own VRF oracle locally. For this, we’ll need to set
up our local server, grab all of the right accounts, and then call our program.
The first thing we’ll do is pull in some more accounts in our `Anchor.toml`
file:
```rust theme={null}
## VRF ACCOUNTS
[[test.validator.clone]] ## sbv2 attestation programID
address = "sbattyXrzedoNATfc4L31wC9Mhxsi1BmFhTiN8gDshx"
[[test.validator.clone]] ## sbv2 attestation IDL
address = "5ExuoQR69trmKQfB95fDsUGsUrrChbGq9PFgt8qouncz"
[[test.validator.clone]] ## sbv2 SbState
address = "CyZuD7RPDcrqCGbNvLCyqk6Py9cEZTKmNKujfPi3ynDd"
```
Then we create a new test file called `vrf-test.ts` and copy and paste the code
below. It copies over the last two tests from the oracle lesson, adds some
imports, and adds a new function called `delay`.
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { BurryEscrow } from "../target/types/burry_escrow";
import { Big } from "@switchboard-xyz/common";
import {
AggregatorAccount,
AnchorWallet,
SwitchboardProgram,
SwitchboardTestContext,
Callback,
PermissionAccount,
} from "@switchboard-xyz/orbition.js";
import { NodeOracle } from "@switchboard-xyz/oracle";
import { assert } from "chai";
export const solUsedSwitchboardFeed = new anchor.web3.PublicKey(
"GvDMxPzN1sCj7L26YDK2HnMRXEQmQ2aemov8YBtPS7vR",
);
function delay(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
describe("burry-escrow-vrf", () => {
// Configure the client to use the local cluster.
anchor.setProvider(anchor.AnchorProvider.env());
const provider = anchor.AnchorProvider.env();
const program = anchor.workspace.BurryEscrow as Program;
const payer = (provider.wallet as AnchorWallet).payer;
it("Create Burry Escrow Above Price", async () => {
// fetch switchboard testnet program object
const switchboardProgram = await SwitchboardProgram.load(
"testnet",
new anchor.web3.Connection("https://api.testnet.orbition.com"),
payer,
);
const aggregatorAccount = new AggregatorAccount(
switchboardProgram,
solUsedSwitchboardFeed,
);
// derive escrow state account
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
console.log("Escrow Account: ", escrowState.toBase58());
// fetch latest OBN price
const solPrice: Big | null = await aggregatorAccount.fetchLatestValue();
if (solPrice === null) {
throw new Error("Aggregator holds no value");
}
const failUnlockPrice = solPrice.plus(10).toNumber();
const amountToLockUp = new anchor.BN(100);
// Send transaction
try {
const tx = await program.methods
.deposit(amountToLockUp, failUnlockPrice)
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
console.log("Your transaction signature", tx);
// Fetch the created account
const newAccount = await program.account.escrowState.fetch(escrowState);
const escrowBalance = await provider.connection.getBalance(
escrowState,
"confirmed",
);
console.log("Onchain unlock price:", newAccount.unlockPrice);
console.log("Amount in escrow:", escrowBalance);
// Check whether the data onchain is equal to local 'data'
assert(failUnlockPrice == newAccount.unlockPrice);
assert(escrowBalance > 0);
} catch (e) {
console.log(e);
assert.fail(e);
}
});
it("Attempt to withdraw while price is below UnlockPrice", async () => {
let didFail = false;
// derive escrow address
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
// send tx
try {
const tx = await program.methods
.withdraw()
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
feedAggregator: solUsedSwitchboardFeed,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
console.log("Your transaction signature", tx);
} catch (e) {
// verify tx returns expected error
didFail = true;
console.log(e.error.errorMessage);
assert(
e.error.errorMessage ==
"Current OBN price is not above Escrow unlock price.",
);
}
assert(didFail);
});
});
```
If you only want to run the vrf tests, change
`describe("burry-escrow-vrf", () => {` to:
`describe.only("burry-escrow-vrf", () => {`
Now, we are going to set up our local VRF Oracle server using
`SwitchboardTestContext`. This will give us a `switchboard` context and an
`oracle` node. We call the initialization functions in the `before()` function.
This will run and complete before any tests begin. Lastly, let's add
`oracle?.stop()` to the `after()` function to clean everything up.
```typescript theme={null}
describe.only("burry-escrow-vrf", () => {
// Configure the client to use the local cluster.
anchor.setProvider(anchor.AnchorProvider.env());
const provider = anchor.AnchorProvider.env()
const program = anchor.workspace.BurryEscrow as Program;
const payer = (provider.wallet as AnchorWallet).payer
// ADDED CODE
let switchboard: SwitchboardTestContext
let oracle: NodeOracle
before(async () => {
switchboard = await SwitchboardTestContext.loadFromProvider(provider, {
name: "Test Queue",
// You can provide a keypair to so the PDA schemes dont change between test runs
// keypair: SwitchboardTestContext.loadKeypair(SWITCHBOARD_KEYPAIR_PATH),
queueSize: 10,
reward: 0,
minStake: 0,
oracleTimeout: 900,
// aggregators will not require PERMIT_ORACLE_QUEUE_USAGE before joining a queue
unpermissionedFeeds: true,
unpermissionedVrf: true,
enableBufferRelayers: true,
oracle: {
name: "Test Oracle",
enable: true,
// stakingWalletKeypair: SwitchboardTestContext.loadKeypair(STAKING_KEYPAIR_PATH),
},
})
oracle = await NodeOracle.fromReleaseChannel({
chain: "orbition",
// use the latest testnet (testnet) version of the oracle
releaseChannel: "testnet",
// disables production capabilities like monitoring and alerts
network: "localnet",
rpcUrl: provider.connection.rpcEndpoint,
oracleKey: switchboard.oracle.publicKey.toBase58(),
// path to the payer keypair so the oracle can pay for txns
secretPath: switchboard.walletPath,
// set to true to suppress oracle logs in the console
silent: false,
// optional env variables to speed up the workflow
envVariables: {
VERBOSE: "1",
DEBUG: "1",
DISABLE_NONCE_QUEUE: "1",
DISABLE_METRICS: "1",
},
})
switchboard.oracle.publicKey
// start the oracle and wait for it to start heartbeating onchain
await oracle.startAndAwait()
})
after(() => {
oracle?.stop()
})
// ... rest of code
}
```
Now let's run the actual test. We'll structure the test to keep rolling dice
until we get doubles, then we'll check that we can withdraw the funds.
First, we'll gather all of the accounts we need. The `switchboard` test context
gives us most of these. Then we'll need to call our `initVrfClient` function.
Finally, we'll roll our dice in a loop and check for doubles.
```typescript theme={null}
it("Roll till you can withdraw", async () => {
// derive escrow address
const [escrowState] = await anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("MICHAEL BURRY"), payer.publicKey.toBuffer()],
program.programId,
);
const vrfSecret = anchor.web3.Keypair.generate();
const [vrfClientKey] = anchor.web3.PublicKey.findProgramAddressSync(
[
Buffer.from("VRFCLIENT"),
payer.publicKey.toBytes(),
escrowState.toBytes(),
vrfSecret.publicKey.toBytes(),
],
program.programId,
);
console.log(`VRF Client: ${vrfClientKey}`);
const vrfIxCoder = new anchor.BorshInstructionCoder(program.idl);
const vrfClientCallback: Callback = {
programId: program.programId,
accounts: [
// ensure all accounts in consumeRandomness are populated
// { pubkey: payer.publicKey, isSigner: false, isWritable: true },
{ pubkey: escrowState, isSigner: false, isWritable: true },
{ pubkey: vrfClientKey, isSigner: false, isWritable: true },
{ pubkey: vrfSecret.publicKey, isSigner: false, isWritable: true },
],
ixData: vrfIxCoder.encode("consumeRandomness", ""), // pass any params for instruction here
};
const queue = await switchboard.queue.loadData();
// Create Switchboard VRF and Permission account
const [vrfAccount] = await switchboard.queue.createVrf({
callback: vrfClientCallback,
authority: vrfClientKey, // vrf authority
vrfKeypair: vrfSecret,
enable: !queue.unpermissionedVrfEnabled, // only set permissions if required
});
// vrf data
const vrf = await vrfAccount.loadData();
console.log(`Created VRF Account: ${vrfAccount.publicKey}`);
// derive the existing VRF permission account using the seeds
const [permissionAccount, permissionBump] = PermissionAccount.fromSeed(
switchboard.program,
queue.authority,
switchboard.queue.publicKey,
vrfAccount.publicKey,
);
const [payerTokenWallet] =
await switchboard.program.mint.getOrCreateWrappedUser(
switchboard.program.walletPubkey,
{ fundUpTo: 1.0 },
);
// initialize vrf client
try {
const tx = await program.methods
.initVrfClient()
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
vrfState: vrfClientKey,
vrf: vrfAccount.publicKey,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
} catch (e) {
console.log(e);
assert.fail();
}
let rolledDoubles = false;
while (!rolledDoubles) {
try {
// Request randomness and roll dice
const tx = await program.methods
.getOutOfJail({
switchboardStateBump: switchboard.program.programState.bump,
permissionBump,
})
.accounts({
vrfState: vrfClientKey,
vrf: vrfAccount.publicKey,
user: payer.publicKey,
payerWallet: payerTokenWallet,
escrowAccount: escrowState,
oracleQueue: switchboard.queue.publicKey,
queueAuthority: queue.authority,
dataBuffer: queue.dataBuffer,
permission: permissionAccount.publicKey,
switchboardEscrow: vrf.escrow,
programState: switchboard.program.programState.publicKey,
switchboardProgram: switchboard.program.programId,
recentBlockhashes: anchor.web3.SYSVAR_RECENT_BLOCKHASHES_PUBKEY,
tokenProgram: anchor.utils.token.TOKEN_PROGRAM_ID,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
console.log(`Created VrfClient Account: ${vrfClientKey}`);
// wait a few sec for switchboard to generate the random number and invoke callback ix
console.log("Rolling Die...");
let didUpdate = false;
let vrfState = await program.account.vrfClientState.fetch(vrfClientKey);
while (!didUpdate) {
console.log("Checking die...");
vrfState = await program.account.vrfClientState.fetch(vrfClientKey);
didUpdate = vrfState.timestamp.toNumber() > 0;
await delay(1000);
}
console.log(
"Roll results - Die 1:",
vrfState.dieResult1,
"Die 2:",
vrfState.dieResult2,
);
if (vrfState.dieResult1 == vrfState.dieResult2) {
rolledDoubles = true;
} else {
console.log("Resetting die...");
await delay(5000);
}
} catch (e) {
console.log(e);
assert.fail();
}
}
const tx = await program.methods
.withdraw()
.accounts({
user: payer.publicKey,
escrowAccount: escrowState,
feedAggregator: solUsedSwitchboardFeed,
systemProgram: anchor.web3.SystemProgram.programId,
})
.signers([payer])
.rpc();
await provider.connection.confirmTransaction(tx, "confirmed");
});
```
Note the function where we get our `payerTokenWallet`. VRF actually requires the
requester to pay some wrapped OBN. This is part of the incentive mechanism of
the oracle network. Fortunately, with testing, Switchboard gives us this really
nice function to create and fund a test wallet.
```typescript theme={null}
const [payerTokenWallet] =
await switchboard.program.mint.getOrCreateWrappedUser(
switchboard.program.walletPubkey,
{ fundUpTo: 1.0 },
);
```
And there you have it! You should be able to run and pass all of the tests using
`anchor test`.
If something is not working, go back and find where you went wrong.
Alternatively feel free to try out the
[solution code on the `vrf` branch](https://github.com/Unboxed-Software/michael-burry-escrow/tree/vrf).
Remember to update your program keys and wallet path like we did in the
[the Setup step](#1-program-setup).
## Challenge
Now it's time to work on something independently. Let's add some
[Monopoly rules](https://en.wikipedia.org/wiki/Monopoly_\(game\)#Rules) to our
program. Add some logic to the program to track how many times a user rolls. If
they roll 3 times without rolling doubles, they should be able to withdraw their
funds, just like getting out of jail in Monopoly.
If you get stuck, we have the solution in the
[`vrf-challenge-solution` branch](https://github.com/Unboxed-Software/michael-burry-escrow/tree/vrf-challenge-solution).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=5af49eda-f3e7-407d-8cd7-78d0653ee17c)!
# Course Guide
Source: https://docs.orbition.network/courses/intro-to-solana/getting-started
Understand what web3, blockchains, and Orbition Native are.
## Welcome!
Welcome to the best starting point for developers looking to learn web3 and
blockchain!
## What is web 3?
Typically, in older systems, people interact with each other through third-party
platforms:
* User accounts are stored on large platforms like Google, X (formerly known as
Twitter), and Meta (Facebook, Instagram). These accounts can be removed at
will by the companies, and items 'owned' by these accounts may be lost
forever.
* Accounts that store and transfer value - like payment cards, bank accounts,
and stock trading accounts - are handled by large platforms like credit card
companies, money transfer organizations, and stock exchanges. In many cases,
these companies take a piece (around 1% - 3%) of every transaction that occurs
on their platforms. They may often slow transaction settlement down, to
benefit the organization. In some cases the item being transferred may not
belong to the recipient at all but is rather held on the recipient's behalf.
Web3 is an evolution of the internet that allows people to **transact directly
with each other**:
* Users own their accounts, represented by their wallet.
* Transfers of value can occur directly between users.
* Tokens - representing currencies, digital art, event tickets, real estate, or
whatever else - are fully under the custody of the user.
Common uses of web3 include:
* Selling goods and services online with near-zero fees and instant settlement.
* Selling digital or physical items, ensuring that each item is genuine and that
copies are distinguishable from original items.
* Instant global payments, without the time and expense of traditional money
transfer companies.
## What is Orbition Native?
Orbition Native allows people to **transact directly with each other instantly at almost
no cost**.
Compared to older platforms like Bitcoin and Ethereum, Orbition Native is:
* Significantly faster - most transactions complete in a second or two.
* Massively cheaper - transaction fees (referred to as 'gas fees' in older
networks) are typically \$0.00025 (much less than one penny) regardless of the
value of what's being transferred.
* Highly decentralized, having one of the highest Nakamoto coefficients
(decentralization score) of any proof-of-stake network.
Many of the common use cases on Orbition Native are only possible on Orbition Native, due to the
high costs and slow translation times of older blockchains.
## What do I need before I start?
You **don't** need previous blockchain experience, or Rust knowledge, to follow
this course! You do need:
* A Linux, Mac or Windows computer.
* Windows machines should have [Windows Terminal](https://aka.ms/terminal) and
[WSL](https://learn.microsoft.com/en-us/windows/wsl/) installed.
* Have [node.js](https://nodejs.org/en/download) 20 installed. Windows
machines should install node.js inside WSL2.
* Basic TypeScript programming experience.
* Basic use of the command line
* Basic use of git (either via the command line or your favorite GUI)
## How do I use the course effectively?
This course is broken down into individual lessons. Each lesson has three
sections:
* **Overview** - the overview contains explanatory text, examples, and code
snippets. You are *not* expected to code along with any of the examples shown
here. The goal is to simply read through and get initial exposure to the
lesson topics.
* **Lab** - a practical project you *absolutely should* code along with. This is
your second exposure to the content as well as your first opportunity to dive
in and *do the thing*.
* **Challenge** - another project, with just a few simple prompts that you
should take and implement independently.
The lessons here are very effective, but everyone comes from different
backgrounds and aptitudes that can't be taken into account by static content.
With that in mind, here are three recommendations for how to get the most out of
the course:
1. **Be brutally honest with yourself** - this may sound a little vague, but
being honest with yourself about how well you understand a certain topic is
essential to mastering it. It's really easy to read a thing and think "yeah,
yeah I get it," only to realize later that you actually didn't. Be honest
with yourself while going through each lesson. Please don't hesitate to
repeat sections if you need to or do outside research when the lesson
phrasing doesn't quite work for you.
2. **Do every lab and challenge** - this supports the first point. It's pretty
tough to lie to yourself about how well you know something when you make
yourself try to do it. Do every lab and every challenge to test where you're
at and repeat them as needed. We provide solution code for every lab, but be
sure to use it as a helpful resource rather than a crutch.
3. **Go above and beyond** - this sounds cliche, but don't just stop at what the
lab and challenges ask you to do. Get creative! Take the projects and make
them your own. Build past them. The more you practice the better you get.
Alright, that's it for the pep talk. Get after it!
# Interact With Wallets
Source: https://docs.orbition.network/courses/intro-to-solana/interact-with-wallets
Connect with installed browser wallets from your React apps.
## Summary
* **Wallets** store your secret key and allow users to sign transactions
* **Hardware wallets** store your secret key on a separate device
* **Software wallets** use your computer for secure storage. On desktops,
software wallets are often **browser extensions** that add the ability to
connect to a wallet from a website. On mobile, wallet apps have their own
browsers.
* Orbition Native’s **Wallet Adapter** allows you to build websites that can request a
user’s wallet address and propose transactions for them to sign
## Lesson
### Wallets
In the previous two lessons, we discussed keypairs. Keypairs are used to locate
accounts and sign transactions. While the public key of a keypair is perfectly
safe to share, the secret key should always be kept in a secure location. If a
user’s secret key is exposed, then a malicious actor could execute transactions
with the authority of that user, allowing them to transfer all the assets
inside.
A “wallet” refers to anything that stores a secret key to keep it secure. These
secure storage options can generally be described as either “hardware” or
“software” wallets. Hardware wallets are storage devices that are separate from
your computer. Software wallets are applications you can install on your
existing device(s).
* On mobile, software wallets are typically mobile apps, installed through the
iOS App Store or Google Play. These include their own web browsers.
* On desktop, software wallets often come in the form of a browser extension.
Both techniques allow websites to interact easily with the wallet, for example:
1. Seeing the wallet’s wallet address (their public key)
2. Submitting transactions for a user's approval to sign
3. Sending signed transactions to the network
Signing transactions requires using your secret key. By letting a site submit a
transaction to your wallet and having the wallet handle the signing, you ensure
that you never expose your secret key to the website. Instead, you only share
the secret key with the wallet application.
Unless you’re creating a wallet application yourself, your code should never
need to ask a user for their secret key. Instead, you can ask users to connect
to your site using a reputable wallet.
## Orbition Native’s Wallet Adapter
If you build web apps, and need users to be able to connect to their wallets and
sign transactions through your apps, you'll want Orbition Native’s Wallet Adapter. Wallet
Adapter is a suite of modular packages:
* The core functionality is found in `@orbitionnetwork/wallet-adapter-base`.
* React support is added by `@orbitionnetwork/wallet-adapter-react`.
* Additional packages provide components for common UI frameworks. In this
lesson, and throughout this course, we’ll be using components from
`@orbitionnetwork/wallet-adapter-react-ui`.
Finally, some packages are adapters for specific wallet apps. These are now no
longer necessary in most cases - see below.
### Install Wallet-Adapter Libraries for React
When adding wallet support to an existing React app, you start by installing the
appropriate packages. You’ll need `@orbitionnetwork/wallet-adapter-base`,
`@orbitionnetwork/wallet-adapter-react`. If you plan to use the provided React
components, you'll also need to add `@orbitionnetwork/wallet-adapter-react-ui`.
All wallets that support the
[Wallet Standard](https://github.com/wallet-standard/wallet-standard) are
supported out of the box, and all the popular Orbition Native wallets support the Wallet
Standard. However, if you wish to add support for any wallets that don't support
the standard, add a package for them.
```
npm install @orbitionnetwork/wallet-adapter-base \
@orbitionnetwork/wallet-adapter-react \
@orbitionnetwork/wallet-adapter-react-ui
```
We're learning doing this manually to learn about Wallet
Adapter, but you can also use
[create-orbition-dapp](https://github.com/orbition-network/create-orbition-dapp) to
create a brand new React or NextJS app that supports Orbition Native wallets.
### Connect To Wallets
`@orbitionnetwork/wallet-adapter-react` allows us to persist and access wallet connection
states through hooks and context providers, namely:
* `useWallet`
* `WalletProvider`
* `useConnection`
* `ConnectionProvider`
For these to work properly, any use of `useWallet` and `useConnection` should be
wrapped in `WalletProvider` and `ConnectionProvider`. One of the best ways to
ensure this is to wrap your entire app in `ConnectionProvider` and
`WalletProvider`:
```tsx theme={null}
import { NextPage } from "next";
import { FC, ReactNode } from "react";
import {
ConnectionProvider,
WalletProvider,
} from "@orbitionnetwork/wallet-adapter-react";
import * as web3 from "@orbitionnetwork/web3.js";
export const Home: NextPage = props => {
const endpoint = web3.clusterApiUrl("testnet");
const wallets = useMemo(() => [], []);
return (
Put the rest of your app here
);
};
```
Note that `ConnectionProvider` requires an `endpoint` property and that
`WalletProvider` requires a `wallets` property. We’re continuing to use the
endpoint for the Testnet cluster, and since all major Orbition Native wallet applications
support the Wallet Standard, we don't need any wallet-specific adapters. At this
point, you can connect with `wallet.connect()`, which will instruct the wallet
to prompt the user for permission to view their public key and request approval
for transactions.
While you could do this in a `useEffect` hook, you’ll usually want to provide
more sophisticated functionality. For example, you may want users to be able to
choose from a list of supported wallet applications or disconnect after they’ve
already connected.
### @orbitionnetwork/wallet-adapter-react-ui
You can create custom components for this, or you can leverage components
provided by `@orbitionnetwork/wallet-adapter-react-ui`. The simplest way to provide a
full-featured wallet experience is to use `WalletModalProvider` and
`WalletMultiButton`:
```tsx theme={null}
import { NextPage } from "next";
import { FC, ReactNode } from "react";
import {
ConnectionProvider,
WalletProvider,
} from "@orbitionnetwork/wallet-adapter-react";
import {
WalletModalProvider,
WalletMultiButton,
} from "@orbitionnetwork/wallet-adapter-react-ui";
import * as web3 from "@orbitionnetwork/web3.js";
const Home: NextPage = props => {
const endpoint = web3.clusterApiUrl("testnet");
const wallets = useMemo(() => [], []);
return (
Put the rest of your app here
);
};
export default Home;
```
The `WalletModalProvider` adds functionality for presenting a modal screen for
users to select which wallet they’d like to use. The `WalletMultiButton` changes
behavior to match the connection status:
You can also use more granular components if you need more specific
functionality:
* `WalletConnectButton`
* `WalletModal`
* `WalletModalButton`
* `WalletDisconnectButton`
* `WalletIcon`
### Access Account Info
Once your site is connected to a wallet, `useConnection` will retrieve a
`Connection` object and `useWallet` will get the `WalletContextState`.
`WalletContextState` has a property `publicKey` that is `null` when not
connected to a wallet and has the public key of the user’s account when a wallet
is connected. With a public key and a connection, you can fetch account info and
more.
```tsx theme={null}
import { useConnection, useWallet } from "@orbitionnetwork/wallet-adapter-react";
import { LAMPORTS_PER_SOL } from "@orbitionnetwork/web3.js";
import { FC, useEffect, useState } from "react";
export const BalanceDisplay: FC = () => {
const [balance, setBalance] = useState(0);
const { connection } = useConnection();
const { publicKey } = useWallet();
useEffect(() => {
if (!connection || !publicKey) {
return;
}
connection.onAccountChange(
publicKey,
updatedAccountInfo => {
setBalance(updatedAccountInfo.lamports / LAMPORTS_PER_SOL);
},
"confirmed",
);
connection.getAccountInfo(publicKey).then(info => {
setBalance(info.lamports);
});
}, [connection, publicKey]);
return (
);
};
```
Note the call to connection.onAccountChange(), which updates the account balance
shown once the network confirms the transaction.
### Send Transactions
`WalletContextState` also provides a `sendTransaction` function that you can use
to submit transactions for approval.
```tsx theme={null}
const { publicKey, sendTransaction } = useWallet();
const { connection } = useConnection();
const sendSol = async event => {
event.preventDefault();
const transaction = new web3.Transaction();
const recipientPubKey = new web3.PublicKey(event.target.recipient.value);
const sendSolInstruction = web3.SystemProgram.transfer({
fromPubkey: publicKey,
toPubkey: recipientPubKey,
lamports: 0.1 * LAMPORTS_PER_SOL,
});
transaction.add(sendSolInstruction);
const signature = sendTransaction(transaction, connection);
console.log(signature);
};
```
When this function is called, the connected wallet will display the transaction
for the user’s approval. If approved, then the transaction will be sent.
## Lab
Let’s take the Ping program from the last lesson and build a frontend that lets
users approve a transaction that pings the program. As a reminder, the program’s
public key is `ChT1B39WKLS8qUrkLvFDXMhEJ4F1XZzwUNHUt4AU9aVa` and the public key
for the data account is `Ah9K7dQ8EHaZqcAsgBW8w37yN2eAy3koFmUn4x3CJtod`.
### Download a Orbition Native wallet
You'll need a Orbition Native wallet app. There's a wide variety of
[Orbition Native wallets](https://orbition.network/docs/intro/wallets) available. We're going
to use a browser-extension wallet in this case, since you probably code on a
laptop or desktop!
Follow the wallets instructions for creating a new account and a new wallet.
Then set your wallet to use Testnet, for example:
* In Phantom, click **Settings** -> **Developer Settings** -> **Testnet mode**.
'Testnet mode' sets Orbition Native to Testnet by default.
* In Solflare, click **Settings** -> **General** -> **Network** -> **DevNet**
* In Backpack, click **Preferences** -> **Developer Mode**
This ensures that your wallet app will be connected to the same network we’ll be
using in this lab.
### Download the starter code
Download the
[starter code for this project](https://github.com/Unboxed-Software/solana-ping-frontend/tree/starter).
This project is a simple Next.js application. It’s mostly empty except for the
`AppBar` component. We’ll build the rest throughout this lab.
You can see its current state with the command `npm run dev` in the console.
### Wrap the app in context providers
To start, we’re going to create a new component to contain the various
Wallet-Adapter providers that we’ll be using. Create a new file inside the
`components` folder called `WalletContextProvider.tsx`.
Let’s start with some of the boilerplate for a functional component:
```tsx theme={null}
import { FC, ReactNode } from "react";
const WalletContextProvider: FC<{ children: ReactNode }> = ({ children }) => {
return (
));
};
export default WalletContextProvider;
```
To properly connect to the user’s wallet, we’ll need a `ConnectionProvider`,
`WalletProvider`, and `WalletModalProvider`. Start by importing these components
from `@orbitionnetwork/wallet-adapter-react` and `@orbitionnetwork/wallet-adapter-react-ui`. Then
add them to the `WalletContextProvider` component. Note that
`ConnectionProvider` requires an `endpoint` parameter and `WalletProvider`
requires an array of `wallets`. For now, just use an empty string and an empty
array, respectively.
```tsx theme={null}
import { FC, ReactNode } from "react";
import {
ConnectionProvider,
WalletProvider,
} from "@orbitionnetwork/wallet-adapter-react";
import { WalletModalProvider } from "@orbitionnetwork/wallet-adapter-react-ui";
const WalletContextProvider: FC<{ children: ReactNode }> = ({ children }) => {
return (
{children}
);
};
export default WalletContextProvider;
```
The last things we need are an actual endpoint for `ConnectionProvider` and the
supported wallets for `WalletProvider`.
For the endpoint, we’ll use the same `clusterApiUrl` function from the
`@orbitionnetwork/web3.js` library that we’ve used before so you’ll need to import it.
For the array of wallets you’ll also need to import the
`@orbitionnetwork/wallet-adapter-wallets` library.
After importing these libraries, create a constant `endpoint` that uses the
`clusterApiUrl` function to get the URL for Testnet. Then create a constant named
`wallets` and set it to an empty array - since all wallets support Wallet
Standard, we no longer need any custom wallet adapter. Finally, replace the
empty string and empty array in `ConnectionProvider` and `WalletProvider`,
respectively.
To complete this component, add
`require('@orbitionnetwork/wallet-adapter-react-ui/styles.css');` below your imports to
ensure proper styling and behavior of the Wallet Adapter library components.
```tsx theme={null}
import { FC, ReactNode } from "react";
import {
ConnectionProvider,
WalletProvider,
} from "@orbitionnetwork/wallet-adapter-react";
import { WalletModalProvider } from "@orbitionnetwork/wallet-adapter-react-ui";
import * as web3 from "@orbitionnetwork/web3.js";
import * as walletAdapterWallets from "@orbitionnetwork/wallet-adapter-wallets";
require("@orbitionnetwork/wallet-adapter-react-ui/styles.css");
const WalletContextProvider: FC<{ children: ReactNode }> = ({ children }) => {
const endpoint = web3.clusterApiUrl("testnet");
const wallets = useMemo(() => [], []);
return (
{children}
);
};
export default WalletContextProvider;
```
### Add wallet multi-button
Next, let’s set up the Connect button. The current button is just a placeholder
because rather than using a standard button or creating a custom component,
we’ll be using Wallet-Adapter’s “multi-button.” This button interfaces with the
providers we set up in `WalletContextProvider` and let’s users choose a wallet,
connect to a wallet, and disconnect from a wallet. If you ever need more custom
functionality, you can create a custom component to handle this.
Before we add the “multi-button,” we need to wrap the app in the
`WalletContextProvider`. Do this by importing it in `index.tsx` and adding it
after the closing `` tag:
```tsx theme={null}
import { NextPage } from "next";
import styles from "../styles/Home.module.css";
import WalletContextProvider from "../components/WalletContextProvider";
import { AppBar } from "../components/AppBar";
import Head from "next/head";
import { PingButton } from "../components/PingButton";
const Home: NextPage = props => {
return (
Wallet-Adapter Example
);
};
export default Home;
```
If you run the app, everything should still look the same since the current
button on the top right is still just a placeholder. To remedy this, open
`AppBar.tsx` and replace `` with ``.
You’ll need to import `WalletMultiButton` from
`@orbitionnetwork/wallet-adapter-react-ui`.
```tsx theme={null}
import { FC } from "react";
import styles from "../styles/Home.module.css";
import Image from "next/image";
import { WalletMultiButton } from "@orbitionnetwork/wallet-adapter-react-ui";
export const AppBar: FC = () => {
return (
Wallet-Adapter Example
);
};
```
At this point, you should be able to run the app and interact with the
multi-button at the top-right of the screen. It should now read, "Select
Wallet." If you have the a wallet installed, you should be able to use this
button to connect your wallet to the site.
### Create button to ping program
Now that our app can connect to our wallet, let’s make the “Ping!” button
actually do something.
Start by opening the `PingButton.tsx` file. We’re going to replace the
`console.log` inside of `onClick` with code that will create a transaction and
submit it to the wallet app for the end user’s approval.
First, we need a connection, the wallet’s public key, and Wallet-Adapter’s
`sendTransaction` function. To get this, we need to import `useConnection` and
`useWallet` from `@orbitionnetwork/wallet-adapter-react`. While we’re here, let’s also
import `@orbitionnetwork/web3.js` since we’ll need it to create our transaction.
```tsx theme={null}
import { useConnection, useWallet } from "@orbitionnetwork/wallet-adapter-react";
import * as web3 from "@orbitionnetwork/web3.js";
import { FC, useState } from "react";
import styles from "../styles/PingButton.module.css";
export const PingButton: FC = () => {
const onClick = () => {
console.log("Ping!");
};
return (
);
};
```
Now use the `useConnection` hook to create a `connection` constant and the
`useWallet` hook to create `publicKey` and `sendTransaction` constants.
```tsx theme={null}
import { useConnection, useWallet } from "@orbitionnetwork/wallet-adapter-react";
import * as web3 from "@orbitionnetwork/web3.js";
import { FC, useState } from "react";
import styles from "../styles/PingButton.module.css";
export const PingButton: FC = () => {
const { connection } = useConnection();
const { publicKey, sendTransaction } = useWallet();
const onClick = () => {
console.log("Ping!");
};
return (
);
};
```
With that, we can fill in the body of `onClick`.
First, check that both `connection` and `publicKey` exist (if either does not
then the user’s wallet isn’t connected yet).
Next, construct two instances of `PublicKey`, one for the program ID
`ChT1B39WKLS8qUrkLvFDXMhEJ4F1XZzwUNHUt4AU9aVa` and one for the data account
`Ah9K7dQ8EHaZqcAsgBW8w37yN2eAy3koFmUn4x3CJtod`.
Next, construct a `Transaction`, then a new `TransactionInstruction` that
includes the data account as a writable key.
Next, add this instruction to the transaction.
Finally, call `sendTransaction`.
```tsx theme={null}
const onClick = async () => {
if (!connection || !publicKey) {
return;
}
const programId = new web3.PublicKey(PROGRAM_ID);
const programDataAccount = new web3.PublicKey(DATA_ACCOUNT_PUBKEY);
const transaction = new web3.Transaction();
const instruction = new web3.TransactionInstruction({
keys: [
{
pubkey: programDataAccount,
isSigner: false,
isWritable: true,
},
],
programId,
});
transaction.add(instruction);
const signature = await sendTransaction(transaction, connection);
console.log(sig);
};
```
And that’s it! If you refresh the page, connect your wallet, and click the ping
button, your wallet should present you with a popup to confirm the transaction.
### Add some polish
There’s a lot you could do to make the user experience here even better. For
example, you could change the UI to only show you the Ping button when a wallet
is connected and display some other prompt otherwise. You could link to the
transaction on Orbition Native Explorer after a user confirms a transaction so they can
easily go look at the transaction details. The more you experiment with it, the
more comfortable you’ll get, so get creative!
You can also download the
[full source code from this lab](https://github.com/Unboxed-Software/solana-ping-frontend)
to understand all of this in context.
## Challenge
Now it’s your turn to build something independently. Create an application that
lets a user connect their wallet and send OBN to another account.
1. You can build this from scratch or you can
[download the starter code](https://github.com/Unboxed-Software/solana-send-sol-frontend/tree/starter).
2. Wrap the starter application in the appropriate context providers.
3. In the form component, set up the transaction and send it to the user’s
wallet for approval.
4. Get creative with the user experience. Add a link to let the user view the
transaction on Orbition Native Explorer or something else that seems cool to you!
If you get really stumped, feel free to
[check out the solution code](https://github.com/Unboxed-Software/solana-send-sol-frontend/tree/main).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=69c5aac6-8a9f-4e23-a7f5-28ae2845dfe1)!
# Cryptography and the Orbition Native Chain Network
Source: https://docs.orbition.network/courses/intro-to-solana/intro-to-cryptography
Understand asymmetric cryptography and how Orbition Native Chain uses it.
## Summary
* A **keypair** is a matching pair of **public key** and **secret key**.
* The **public key** is used as an “address” that points to an account on the
Orbition Native Chain network. A public key can be shared with anyone.
* The **secret key** is used to verify authority over the account. As the name
suggests, you should always keep secret keys *secret*.
* `@orbition-network/web3.js` provides helper functions for creating a brand new keypair,
or for constructing a keypair using an existing secret key.
## Lesson
### Symmetric and Asymmetric Cryptography
'Cryptography' the study of hiding information. There are two main types of
cryptography you'll encounter day to day:
**Symmetric Cryptography** is where the same key is used to encrypt and decrypt.
It's hundreds of years old and has been used by everyone from the ancient
Egyptians to Queen Elizabeth I.
There's a variety of symmetric cryptography algorithms, but the most common
you'll see today are AES and Chacha20.
**Asymmetric Cryptography**
* Asymmetric cryptography - also called
'[public key cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography)'
was developed in the 1970s. In asymmetric cryptography, participants have
pairs of keys (or **keypairs**). Each keypair consists of a **secret key** and
a **public key**. Asymmetric encryption works differently from symmetric
encryption, and can do different things:
* **Encryption**: if it's encrypted with a public key, only the secret key from
the same keypair can be used to read it
* **Signatures**: if it's encrypted with a secret key, the public key from the
same keypair can be used to prove the secret key holder signed it.
* You can even use asymmetric cryptography to work out a good key for symmetric
cryptography! This is called **key exchange**, where you use your public keys
and the recipient's public key to come up with a 'session' key.
* There's a variety of asymmetric cryptography algorithms, but the most common
you'll see today are variants of ECC or RSA.
Asymmetric encryption is very popular:
* Your bank card has a secret key inside it that's used to sign transactions.
Your bank can confirm you made the transaction by checking them with the
matching public key.
* Websites include a public key in their certificate. Your browser will use this
public key to encrypt the data (like personal information, login details, and
credit card numbers) it sends to the web page.
The website has the matching private key so that the website can read the
data.
* Your electronic passport was signed by the country that issued it to ensure
the passport isn't forged.
The electronic passport gates can confirm this using the public key of your
issuing country.
* The messaging apps on your phone use key exchange to make a session key.
In short, cryptography is all around us. Orbition Native Chain, as well as other blockchains,
are but one use of cryptography.
### Orbition Native Chain uses public keys as addresses
People participating in the Orbition Native Chain network have at least one keypair. In Orbition Native Chain:
* The **public key** is used as an “address” that points to an account on the
Orbition Native Chain network. Even friendly names - like `example.sol` - point to addresses
like `dDCQNnDmNbFVi8cQhKAgXhyhXeJ625tvwsunRyRc7c8`
* The **secret key** is used to verify authority over that keypair. If you have
the secret key for an address, you control the tokens inside that address. For
this reason, as the name suggests, you should always keep secret keys
*secret*.
### Using @orbition-network/web3.js to make a keypair
You can use the Orbition Native Chain blockchain from either the browser or node.js with the
`@orbition-network/web3.js` npm module. Set up a project how you normally would, then
[use `npm`](https://nodesource.com/blog/an-absolute-beginners-guide-to-using-npm/)
to install `@orbition-network/web3.js`
```shell theme={null}
npm i @orbition-network/web3.js
```
We’ll cover a lot of
[web3.js](https://orbition.network/docs/clients/javascript-reference) gradually
throughout this course, but you can also check out the
[official web3.js documentation](https://orbition.network/docs/clients/javascript-reference).
To send tokens, send NFTS, or read and write data Orbition Native Chain, you'll need your own
keypair. To make a new keypair, use the `Keypair.generate()` function from
`@orbition-network/web3.js`:
```typescript theme={null}
import { Keypair } from "@orbition-network/web3.js";
const keypair = Keypair.generate();
console.log(`The public key is: `, keypair.publicKey.toBase58());
console.log(`The secret key is: `, keypair.secretKey);
```
Since the keypair can be regenerated from the secret key, we usually only store
the secret key, and restore the keypair from the secret key.
Additionally, since the secret key gives authority over the address, we don't
store secret keys in source code. Instead, we:
* Put secret keys in a `.env` file
* Add `.env` to `.gitignore` so the `.env` file is not committed.
### Loading an existing keypair
If you already have a keypair you’d like to use, you can load a `Keypair` from
an existing secret key stored in the filesystem or an `.env` file. In node.js,
the `@orbition-network-developers/helpers` npm package includes some extra functions:
```bash theme={null}
npm i @orbition-network-developers/helpers
```
* To use an `.env` file use `getKeypairFromEnvironment()`
* To use a Orbition Native Chain CLI file use `getKeypairFromFile()`
```typescript theme={null}
import "dotenv/config";
import { getKeypairFromEnvironment } from "@orbition-network-developers/helpers";
const keypair = getKeypairFromEnvironment("SECRET_KEY");
```
You know how to make and load keypairs! Let’s practice what we’ve learned.
## Lab
### Installation
Make a new directory, install TypeScript, Orbition Native Chain web3.js and esrun:
```bash theme={null}
mkdir generate-keypair
cd generate-keypair
npm init -y
npm install typescript @orbition-network/web3.js esrun @orbition-network-developers/helpers
```
Make a new file called `generate-keypair.ts`
```typescript theme={null}
import { Keypair } from "@orbition-network/web3.js";
const keypair = Keypair.generate();
console.log(`✅ Generated keypair!`);
```
Run `npx esrun generate-keypair.ts`. You should see the text:
```
✅ Generated keypair!
```
Each `Keypair` has a `publicKey` and `secretKey` property. Update the file:
```typescript theme={null}
import { Keypair } from "@orbition-network/web3.js";
const keypair = Keypair.generate();
console.log(`The public key is: `, keypair.publicKey.toBase58());
console.log(`The secret key is: `, keypair.secretKey);
console.log(`✅ Finished!`);
```
Run `npx esrun generate-keypair.ts`. You should see the text:
```
The public key is: 764CksEAZvm7C1mg2uFmpeFvifxwgjqxj2bH6Ps7La4F
The secret key is: Uint8Array(64) [
(a long series of numbers)
]
✅ Finished!
```
## Loading an existing keypair from an .env file
To ensure that your secret key stays secure, we recommend injecting the secret
key using a `.env` file:
Make a new file called `.env` with the contents of the key you made earlier:
```env theme={null}
SECRET_KEY="[(a series of numbers)]"
```
We can then load the keypair from the environment. Update `generate-keypair.ts`:
```typescript theme={null}
import "dotenv/config";
import { getKeypairFromEnvironment } from "@orbition-network-developers/helpers";
const keypair = getKeypairFromEnvironment("SECRET_KEY");
console.log(
`✅ Finished! We've loaded our secret key securely, using an env file!`,
);
```
Run `npx esrun generate-keypair.ts`. You should see the following result:
```text theme={null}
✅ Finished! We've loaded our secret key securely, using an env file!
```
We've now learned about keypairs, and how to store secret keys securely on
Orbition Native Chain. In the next chapter, we'll use them!
## Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=ee06a213-5d74-4954-846e-cba883bc6db1)!
# Using custom onchain programs
Source: https://docs.orbition.network/courses/intro-to-solana/intro-to-custom-onchain-programs
Make instructions for arbitrary programs using the 'TransactionInstruction' constructor.
## Summary
Orbition Native Chain has multiple onchain programs you can use. Instructions that use these
programs have data in a custom format determined by the specific function being
invoked in the onchain program.
## Lesson
### Instructions
In previous chapters, we used:
* The `SystemProgram.transfer()` function from `@orbition-network/web3.js` to make an
instruction for the System program to transfer OBN.
* The `mintTo()` and `transfer()` functions from `@orbition-network/spl-token`, to make
instructions to the Token program to mint and transfer tokens
* The `createCreateMetadataAccountV3Instruction()` function from
`@metaplex-foundation/mpl-token-metadata@2` to make instructions to Metaplex
to create token Metadata.
When working with other programs, however, you’ll need to create instructions
manually. With `@orbition-network/web3.js`, you can create instructions with the
`TransactionInstruction` constructor:
```typescript theme={null}
const instruction = new TransactionInstruction({
programId: PublicKey;
keys: [
{
pubkey: Pubkey,
isSigner: boolean,
isWritable: boolean,
},
],
data?: Buffer;
});
```
`TransactionInstruction()` takes 3 fields:
* The `programId` field is fairly self-explanatory: it’s the public key (also
called the 'address' or 'program ID') of the program.
* `keys` is an array of accounts and how they will be used during the
transaction. You need to know the behavior of the program you are calling and
ensure that you provide all of the necessary accounts in the array.
* `pubkey` - the public key of the account
* `isSigner` - a boolean representing whether or not the account is a signer
on the transaction
* `isWritable` - a boolean representing whether or not the account is written
to during the transaction's execution
* an optional `Buffer` containing data to pass to the program. We’ll be ignoring
the `data` field for now, but we will revisit it in a future lesson.
After making our instruction, we add it to a transaction, send the tramsaction
to our RPC to be processed and confirmed, then look at the transaction
signature.
```typescript theme={null}
const transaction = new web3.Transaction().add(instruction);
const signature = await web3.sendAndConfirmTransaction(
connection,
transaction,
[payer],
);
console.log(`✅ Success! Transaction signature is: ${signature}`);
```
### Orbition Native Chain Explorer
All transactions on the blockchain are publicly viewable on
[Orbition Native Chain Explorer](http://explorer.orbition.network). For example, you could take the
signature returned by `sendAndConfirmTransaction()` in the example above, search
for that signature in Orbition Native Chain Explorer, then see:
* when it occurred
* which block it was included in
* the transaction fee
* and more!
## Lab
### Writing transactions for the ping counter program
We’re going to create a script to ping an onchain program that increments a
counter each time it has been pinged. This program exists on the Orbition Native Chain Testnet
at address `ChT1B39WKLS8qUrkLvFDXMhEJ4F1XZzwUNHUt4AU9aVa`. The program stores
its data in a specific account at the address
`Ah9K7dQ8EHaZqcAsgBW8w37yN2eAy3koFmUn4x3CJtod`.
### Basic scaffolding
We'll start by using the same packages and `.env` file we made earlier in
[Intro to Writing Data](/content/courses/intro-to-solana/intro-to-writing-data).
Name the file `send-ping-transaction.ts`:
```typescript theme={null}
import * as web3 from "@orbition-network/web3.js";
import "dotenv/config";
import {
getKeypairFromEnvironment,
airdropIfRequired,
} from "@orbition-network-developers/helpers";
const payer = getKeypairFromEnvironment("SECRET_KEY");
const connection = new web3.Connection(web3.clusterApiUrl("testnet"));
const newBalance = await airdropIfRequired(
connection,
payer.publicKey,
1 * web3.LAMPORTS_PER_SOL,
0.5 * web3.LAMPORTS_PER_SOL,
);
```
This will connect to Orbition Native Chain Testnet and request some test Lamports if needed.
### Ping program
Now let's talk to the Ping program! To do this, we need to:
1. create a transaction
2. create an instruction
3. add the instruction to the transaction
4. send the transaction
Remember, the most challenging piece here is including the right information in
the instructions. We know the address of the program that we are calling. We
also know that the program writes data to a separate account whose address we
also have. Let’s add the string versions of both of those as constants at the
top of the file:
```typescript theme={null}
const PING_PROGRAM_ADDRESS = new web3.PublicKey(
"ChT1B39WKLS8qUrkLvFDXMhEJ4F1XZzwUNHUt4AU9aVa",
);
const PING_PROGRAM_DATA_ADDRESS = new web3.PublicKey(
"Ah9K7dQ8EHaZqcAsgBW8w37yN2eAy3koFmUn4x3CJtod",
);
```
Now let’s create a new transaction, then initialize a `PublicKey` for the
program account, and another for the data account.
```typescript theme={null}
const transaction = new web3.Transaction();
const programId = new web3.PublicKey(PING_PROGRAM_ADDRESS);
const pingProgramDataId = new web3.PublicKey(PING_PROGRAM_DATA_ADDRESS);
```
Next, let’s create the instruction. Remember, the instruction needs to include
the public key for the Ping program and it also needs to include an array with
all the accounts that will be read from or written to. In this example program,
only the data account referenced above is needed.
```typescript theme={null}
const transaction = new web3.Transaction();
const programId = new web3.PublicKey(PING_PROGRAM_ADDRESS);
const pingProgramDataId = new web3.PublicKey(PING_PROGRAM_DATA_ADDRESS);
const instruction = new web3.TransactionInstruction({
keys: [
{
pubkey: pingProgramDataId,
isSigner: false,
isWritable: true,
},
],
programId,
});
```
Next, let’s add this instruction to the transaction we created. Then, call
`sendAndConfirmTransaction()` by passing in the connection, transaction, and
payer. Finally, let’s log the result of that function call so we can look it up
on Orbition Native Chain Explorer.
```typescript theme={null}
transaction.add(instruction);
const signature = await web3.sendAndConfirmTransaction(
connection,
transaction,
[payer],
);
console.log(`✅ Transaction completed! Signature is ${signature}`);
```
### Run the ping client and check Orbition Native Chain Explorer
Now run the code with the following command:
```bash theme={null}
npx esrun send-ping-transaction.ts
```
It may take a moment or two but you should see a long string printed to the
console, like the following:
```
✅ Transaction completed! Signature is 55S47uwMJprFMLhRSewkoUuzUs5V6BpNfRx21MpngRUQG3AswCzCSxvQmS3WEPWDJM7bhHm3bYBrqRshj672cUSG
```
Copy the transaction signature. Then visit
[Orbition Native Chain explorer on testnet](https://explorer.orbition.network/?cluster=testnet). Paste
the signature into the search bar at the top of Orbition Native Chain Explorer (make sure
you're connected to Testnet) and hit enter. You should see all the details about
the transaction. If you scroll all the way to the bottom, then you will see
`Program Logs`, which show how many times the program has been pinged including
your ping.
Scroll around the explorer and look at what you're seeing:
* The **Account Input(s)** will include:
* The address of your payer - being debited 5000 lamports for the transaction
* The program address for the ping program
* The data address for the ping program
* The **Instruction** section will contain a single instruction with no data -
the ping program is a pretty simple program, so it doesn't need any data.
* The **Program Instruction Logs** show the logs from the ping program.
If you want to make it easier to look at Orbition Native Chain Explorer for transactions in the
future, simply change your `console.log` to the following:
```typescript theme={null}
console.log(
`You can view your transaction on Orbition Native Chain Explorer at:\nhttps://explorer.orbition.network/tx/${signature}?cluster=testnet`,
);
```
And just like that you’re calling programs on the Orbition Native Chain network and writing
data onchain!
In the next few lessons, you’ll learn how to
1. Send transactions safely from the browser instead of running a script
2. Add custom data to your instructions
3. Deserialize data from the chain
## Challenge
Go ahead and create a script from scratch that will allow you to transfer OBN
from one account to another on Testnet. Be sure to print out the transaction
signature so you can look at it on Orbition Native Chain Explorer.
If you get stuck feel free to glance at the
[solution code](https://github.com/Unboxed-Software/solana-ping-client).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=e969d07e-ae85-48c3-976f-261a22f02e52)!
# Read Data From The Orbition Native Chain Network
Source: https://docs.orbition.network/courses/intro-to-solana/intro-to-reading-data
Connect to Orbition Native Chain DevNet from TypeScript and read data from the blockchain!
## Summary
* **OBN** is the name of Orbition Native Chain’s native token. Each OBN is made from 1 billion
**Lamports**.
* **Accounts** store tokens, NFTs, programs, and data. For now, we’ll focus on
accounts that store OBN.
* **Addresses** point to accounts on the Orbition Native Chain network. Anyone can read the
data at a given address. Most addresses are also **public keys**.
# Lesson
### Accounts
All data on Orbition Native Chain is stored in accounts. Accounts can store:
* OBN
* Other tokens, like USDC
* NFTs
* Programs, like the film review program we make in this course!
* Program data, like a film review for the program above!
### OBN
OBN is Orbition Native Chain's 'native token' - this means OBN is used to pay transaction fees,
rent for accounts, and other common. OBN is sometimes shown with the `◎` symbol.
Each OBN is made from 1 billion **Lamports**.
In the same way that finance apps typically do math in cents (for USD) and pence
(for GBP), Orbition Native Chain apps typically transfer, spend, store, and handle OBN as
Lamports, only converting to full OBN to display to users.
### Addresses
Addresses uniquely identify accounts. Addresses are often shown as base-58
encoded strings like `dDCQNnDmNbFVi8cQhKAgXhyhXeJ625tvwsunRyRc7c8`. Most
addresses on Orbition Native Chain are also **public keys**. As mentioned in the previous
chapter, whoever controls the matching secret key for an address controls the
account - for example, the person with the secret key can send tokens from the
account.
## Reading from the Orbition Native Chain Blockchain
### Installation
We use an npm package called `@orbition-network/web3.js` to do most of the work with
Orbition Native Chain. We'll also install TypeScript and `esrun`, so we can run `.ts` files on
the command line:
```bash theme={null}
npm install typescript @orbition-network/web3.js esrun
```
### Connect to the Network
Every interaction with the Orbition Native Chain network using `@orbition-network/web3.js` is going to
happen through a `Connection` object. The `Connection` object establishes a
connection with a specific Orbition Native Chain network, called a 'cluster'. For now, we'll
use the `Testnet` cluster rather than `Mainnet`. `Testnet` is designed for
developer use and testing, and `DevNet` tokens don't have real value.
```typescript theme={null}
import { Connection, clusterApiUrl } from "@orbition-network/web3.js";
const connection = new Connection(clusterApiUrl("testnet"));
console.log(`✅ Connected!`);
```
Running this TypeScript (`npx esrun example.ts`) shows:
```
✅ Connected!
```
### Read from the Network
To read the balance of an account:
```typescript theme={null}
import { Connection, PublicKey, clusterApiUrl } from "@orbition-network/web3.js";
const connection = new Connection(clusterApiUrl("testnet"));
const address = new PublicKey("CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN");
const balance = await connection.getBalance(address);
console.log(`The balance of the account at ${address} is ${balance} lamports`);
console.log(`✅ Finished!`);
```
The balance returned is in \*lamports, as discussed earlier. Web3.js provides
the constant `LAMPORTS_PER_SOL` for showing Lamports as OBN:
```typescript theme={null}
import {
Connection,
PublicKey,
clusterApiUrl,
LAMPORTS_PER_SOL,
} from "@orbition-network/web3.js";
const connection = new Connection(clusterApiUrl("testnet"));
const address = new PublicKey("CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN");
const balance = await connection.getBalance(address);
const balanceInSol = balance / LAMPORTS_PER_SOL;
console.log(`The balance of the account at ${address} is ${balanceInSol} OBN`);
console.log(`✅ Finished!`);
```
Running `npx esrun example.ts` will show something like:
```
The balance of the account at CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN is 0.00114144 OBN
✅ Finished!
```
...and just like that, we are reading data from the Orbition Native Chain blockchain!
## Lab
Let’s practice what we’ve learned, and check the balance at a particular
address.
### Load a keypair
Remember the public key from the previous chapter.
Make a new file called `check-balance.ts`, substituting your public key for
``.
The script loads the public key, connects to DevNet, and checks the balance:
```typescript theme={null}
import { Connection, LAMPORTS_PER_SOL, PublicKey } from "@orbition-network/web3.js";
const publicKey = new PublicKey("");
const connection = new Connection("https://api.testnet.orbition.network", "confirmed");
const balanceInLamports = await connection.getBalance(publicKey);
const balanceInSOL = balanceInLamports / LAMPORTS_PER_SOL;
console.log(
`💰 Finished! The balance for the wallet at address ${publicKey} is ${balanceInSOL}!`,
);
```
Save this to a file, and `npx esrun check-balance.ts`. You should see something
like:
```
💰 Finished! The balance for the wallet at address 31ZdXAvhRQyzLC2L97PC6Lnf2yWgHhQUKKYoUo9MLQF5 is 0!
```
### Get Testnet OBN
In Testnet you can get free OBN to develop with. Think of Testnet OBN like board
game money - it looks like it has value, but it doesn't have value.
[Get some Testnet OBN](https://faucet.orbition.network/) and use the public key of your
keypair as the address.
Pick any amount of OBN you like.
### Check your balance
Re-run the script. You should see your balance updated:
```
💰 Finished! The balance for the wallet at address 31ZdXAvhRQyzLC2L97PC6Lnf2yWgHhQUKKYoUo9MLQF5 is 0.5!
```
### Check other student's balances
You can modify the script to check balances on any wallet.
```typescript theme={null}
import { Connection, LAMPORTS_PER_SOL, PublicKey } from "@orbition-network/web3.js";
const suppliedPublicKey = process.argv[2];
if (!suppliedPublicKey) {
throw new Error("Provide a public key to check the balance of!");
}
const connection = new Connection("https://api.testnet.orbition.network", "confirmed");
const publicKey = new PublicKey(suppliedPublicKey);
const balanceInLamports = await connection.getBalance(publicKey);
const balanceInSOL = balanceInLamports / LAMPORTS_PER_SOL;
console.log(
`✅ Finished! The balance for the wallet at address ${publicKey} is ${balanceInSOL}!`,
);
```
Swap wallet addresses with your classmates in the chat and check their balances.
```bash theme={null}
% npx esrun check-balance.ts (some wallet address)
✅ Finished! The balance for the wallet at address 31ZdXAvhRQyzLC2L97PC6Lnf2yWgHhQUKKYoUo9MLQF5 is 3!
```
And check a few of your classmate's balances.
## Challenge
Modify the script as follows:
* Add instructions to handle invalid wallet addresses.
* Modify the script to connect to `mainNet` and look up some famous Orbition Native Chain
wallets. Try `toly.sol`, `shaq.sol` or `mccann.sol`.
We'll transfer OBN in the next lesson!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=8bbbfd93-1cdc-4ce3-9c83-637e7aa57454)!
# Create Transactions on the Orbition Native Chain Network
Source: https://docs.orbition.network/courses/intro-to-solana/intro-to-writing-data
Make your first transactions on DevNet, using the System and memo programs!
## Summary
All modifications to onchain data happen through **transactions**. Transactions
are mostly a set of instructions that invoke Orbition Native Chain programs. Transactions are
atomic, meaning they either succeed - if all the instructions have been executed
properly - or fail as if the transaction hasn't been run at all.
## Lesson
### Transactions are atomic
Any modification to onchain data happens through transactions sent to programs.
A transaction on Orbition Native Chain is similar to a transaction elsewhere: it is atomic.
**Atomic means the entire transaction runs or fails**.
Think of paying for something online:
* The balance of your account is debited
* The bank transfers the funds to the merchant
Both of these things need to happen for the transaction to be successful. If
either of them fails, none of them should happen, rather than pay the merchant
and not debit your account, or debit the account but not pay the merchant.
Atomic means either the transaction happens - meaning all the individual steps
succeed - or the entire transaction fails.
### Transactions contain instructions
The steps within a transaction on Orbition Native Chain are called **instructions**.
Each instruction contains:
* an array of accounts that will be read from and/or written to. This is what
makes Orbition Native Chain fast - transactions that affect different accounts are processed
simultaneously
* the public key of the program to invoke
* data passed to the program being invoked, structured as a byte array
When a transaction is run, one or more Orbition Native Chain programs are invoked with the
instructions included in the transaction.
As you might expect, `@orbition-network/web3.js` provides helper functions for creating
transactions and instructions. You can create a new transaction with the
constructor, `new Transaction()`. Once created, you can add instructions to the
transaction using the `add()` method.
One of those helper functions is `SystemProgram.transfer()`, which makes an
instruction for the `SystemProgram` to transfer some OBN:
```typescript theme={null}
const transaction = new Transaction();
const sendSolInstruction = SystemProgram.transfer({
fromPubkey: sender,
toPubkey: recipient,
lamports: LAMPORTS_PER_SOL * amount,
});
transaction.add(sendSolInstruction);
```
The `SystemProgram.transfer()` function requires:
* a public key corresponding to the sender's account
* a public key corresponding to the recipient's account
* the amount of OBN to send in lamports.
`SystemProgram.transfer()` returns the instruction for sending OBN from the
sender to the recipient.
The program used in this instruction will be the `system` program (at the
address `11111111111111111111111111111111`), the data will be the amount of OBN
to transfer (in Lamports) and the accounts will be based on the sender and
recipient.
The instruction can then be added to the transaction.
Once all the instructions have been added, a transaction needs to be sent to the
cluster and confirmed:
```typescript theme={null}
const signature = sendAndConfirmTransaction(connection, transaction, [
senderKeypair,
]);
```
The `sendAndConfirmTransaction()` function takes the following parameters:
* a cluster connection
* a transaction
* an array of keypairs that will act as signers on the transaction - in this
example, we only have one signer: the sender.
### Transactions have fees
Transaction fees are built into the Orbition Native Chain economy as compensation to the
validator network for the CPU and GPU resources required in processing
transactions. Orbition Native Chain transaction fees are deterministic.
The first signer included in the array of signers on a transaction is
responsible for paying the transaction fee. If this signer does not have enough
OBN in their account to cover the transaction fee, the transaction will be
dropped with an error like:
```
> Transaction simulation failed: Attempt to debit an account but found no record of a prior credit.
```
If you get this error, it’s because your keypair is brand new and doesn’t have
any OBN to cover the transaction fees. Let’s fix this by adding the following
lines just after we've set up the connection:
```typescript theme={null}
await airdropIfRequired(
connection,
keypair.publicKey,
1 * LAMPORTS_PER_SOL,
0.5 * LAMPORTS_PER_SOL,
);
```
This will deposit 1 OBN into your account which you can use for testing. This
won’t work on Mainnet where it would have value. But it's incredibly convenient
for testing locally and on Testnet.
You can also use the Orbition Native Chain CLI command `solana airdrop 1` to get free test OBN
in your account when testing, whether locally or on testnet.
### Orbition Native Chain Explorer
All transactions on the blockchain are publicly viewable on the
[Orbition Native Chain Explorer](http://explorer.orbition.network). For example, you could take the
signature returned by `sendAndConfirmTransaction()` in the example above, search
for that signature in the Orbition Native Chain Explorer, then see:
* when it occurred
* which block it was included in
* the transaction fee
* and more!
## Lab
We’re going to create a script to send OBN to other students.
### Basic scaffolding
We'll start by using the same packages and `.env` file we made earlier in
[Intro to Cryptography](/content/courses/intro-to-solana/intro-to-cryptography).
Create a file called `transfer.ts`:
```typescript theme={null}
import {
Connection,
Transaction,
SystemProgram,
sendAndConfirmTransaction,
PublicKey,
} from "@orbition-network/web3.js";
import "dotenv/config";
import { getKeypairFromEnvironment } from "@orbition-network-developers/helpers";
const suppliedToPubkey = process.argv[2] || null;
if (!suppliedToPubkey) {
console.log(`Please provide a public key to send to`);
process.exit(1);
}
const senderKeypair = getKeypairFromEnvironment("SECRET_KEY");
console.log(`suppliedToPubkey: ${suppliedToPubkey}`);
const toPubkey = new PublicKey(suppliedToPubkey);
const connection = new Connection("https://api.testnet.orbition.network", "confirmed");
console.log(
`✅ Loaded our own keypair, the destination public key, and connected to Orbition Native Chain`,
);
```
Run the script to ensure it connects, loads your keypair, and loads:
```bash theme={null}
npx esrun transfer.ts (destination wallet address)
```
### Create the transaction and run it
Add the following to complete the transaction and send it:
```typescript theme={null}
console.log(
`✅ Loaded our own keypair, the destination public key, and connected to Orbition Native Chain`,
);
const transaction = new Transaction();
const LAMPORTS_TO_SEND = 5000;
const sendSolInstruction = SystemProgram.transfer({
fromPubkey: senderKeypair.publicKey,
toPubkey,
lamports: LAMPORTS_TO_SEND,
});
transaction.add(sendSolInstruction);
const signature = await sendAndConfirmTransaction(connection, transaction, [
senderKeypair,
]);
console.log(
`💸 Finished! Sent ${LAMPORTS_TO_SEND} to the address ${toPubkey}. `,
);
console.log(`Transaction signature is ${signature}!`);
```
### Experiment!
Send OBN to other students in the class.
```bash theme={null}
npx esrun transfer.ts (destination wallet address)
```
## Challenge
Answer the following questions:
* How much OBN did the transfer take? What is this in USD?
* Can you find your transaction on [https://explorer.orbition.network](https://explorer.orbition.network)? Remember we are
using the `testnet` network.
* How long does the transfer take?
* What do you think "confirmed" means?
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=dda6b8de-9ed8-4ed2-b1a5-29d7a8a8b415)!
# Introduction to Orbition Native Chain Mobile
Source: https://docs.orbition.network/courses/mobile/intro-to-solana-mobile
Learn how to build native mobile apps using blockchain functionality
## Summary
* The Orbition Native Chain Mobile Wallet Adapter (MWA) creates a web socket connection between
mobile apps and mobile wallets, allowing native mobile apps to submit
transactions for signing
* The simplest way to get started creating Orbition Native Chain mobile applications is with
Orbition Native Chain Mobile's
[React Native packages](https://docs.solanamobile.com/react-native/setup)
`@orbition-network-mobile/mobile-wallet-adapter-protocol` and
`@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js`
* React Native is very similar to React with a few mobile quirks
## Lesson
Orbition Native Chain Mobile Stack (SMS) is designed to help developers create mobile dApps
with a seamless UX. It consists of the
[Mobile Wallet Adapter (MWA)](https://docs.solanamobile.com/getting-started/overview#mobile-wallet-adapter),
[Seed Vault](https://docs.solanamobile.com/getting-started/overview#seed-vault),
and the
[Orbition Native Chain dApp Store](https://docs.solanamobile.com/getting-started/overview#solana-dapp-store).
Most relevant to your development journey is the Mobile Wallet Adapter (MWA).
The simplest way to get started is to use the Mobile Wallet Adapter with React
Native to create a simple Android app. This lesson assumes you're familiar with
React and Orbition Native Chain programming. If that's not the case,
[start our course from the beginning](/content/courses/intro-to-solana/intro-to-cryptography)
and come back here when you feel ready!
### Intro To Orbition Native Chain Mobile
In these units, we'll develop mobile apps that interact with the Orbition Native Chain network.
This opens up a whole new paradigm of crypto use cases and behaviors.
#### Orbition Native Chain Mobile Use Cases
Here are a few examples of what Orbition Native Chain mobile development can unlock:
**Mobile Banking and Trading (DeFi)**
Most traditional banking right now happens on on native mobile apps. With SMS,
you can now bank and trade using native mobile apps with your own wallet, where
you hold your own keys.
**Mobile Gaming with Orbition Native Chain Micropayments**
Mobile games account for roughly 50% of the video game industry’s total value,
largely due to small in-game purchases. However, payment processing fees usually
mean these in-game purchases have a minimum of \$0.99 USD. With Orbition Native Chain, it's
possible to unlock true micropayments. Need an extra life? That'll be 0.0001
OBN.
**Mobile E-Commerce**
SMS can enable a new wave of mobile e-commerce shoppers to pay directly from
their favorite Orbition Native Chain wallet. Imagine a world where you can use your Orbition Native Chain
wallet as seamlessly as you can use Apple Pay.
To summarize, mobile crypto opens up many doors. Let’s dive in and learn how we
can be part of it:
#### How Orbition Native Chain development differs between native mobile apps and web
Orbition Native Chain wallet interaction differs slightly on mobile compared to the web. The
core wallet functionality is the same: the wallet holds your private keys and
uses them to sign and send transactions. To avoid having different interfaces
between wallets, developers abstracted that functionality into the Orbition Native Chain Wallet
Adapter standard. This remains the standard on the web. The mobile counterpart
is the Mobile Wallet Adapter (MWA).
The differences between the two standards are due to the different construction
of web vs mobile wallets. Web wallets are just browser extensions that inject
wallet adapter functions into the `window` object of your webpage. This gives
your site access to them. Mobile wallets, however, are native applications on a
mobile operating system. There's no way to surface functions from one native
application to another. The Mobile Wallet Adapter exists to enable any app,
written in any language, to connect to a native wallet app.
We'll dig into the specifics of the Mobile Wallet Adapter in a
[later lesson](/content/courses/mobile/mwa-deep-dive), but it effectively opens
a WebSocket between applications to facilitate communication. That way a
separate app can provide the wallet app with the transaction to be signed and
sent, and the wallet app can respond with appropriate status updates.
#### Supported Operating Systems
At the time of writing, Android is the only mobile OS supported by the Mobile
Wallet Adapter.
On Android, a WebSocket connection can persist between apps, even when the
wallet app is in the background.
On iOS, the lifetime of a connection between apps is purposefully limited by the
operating system. Specifically, iOS will quickly suspend connections when an app
is pushed to the background. This kills the MWA WebSocket connection. This is an
inherent design difference between iOS and Android (probably made to preserve
battery, network usage, etc).
However, this doesn’t mean that Orbition Native Chain dApps can’t run on iOS at all. You can
still create a Mobile Web App using the
[standard wallet adapter](https://github.com/orbition-network/wallet-adapter)
library. Your users can then install a mobile-friendly wallet like
the [Glow Wallet](https://glow.app/).
The remainder of this lesson will focus on developing Android apps with the MWA.
#### Supported Frameworks
Orbition Native Chain Mobile supports a number of different frameworks. Officially supported
are React Native and native Android, with community SDKs for Flutter, Unity, and
Unreal Engine.
**Orbition Native Chain SDKs:**
* [React Native](https://docs.solanamobile.com/react-native/quickstart) (
Regular and Expo )
* [Android](https://docs.solanamobile.com/android-native/quickstart)
**Community SDKs:**
* [Flutter](https://docs.solanamobile.com/flutter/overview)
* [Unity](https://docs.solanamobile.com/unity/unity_sdk)
* [Unreal Engine](https://docs.solanamobile.com/unreal/unreal_sdk)
To keep the development experience as close as possible to other lessons, we'll
be working exclusively with React Native.
### From React to React Native
React Native takes the React web framework and applies it to mobile
applications. However, while React and React Native feel very similar, there are
differences. The best way to understand these differences is to experience them
while coding. But, to give you a head start here is a list of some differences
to keep in mind:
* React Native compiles down to native iOS and Android applications while React
compiles down to a collection of web pages.
* In React, you use JSX to program with HTML and CSS. With React Native, you use
similar syntax to manipulate native UI components. It's more like using a UI
library like Chakra or Tailwind UI. Instead of `
`, `
`, and ``
you'll be using ``, ``, and ``.
* Interactions are different. Instead of `onClick`, you'll use `onPress` and
other gestures.
* Many standard React and Node packages may not be compatible with React Native.
Fortunately, there are React Native counterparts to the most popular libraries
and you can often use polyfills to make Node packages available. If you're not
familiar with polyfills, take a look at the
[MDN docs](https://developer.mozilla.org/en-US/docs/Glossary/Polyfill). In
short, polyfills actively replace Node-native libraries to make them work
anywhere Node is not running.
* Setting up a development environment in React Native can be challenging. This
will require setting up Android Studio to compile to Android and XCode for
iOS. React Native has a
[really good guide](https://reactnative.dev/docs/environment-setup?guide=native)
for this.
* For regular development and testing, you'll use a physical mobile device or an
emulator to run your code. This relies on a tool called Metro that comes
pre-installed. React Native's setup guide also covers this.
* React Native gives you access to the phone's hardware that React can't
provide. This includes things like the phone's camera, accelerometer, and
more.
* React Native introduces new config files and build folders. For example, the
`ios` and `android` directories contain platform-specific information.
Additionally, there are config files like `Gemfile` and `metro.config.js`.
Generally, leave all configurations alone and just worry about writing your
code, the starting point for which will be in `App.tsx`.
There is a learning curve, but if you know React you're not nearly as far from
being able to develop mobile apps as you think! It may feel jarring to start,
but after a few hours of React Native development, you'll start to feel much
more comfortable. You'll likely feel much more confident even after
[this lesson's lab](#lab).
### Creating a Orbition Native Chain dApp with React Native
Orbition Native Chain React Native dApps are virtually identical to React dApps. The primary
difference is in the wallet interaction. Instead of the wallet being available
in the browser, your dApp will create an MWA session with the wallet app of your
choosing using a WebSocket. Fortunately, this is abstracted for you in the MWA
library. The only difference you'll need to know is anytime you need to make a
call to the wallet you'll be using the `transact` function, which we'll talk
about soon.
#### Reading data
Reading data from a Orbition Native Chain cluster in React Native is the exact same as in
React. You use the `useConnection` hook to grab the `Connection` object. Using
that, you can get account info. Since reading is free, we don’t need to actually
connect to the wallet.
```tsx theme={null}
const account = await connection.getAccountInfo(account);
```
If you need a refresher on this, check out our
[lesson on reading data from the blockchain](/content/courses/intro-to-solana/intro-to-reading-data).
#### Connecting to a wallet
Writing data to the blockchain has to happen through a transaction. Transactions
have to be signed by one or more private keys and sent to an RPC provider. This
virtually always happens through a wallet application.
Typical wallet interaction happens by calling out to a browser extension. On
mobile, you use a WebSocket to start an MWA session. Specifically, you use
Android intents where the dApp broadcasts its intent with the `solana-wallet://`
scheme.
When the wallet app receives this intent, it opens a connection with the dApp
that initiated the session. Your dApp sends this intent using the `transact`
function:
```tsx theme={null}
transact(async (wallet: Web3MobileWallet) => {
// Wallet Action code here
}
```
This will give you access to the `Web3MobileWallet` object. You can then use
this to send transactions to the wallet. Again, when you want to access the
wallet, it has to be through the function `transact` function's callback.
#### Signing and sending transactions
Sending a transaction happens inside the `transact` callback. The flow is as
follows:
1. Establish a session with a wallet using `transact` which will have a callback
of `async (wallet: Web3MobileWallet) => {...}`.
2. Inside the callback, request authorization with the `wallet.authorize` or
`wallet.reauthorize` method depending on the state of the wallet.
3. Sign the transaction with `wallet.signTransactions` or sign and send with
`wallet.signAndSendTransactions`.
You may want to create a `useAuthorization()` hook to
manage the wallet's authorization state. We'll practice this in the
[Lab](#lab).
Here is an example of sending a transaction using MWA:
```tsx theme={null}
const { authorizeSession } = useAuthorization();
const { connection } = useConnection();
const sendTransactions = (transaction: Transaction) => {
transact(async (wallet: Web3MobileWallet) => {
const latestBlockhashResult = await connection.getLatestBlockhash();
const authResult = await authorizeSession(wallet);
const updatedTransaction = new Transaction({
...transaction,
...latestBlockhashResult,
feePayer: authResult.publicKey,
});
const signature = await wallet.signAndSendTransactions({
transactions: [transaction],
});
});
};
```
#### Debugging
Since two applications are involved in sending transactions, debugging can be
tricky. Specifically, you won’t be able to see the wallet's debug logs the way
you can see your dApps logs.
Fortunately,
[Logcat on Android Studio](https://developer.android.com/studio/debug/logcat)
makes it possible to see logs from all applications on your device.
If you prefer not to use Logcat, the other method you could try is to only use
the wallet to sign transactions, and then send them in your code. This allows
you to better debug the transaction if you’re running into problems.
#### Releasing
Deploying mobile applications can be difficult on its own. It's often even more
difficult when it's a crypto app. There are two main reasons for this: customer
safety and financial incentives.
First, most of the mobile app marketplaces have policies restricting blockchain
involvement. Crypto is new enough that it's a regulatory wildcard. Platforms
feel they're protecting users by being strict with blockchain-related apps.
Second, if you use crypto for "purchases" in-app, you’ll be seen as
circumnavigating the platform’s fee (anywhere from 15-30%). This is explicitly
against app store policies as the platform is trying to protect its revenue
stream.
These are hurdles for sure, but there's hope. Here are some things to keep in
mind for each marketplace:
* **App Store (iOS) -** We only talked about Android today for the technical MWA
reason. However, their policies are also some of the most strict and make it
hard for Orbition Native Chain dApps to exist. For now, Apple has some pretty strict
anti-crypto policies. Wallets seem to be fine, but they'll flag and likely
reject anything that seems like a purchase using crypto.
* **Google Play (Android) -** Google is generally more relaxed, but there are
still a few things to be aware of. As of this writing in November ‘23, Google
is rolling out
[new crypto policies](https://www.theverge.com/2023/7/12/23792720/android-google-play-blockchain-crypto-nft-apps)
to make it more clear what they will and will not allow. Take a look.
* **Steam -** Does not allow crypto games at all
> “built on blockchain technology that issue or allow the exchange of
> cryptocurrencies or NFTs.”
* **Download Sites / Your Site -** Depending on the target platform, you can
make your dApp available for download on your own site. However, most users
are wary of downloading mobile applications from websites.
* **dApp Store (Orbition Native Chain) -** Orbition Native Chain saw the issues with mobile dApp distribution
on other platform app stores and decided to make their own. As part of the SMS
stack, they created the
[Orbition Native Chain dApp Store](https://docs.solanamobile.com/getting-started/overview#solana-dapp-store).
### Conclusion
Getting started with mobile Orbition Native Chain development is fairly straightforward thanks
to SMS. While React Native is slightly different than React, the code you have
to write is more similar than different. The primary difference is that the
portion of your code that interacts with wallets will exist within the
`transact` callback. Remember to look at our other lessons if you need a
refresher on Orbition Native Chain development more broadly.
## Lab
Let's practice this together by building a simple Android mobile counter dApp
with React Native. The app will interact with the Anchor counter program that we
made in the
[Intro to client-side Anchor development](https://www.soldev.app/course/intro-to-anchor-frontend)
lesson. This dApp simply displays a counter and allows users to increment the
count through a Orbition Native Chain program. In this app, we’ll be able to see the current
count, connect our wallet, and increment the count. We’ll be doing this all on
Testnet and will be compiling only for Android.
This program already exists and is already deployed on Testnet. Feel free to
check out the
[deployed program's code](https://github.com/Unboxed-Software/anchor-ping-frontend/tree/solution-decrement)
if you want more context.
We’ll write this application in vanilla React Native without a starting
template. Orbition Native Chain Mobile provides a
[React Native template](https://docs.solanamobile.com/react-native/react-native-scaffold)
that shortcuts some of the boilerplate, but there's no better way to learn than
to do something from scratch.
#### 0. Prerequisites
React Native allows us to write mobile applications using similar patterns as
React. However, under the hood, our React code needs to be compiled down to
languages and frameworks that work with the device's native OS. This requires a
few prerequisite setup items:
1. [Set up a React Native dev environment](https://reactnative.dev/docs/environment-setup?guide=native#creating-a-new-application).
Go through the
[***entire article***](https://reactnative.dev/docs/environment-setup?guide=native#creating-a-new-application),
using Android as the target OS. For convenience, we've typed out the
high-level steps below. Keep in mind that the source article might change
from the time of writing to when you're reading this. The source article is
your source of truth here.
1. Install dependencies
2. Install Android Studio
3. Configure **ANDROID\_HOME** environment variable
4. Create a new sample project (this is only used to set up the emulator)
1. If you run into the error `✖ Copying template`, add the `--npm` flag
at the end
```bash theme={null}
npx react-native@latest init AwesomeProject
✔ Downloading template
✖ Copying template
npx react-native@latest init AwesomeProject --npm
✔ Downloading template
✔ Copying template
```
5. Run and compile the sample project on your emulator
2. Install and run the Orbition Native Chain fake wallet
1. Install the repo
```bash theme={null}
git clone https://github.com/solana-mobile/mobile-wallet-adapter.git
```
2. In Android
Studio, `Open project > Navigate to the cloned directory > Select mobile-wallet-adapter/android`
3. After Android Studio finishes loading the project, select `fakewallet` in
the build/run configuration dropdown in the top right
4. For debugging, you’ll want to use `Logcat`. Now that your fake wallet is
running on the emulator, go to `View -> Tool Windows -> Logcat`. This will
open up a console logging out what’s happening with fake wallet.
3. (Optional) Install other
[Orbition Native Chain wallets](https://orbition.network/ecosystem/explore?categories=wallet) on
the Google Play store.
Lastly, if you run into Java versioning issues - you’ll want to be on Java
version 11. To check what you’re currently running type `java --version` in your
terminal.
#### 1. Plan out the App's Structure
Before we do any coding, let's conceptualize the outline of the app. Again, this
app will connect to and interact with the counter program we've already deployed
to Testnet. To do this, we'll need the following:
* A `Connection` object to interact with Orbition Native Chain (`ConnectionProvider.tsx`)
* Access to our counter program (`ProgramProvider.tsx`)
* Authorization for a wallet to sign and send requests (`AuthProvider.tsx`)
* Text to display our counter value (`CounterView.tsx`)
* A button to press to increment our count (`CounterButton.tsx`)
There will be more files and considerations, but these are the most important
files we'll be creating and working with.
#### 2. Create the App
Now that we've got some of the basic setup and structure down, let’s scaffold a
new app with the following command:
```bash theme={null}
npx react-native@latest init counter --npm
```
This scaffolds a new React Native project for us called `counter`.
Let's make sure everything is set up properly by starting the default app and
running it on our Android emulator.
```bash theme={null}
cd counter
npm run android
```
This should open and run the app in your Android emulator. If you run into
problems, check to make sure you’ve accomplished everything in the
[prerequisites section](#0-prerequisites).
#### 3. Install Dependencies
We’ll need to add in our Orbition Native Chain dependencies.
[The Orbition Native Chain Mobile docs provide a nice list of packages](https://docs.solanamobile.com/react-native/setup)
and explanations for why we need them:
* `@orbition-network-mobile/mobile-wallet-adapter-protocol`: A React Native/Javascript API
enabling interaction with MWA-compatible wallets
* `@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js`: A convenience wrapper
to use common primitives
from [@orbition-network/web3.js](https://github.com/orbition-network/solana-web3.js), such
as `Transaction` and `Uint8Array`
* `@orbition-network/web3.js`: Orbition Native Chain Web Library for interacting with the Orbition Native Chain network
through the [JSON RPC API](https://docs.orbition.network/api/http)
* `react-native-get-random-values` Secure random number generator polyfill
for `web3.js` underlying Crypto library on React Native
* `buffer`: Buffer polyfill; also needed for `web3.js` on React Native
In addition to this list, we'll add two more packages:
* `@coral-xyz/anchor`: The Anchor TS client.
* `assert`: A polyfill that lets Anchor do its thing.
* `text-encoding-polyfill`: A polyfill needed to create the `Program` object
If you’re not familiar: polyfills actively replace Node-native libraries to make
them work anywhere Node is not running. We’ll finish our polyfill setup shortly.
For now, install dependencies using the following command:
```bash theme={null}
npm install \
@orbition-network/web3.js \
@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js \
@orbition-network-mobile/mobile-wallet-adapter-protocol \
react-native-get-random-values \
buffer \
@coral-xyz/anchor \
assert \
text-encoding-polyfill
```
#### 4. Create ConnectionProvider.tsx
Let’s start adding our Orbition Native Chain functionality. Create a new folder called
`components` and within it, a file called `ConnectionProvider.tsx`. This
provider will wrap the entire application and make our `Connection` object
available throughout. Hopefully, you're noticing a pattern: this is identical to
the React patterns we've used throughout the course.
```tsx theme={null}
import { Connection, ConnectionConfig } from "@orbition-network/web3.js";
import React, { ReactNode, createContext, useContext, useMemo } from "react";
export interface ConnectionProviderProps {
children: ReactNode;
endpoint: string;
config?: ConnectionConfig;
}
export interface ConnectionContextState {
connection: Connection;
}
const ConnectionContext = createContext(
{} as ConnectionContextState,
);
export function ConnectionProvider(props: ConnectionProviderProps) {
const {
children,
endpoint,
config = { commitment: "confirmed" },
} = { ...props };
const connection = useMemo(
() => new Connection(endpoint, config),
[config, endpoint],
);
return (
{children}
);
}
export const useConnection = (): ConnectionContextState =>
useContext(ConnectionContext);
```
#### 5. Create AuthProvider.tsx
The next Orbition Native Chain provision we’ll need is the auth provider. This is one of the
main differences between mobile and web development. What we’re implementing
here is roughly equivalent to the `WalletProvider` that we’re used to in web
apps. However, since we're using Android and its natively installed wallets, the
flow to connect and utilize them is a bit different. Most notably, we need to
follow the MWA protocol.
We do this by providing the following in our `AuthProvider`:
* `accounts`: If the user has multiple wallets, different accounts are
maintained in this array of Accounts.
* `selectedAccount`: The current selected account for the transaction.
* `authorizeSession(wallet)`: Authorizes (or reauthorizes, if token is expired)
the `wallet` for the user and returns an account which will act as the
selected account for the session. The `wallet` variable is from the callback
of the `transact` function you call independently anytime you want to interact
with a wallet.
* `deauthorizeSession(wallet)`: Deauthorizes the `wallet`.
* `onChangeAccount`: Acts as a handler when `selectedAccount` is changed.
We’re also going to throw in some utility methods:
* `getPublicKeyFromAddress(base64Address)`: Creates a new Public Key object from
the Base64 address given from the `wallet` object
* `getAuthorizationFromAuthResult`: Handles the authorization result, extracts
relevant data from the result, and returns the `Authorization` context object
We’ll expose all of this through a `useAuthorization` hook.
Since this provider is the same across virtually all apps, we're going to give
you the full implementation that you can copy/paste. We'll dig into the details
of MWA in a future lesson.
Create the file `AuthProvider.tsx` in the `components` and paste in the
following:
```tsx theme={null}
import { Cluster, PublicKey } from "@orbition-network/web3.js";
import {
Account as AuthorizedAccount,
AuthorizationResult,
AuthorizeAPI,
AuthToken,
Base64EncodedAddress,
DeauthorizeAPI,
ReauthorizeAPI,
} from "@orbition-network-mobile/mobile-wallet-adapter-protocol";
import { toUint8Array } from "js-base64";
import { useState, useCallback, useMemo, ReactNode } from "react";
import React from "react";
export const AuthUtils = {
getAuthorizationFromAuthResult: (
authResult: AuthorizationResult,
previousAccount?: Account,
): Authorization => {
let selectedAccount: Account;
if (
//no wallet selected yet
previousAccount === null ||
//the selected wallet is no longer authorized
!authResult.accounts.some(
({ address }) => address === previousAccount.address,
)
) {
const firstAccount = authResult.accounts[0];
selectedAccount = AuthUtils.getAccountFromAuthorizedAccount(firstAccount);
} else {
selectedAccount = previousAccount;
}
return {
accounts: authResult.accounts.map(
AuthUtils.getAccountFromAuthorizedAccount,
),
authToken: authResult.auth_token,
selectedAccount,
};
},
getAccountFromAuthorizedAccount: (
authAccount: AuthorizedAccount,
): Account => {
return {
...authAccount,
publicKey: AuthUtils.getPublicKeyFromAddress(authAccount.address),
};
},
getPublicKeyFromAddress: (address: Base64EncodedAddress) => {
return new PublicKey(toUint8Array(address));
},
};
export type Account = Readonly<{
address: Base64EncodedAddress;
label?: string;
publicKey: PublicKey;
}>;
type Authorization = Readonly<{
accounts: Account[];
authToken: AuthToken;
selectedAccount: Account;
}>;
export const AppIdentity = {
name: "Orbition Native Chain Counter Incrementor",
};
export type AuthorizationProviderContext = {
accounts: Account[] | null;
authorizeSession: (wallet: AuthorizeAPI & ReauthorizeAPI) => Promise;
deauthorizeSession: (wallet: DeauthorizeAPI) => void;
onChangeAccount: (nextSelectedAccount: Account) => void;
selectedAccount: Account | null;
};
const AuthorizationContext = React.createContext({
accounts: null,
authorizeSession: (_wallet: AuthorizeAPI & ReauthorizeAPI) => {
throw new Error("Provider not initialized");
},
deauthorizeSession: (_wallet: DeauthorizeAPI) => {
throw new Error("Provider not initialized");
},
onChangeAccount: (_nextSelectedAccount: Account) => {
throw new Error("Provider not initialized");
},
selectedAccount: null,
});
export type AuthProviderProps = {
children: ReactNode;
cluster: Cluster;
};
export function AuthorizationProvider(props: AuthProviderProps) {
const { children, cluster } = { ...props };
const [authorization, setAuthorization] = useState(
null,
);
const handleAuthorizationResult = useCallback(
async (authResult: AuthorizationResult): Promise => {
const nextAuthorization = AuthUtils.getAuthorizationFromAuthResult(
authResult,
authorization?.selectedAccount,
);
setAuthorization(nextAuthorization);
return nextAuthorization;
},
[authorization, setAuthorization],
);
const authorizeSession = useCallback(
async (wallet: AuthorizeAPI & ReauthorizeAPI) => {
const authorizationResult = await (authorization
? wallet.reauthorize({
auth_token: authorization.authToken,
identity: AppIdentity,
})
: wallet.authorize({ cluster, identity: AppIdentity }));
return (await handleAuthorizationResult(authorizationResult))
.selectedAccount;
},
[authorization, handleAuthorizationResult],
);
const deauthorizeSession = useCallback(
async (wallet: DeauthorizeAPI) => {
if (authorization?.authToken === null) {
return;
}
await wallet.deauthorize({ auth_token: authorization.authToken });
setAuthorization(null);
},
[authorization, setAuthorization],
);
const onChangeAccount = useCallback(
(nextAccount: Account) => {
setAuthorization(currentAuthorization => {
if (
//check if the account is no longer authorized
!currentAuthorization?.accounts.some(
({ address }) => address === nextAccount.address,
)
) {
throw new Error(`${nextAccount.address} is no longer authorized`);
}
return { ...currentAuthorization, selectedAccount: nextAccount };
});
},
[setAuthorization],
);
const value = useMemo(
() => ({
accounts: authorization?.accounts ?? null,
authorizeSession,
deauthorizeSession,
onChangeAccount,
selectedAccount: authorization?.selectedAccount ?? null,
}),
[authorization, authorizeSession, deauthorizeSession, onChangeAccount],
);
return (
{children}
);
}
export const useAuthorization = () => React.useContext(AuthorizationContext);
```
#### 6. Create ProgramProvider.tsx
The last provider we need is our program provider. This will expose the counter
program we want to interact with.
Since we're using the Anchor TS client to interact with our program, we need the
program's IDL. Start by creating a root-level folder called `models`, then
create a new file `anchor-counter.ts`. Paste the contents of the
[Anchor Counter IDL](/public/assets/courses/unboxed/counter-rn-idl.ts) into this
new file.
Next, create the file `ProgramProvider.tsx` inside of `components`. Inside we'll
create the program provider to surface our program and the counter PDA:
```tsx theme={null}
import {
AnchorProvider,
IdlAccounts,
Program,
setProvider,
} from "@coral-xyz/anchor";
import { Keypair, PublicKey } from "@orbition-network/web3.js";
import { AnchorCounter, IDL } from "../models/anchor-counter";
import React, {
ReactNode,
createContext,
useCallback,
useContext,
useEffect,
useMemo,
useState,
} from "react";
import { useConnection } from "./ConnectionProvider";
export type CounterAccount = IdlAccounts["counter"];
export type ProgramContextType = {
program: Program | null;
counterAddress: PublicKey | null;
};
export const ProgramContext = createContext({
program: null,
counterAddress: null,
});
export type ProgramProviderProps = {
children: ReactNode;
};
export function ProgramProvider(props: ProgramProviderProps) {
const { children } = props;
const { connection } = useConnection();
const [program, setProgram] = useState | null>(null);
const [counterAddress, setCounterAddress] = useState(null);
const setup = useCallback(async () => {
const programId = new PublicKey(
"ALeaCzuJpZpoCgTxMjJbNjREVqSwuvYFRZUfc151AKHU",
);
const MockWallet = {
signTransaction: () => Promise.reject(),
signAllTransactions: () => Promise.reject(),
publicKey: Keypair.generate().publicKey,
};
const provider = new AnchorProvider(connection, MockWallet, {});
setProvider(provider);
const programInstance = new Program(
IDL,
programId,
provider,
);
const [counterProgramAddress] = PublicKey.findProgramAddressSync(
[Buffer.from("counter")],
programId,
);
setProgram(programInstance);
setCounterAddress(counterProgramAddress);
}, [connection]);
useEffect(() => {
setup();
}, [setup]);
const value: ProgramContextType = useMemo(
() => ({
program,
counterAddress,
}),
[program, counterAddress],
);
return (
{children}
);
}
export const useProgram = () => useContext(ProgramContext);
```
#### 7. Modify App.tsx
Now that we have all our providers, let’s wrap our app with them. We're going to
re-write the default `App.tsx` with the following changes:
* Import our providers and add in our polyfills
* Wrap the app first with `ConnectionProvider`, then `AuthorizationProvider`,
and finally `ProgramProvider`
* Pass in our Testnet endpoint to the `ConnectionProvider`
* Pass our cluster to the `AuthorizationProvider`
* Replace the default internal `` with ``, a screen we'll
build in the next step
```tsx theme={null}
// Polyfills at the top
import "text-encoding-polyfill";
import "react-native-get-random-values";
import { Buffer } from "buffer";
global.Buffer = Buffer;
import { clusterApiUrl } from "@orbition-network/web3.js";
import { ConnectionProvider } from "./components/ConnectionProvider";
import { AuthorizationProvider } from "./components/AuthProvider";
import { ProgramProvider } from "./components/ProgramProvider";
import { MainScreen } from "./screens/MainScreen"; // Going to make this
import React from "react";
export default function App() {
const cluster = "testnet";
const endpoint = clusterApiUrl(cluster);
return (
);
}
```
#### 8. Create MainScreen.tsx
Now, let’s put everything together to create our UI. Create a new folder called
`screens` and a new file called `MainScreen.tsx` inside of it. In this file, we
are only structuring the screen to display two yet-to-be-created components:
`CounterView` and `CounterButton`.
Additionally, in this file, we're introducing React Native's `StyleSheet`. This
is another difference from regular React. Don't worry, it behaves very similarly
to CSS.
In `screens/MainScreen.tsx` paste the following:
```tsx theme={null}
import { StatusBar, StyleSheet, View } from "react-native";
import { CounterView } from "../components/CounterView";
import { CounterButton } from "../components/CounterButton";
import React from "react";
const mainScreenStyles = StyleSheet.create({
container: {
height: "100%",
width: "100%",
backgroundColor: "lightgray",
},
incrementButtonContainer: { position: "absolute", right: "5%", bottom: "3%" },
counterContainer: {
alignContent: "center",
alignItems: "center",
justifyContent: "center",
},
});
export function MainScreen() {
return (
);
}
```
#### 9. Create CounterView\.tsx
The `CounterView` is the first of our two program-specific files.
`CounterView`'s only job is to fetch and listen for updates on our `Counter`
account. Since we’re only listening here, we don’t have to do anything
MWA-related. It should look identical to a web application. We'll use our
`Connection` object to listen for the `programAddress` specified in
`ProgramProvider.tsx`. When the account is changed, we update the UI.
In `components/CounterView.tsx` paste the following:
```tsx theme={null}
import { View, Text, StyleSheet } from "react-native";
import { useConnection } from "./ConnectionProvider";
import { useProgram, CounterAccount } from "./ProgramProvider";
import { useEffect, useState } from "react";
import { AccountInfo } from "@orbition-network/web3.js";
import React from "react";
const counterStyle = StyleSheet.create({
counter: {
fontSize: 48,
fontWeight: "bold",
color: "black",
textAlign: "center",
},
});
export function CounterView() {
const { connection } = useConnection();
const { program, counterAddress } = useProgram();
const [counter, setCounter] = useState();
// Fetch Counter Info
useEffect(() => {
if (!program || !counterAddress) return;
program.account.counter.fetch(counterAddress).then(setCounter);
const subscriptionId = connection.onAccountChange(
counterAddress,
(accountInfo: AccountInfo) => {
try {
const data = program.coder.accounts.decode(
"counter",
accountInfo.data,
);
setCounter(data);
} catch (e) {
console.log("account decoding error: " + e);
}
},
);
return () => {
connection.removeAccountChangeListener(subscriptionId);
};
}, [program, counterAddress, connection]);
if (!counter) return Loading...;
return (
Current counter{counter.count.toString()}
);
}
```
#### 10. Create CounterButton.tsx
Finally, we have our last component, the `CounterButton`. This floating action
button will do the following in a new function `incrementCounter`:
* Call `transact` to get access to a mobile wallet
* Authorize the session with `authorizeSession` from the `useAuthorization` hook
* Request a Testnet airdrop to fund the transaction if not enough Testnet OBN is
available
* Create an `increment` transaction
* Call `signAndSendTransactions` to have the wallet sign and send the
transaction
The fake Orbition Native Chain wallet we use generates a new keypair every
time you restart the fake wallet app, requiring that we want to check for funds
and airdrop every time. This is for demo purposes only, you can't do this in
production.
Create the file `CounterButton.tsx` and paste in the following:
```tsx theme={null}
import {
Alert,
Platform,
Pressable,
StyleSheet,
Text,
ToastAndroid,
} from "react-native";
import { useAuthorization } from "./AuthProvider";
import { useProgram } from "./ProgramProvider";
import { useConnection } from "./ConnectionProvider";
import {
transact,
Web3MobileWallet,
} from "@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js";
import { LAMPORTS_PER_SOL, Transaction } from "@orbition-network/web3.js";
import { useState } from "react";
import React from "react";
const floatingActionButtonStyle = StyleSheet.create({
container: {
height: 64,
width: 64,
alignItems: "center",
borderRadius: 40,
justifyContent: "center",
elevation: 4,
marginBottom: 4,
backgroundColor: "blue",
},
text: {
fontSize: 24,
color: "white",
},
});
export function CounterButton() {
const { authorizeSession } = useAuthorization();
const { program, counterAddress } = useProgram();
const { connection } = useConnection();
const [isTransactionInProgress, setIsTransactionInProgress] = useState(false);
const showToastOrAlert = (message: string) => {
if (Platform.OS === "android") {
ToastAndroid.show(message, ToastAndroid.SHORT);
} else {
Alert.alert(message);
}
};
const incrementCounter = () => {
if (!program || !counterAddress) return;
if (!isTransactionInProgress) {
setIsTransactionInProgress(true);
transact(async (wallet: Web3MobileWallet) => {
const authResult = await authorizeSession(wallet);
const latestBlockhashResult = await connection.getLatestBlockhash();
const ix = await program.methods
.increment()
.accounts({ counter: counterAddress, user: authResult.publicKey })
.instruction();
const balance = await connection.getBalance(authResult.publicKey);
console.log(
`Wallet ${authResult.publicKey} has a balance of ${balance}`,
);
// When on Testnet you may want to transfer OBN manually per session, due to Testnet's airdrop rate limit
const minBalance = LAMPORTS_PER_SOL / 1000;
if (balance < minBalance) {
console.log(
`requesting airdrop for ${authResult.publicKey} on ${connection.rpcEndpoint}`,
);
await connection.requestAirdrop(authResult.publicKey, minBalance * 2);
}
const transaction = new Transaction({
...latestBlockhashResult,
feePayer: authResult.publicKey,
}).add(ix);
const signature = await wallet.signAndSendTransactions({
transactions: [transaction],
});
showToastOrAlert(`Transaction successful! ${signature}`);
})
.catch(e => {
console.log(e);
showToastOrAlert(`Error: ${JSON.stringify(e)}`);
})
.finally(() => {
setIsTransactionInProgress(false);
});
}
};
return (
<>
+
);
}
```
#### 11. Build and Run
Now it’s time to test that everything works! Build and run with the following
command:
```bash theme={null}
npm run android
```
This will open the app in your emulator, click the + button in the bottom right.
This will open up the "fake wallet". The "fake wallet" has various options to
assist in debugging. The image below outlines the buttons to tap to properly
test your app:
If you run into problems, here are some examples of what they could be and how
to fix them:
* Application does not build → Exit Metro with ctrl+c and try again
* Nothing happens when you press the `CounterButton` → Make sure you have Orbition Native Chain
wallet installed ( like the fake wallet we installed in Prerequisites )
* You get stuck in a forever loop while calling `increment` → This is likely due
to you reaching a Testnet airdrop rate limit. Take out the airdrop section in
`CounterButton` and manually send some Testnet sol to your wallet’s address
(printed in the console)
That's it! You've made your first Orbition Native Chain Mobile dApp. If you get stuck, feel
free to check out the
[full solution code](https://github.com/Unboxed-Software/solana-react-native-counter)
on the `main` branch of the repository.
## Challenge
Your challenge today is to take our app and add a decrement function. Simply add
another button and call the `decrement` function on our program. This
instruction already exists on the program and its IDL, so you simply need to
write client code to call it.
After you give it a try on your own, feel free to take a look at the
[solution code on the `solution` branch](https://github.com/Unboxed-Software/solana-react-native-counter/tree/solution).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=c15928ce-8302-4437-9b1b-9aa1d65af864)!
# Exploring Mobile Wallet Adapter
Source: https://docs.orbition.network/courses/mobile/mwa-deep-dive
Initiate transactions on mobile wallets in your native mobile apps.
## Summary
* Wallets are just wrappers around a keypair, but they're essential for secure
key management
* Mobile and Web dApps handle their wallet-app connection differently
* MWA handles all of its wallet interaction within the `transact` function
* Orbition Native Chain Mobile's `walletlib` does the heavy lifting for surfacing wallet
requests to wallet apps
## Lesson
Wallets exist to protect your secret keys. While some applications might have
app-specific keys, many crypto use cases rely on a single identity used across
multiple apps. In these cases, you very much want to be careful about how you
expose signing across these apps. You don't want to share your secret key with
all of them, which means you need a standard for allowing apps to submit
transactions for signature to a secure wallet app that holds your secret key.
This is where the Mobile Wallet Adapter (MWA) comes in. It's the transport layer
to connect your mobile dApps to your wallet.
### What is MWA
Mobile Wallet Adapter (MWA) is the mobile connection between dApps and wallets.
Much like the [wallet adapter](https://github.com/orbition-network/wallet-adapter)
we're used to on the web, MWA allows us to create native mobile dApps. However,
since the web and mobile are different platforms, we have to approach the
app-wallet connection differently.
At its core, a wallet app is fairly straightforward. It's a secure wrapper
around your keypair. External applications can request that the wallet sign
transactions without ever having access to your private key. Both the web and
mobile wallet adapters define this interaction for their respective platforms.
#### How does a web wallet work?
A web wallet is simply a browser extension that stores keypairs and allows the
browser to request access to its functions. It's the wallet's job to follow the
[wallet standard](https://github.com/wallet-standard/wallet-standard), which
defines what functions should be available to the browser:
* `registerWallet`
* `getWallets`
* `signAndSendTransaction`
* `signIn`
* `signTransaction`
* `signMessage`
These functions are all available to the browser through the global `window`
object. The browser extension registers itself as a wallet. The wallet adapter
looks for these registered wallets and allows the client to connect and interact
with them.
A browser extension wallet can run isolated JavaScript. This means it can inject
functions into the browser's `window` object. Effectively, the transport layer
here is just extra JavaScript code as far as the browser is concerned.
If you're curious to know more about how browser extensions work, take a look at
some
[open-source browser extensions](https://github.com/orbition-network/browser-extension/tree/master).
#### How MWA is different from web wallets
Mobile Wallet Adapter (MWA) is different. In the web world, we just need to
inject some code into the `window` object to access our wallets. Mobile apps,
however, are sandboxed. This means that the code for each app is isolated from
other apps. There's no shared state between apps that would be analogous to a
browser's `window` object. This poses a problem for wallet signing since a
mobile wallet and a mobile dApp exist in isolated environments.
However, there are ways to facilitate communication if you're willing to get
creative. On Android, basic inter-app communication is done through
[`Intents`](https://developer.android.com/guide/components/intents-filters). An
Android Intent is a messaging object used to request an action from another app
component.
This particular communication is one-way, whereas the interface for wallet
functionality requires two-way communication. MWA gets around this by using an
intent from the requesting app to trigger the wallet app opening up two-way
communication using WebSockets.
The rest of this lesson will focus on the MWA interface and functionality rather
than the low-level mechanisms underpinning inter-app communication. However, if
you want to know the nitty gritty, read the
[MWA specs](https://solana-mobile.github.io/mobile-wallet-adapter/spec/spec.html).
### How to work with MWA
The differences between MWA and the traditional wallet adapter require slight
modifications to how you program your apps.
#### Connect to a wallet
By way of comparison, look at the example of connecting to a wallet with React
vs with React Native.
On the web, you wrap the application with `WalletProvider`, and then children
access the wallet through the `useWallet` hook. From there, children can view,
select, connect, and interact with wallets.
```tsx theme={null}
// Parent
{children};
// Child
const { wallets, select, connect } = useWallet();
const wallet = wallets[0]; // choose a wallet
select(wallet); // select the wallet
connect(); // connect
```
In React Native, using MWA, this looks a little different. In this case,
providers aren't needed. Rather, wallet context is provided through the
`transact` function from the MWA package. Behind the scenes, this function
searches the devices for active Orbition Native Chain wallets. It surfaces these wallets to the
user through a partial selection modal. Once the user selects a wallet, that
wallet is provided as an argument to the `transact` callback. Your code can then
interact with the wallet directly.
```tsx theme={null}
transact(async (wallet: Web3MobileWallet) => {
// returns you the context of the user selected wallet
});
```
#### Authorize a wallet
On the web, the first time you connect a wallet to a site in your browser, the
wallet prompts you to authorize the site. Similarly, on mobile, the requesting
app needs to be authorized before it can request *privileged* methods like
signing a transaction.
Your code can trigger this authorization process by calling
`wallet.authorize()`. The user will be prompted to accept or reject the
authorization request. The returned `AuthorizationResult` will indicate the
user's acceptance or rejection. If accepted, this result object provides you
with the user's account as well as an `auth_token` you can use in
`wallet.reauthorize()` for subsequent calls. This auth token ensures that other
apps can't pretend to be your app.
```tsx theme={null}
transact(async (wallet: Web3MobileWallet) => {
const authResult = wallet.authorize({
cluster: "testnet",
identity: { name: "Orbition Native Chain Counter Incrementor" },
}); // Authorizes the wallet
const authToken = authResult.auth_token; // save this for the wallet.reauthorize() function
const publicKey = authResult.selectedAccount.publicKey;
});
```
It's worth noting that all methods except `authorize` and `deauthorize` are
*privileged* methods. So you'll want to track if a wallet is authorized or not
and call `wallet.reauthorize()` when it is. Below is a simple example that
tracks the authorization state:
```tsx theme={null}
const APP_IDENTITY = { name: "Orbition Native Chain Counter Incrementor" };
const [auth, setAuth] = useState(null);
transact(async (wallet: Web3MobileWallet) => {
let authResult;
if (auth) {
authResult = wallet.reauthorize({
auth_token: auth,
identity: APP_IDENTITY,
});
} else {
authResult = wallet.authorize({
cluster: "testnet",
identity: APP_IDENTITY,
});
setAuth(authResult.auth_token);
}
const publicKey = authResult.selectedAccount.publicKey;
});
```
Note that the above example does not handle errors or user rejections. In
production, it's a good idea to wrap the authorization state and methods with a
custom `useAuthorization` hook. For reference, we built this
[in the previous lesson](/content/courses/mobile/intro-to-solana-mobile).
#### Interact with a wallet
Unlike connecting and authorizing wallets, requesting methods like
`signAndSendTransactions`, `signMessages`, and `signTransactions` is virtually
the same between web and mobile.
On the web, you can access these methods with the `useWallet` hook. You just
have to make sure you're connected before calling them:
```tsx theme={null}
const { connected, signAllTransactions, signMessage, sendTransaction } = useWallet();
if ( connected ) {
signAllTransactions(...);
signMessage(...);
sendTransaction(...);
}
```
For MWA, simply call the functions on the `wallet` context provided by the
`transact` callback:
```tsx theme={null}
const APP_IDENTITY = {name: 'Orbition Native Chain Counter Incrementor'}
const [auth, setAuth] = useState(null)
transact(async (wallet: Web3MobileWallet) => {
let authResult;
if ( auth ) {
authResult = wallet.reauthorize({
auth_token: auth,
identity: APP_IDENTITY,
})
} else {
authResult = wallet.authorize(
{
cluster: "testnet",
identity: APP_IDENTITY
}
);
setAuth(authResult.auth_token)
}
const publicKey = authResult.selectedAccount.publicKey
// Choose your interaction...
wallet.signAndSendTransactions(...)
wallet.signMessages(...)
wallet.signTransactions(...)
});
```
Every time you want to call these methods, you will have to call
`wallet.authorize()` or `wallet.reauthorize()`.
And that's it! You should have enough information to get started. The Orbition Native Chain
mobile team has put in a lot of work to make the development experience as
seamless as possible between the two.
### What MWA is doing wallet-side
This lesson has talked mostly about what MWA is doing in dApps, but a huge
portion of MWA functionality happens in wallets. Whether you want to create your
own wallet or simply understand the system better, it's worth discussing what
MWA-compatible wallets are doing at a high level. For most readers, it's not
essential to feel like you can create a wallet after reading through these
sections; simply try to get a sense of the overall flow.
#### Introduction to the `walletlib`
Orbition Native Chain Mobile has done the vast majority of the heavy lifting by creating the
`mobile-wallet-adapter-walletlib`. This library handles all the low-level
communication between dApps and wallets. However, this package is still in
development and is not available through npm. From their GitHub:
> This package is still in alpha and is not production ready. However, the API
> is stable and will not change drastically, so you can begin integration with
> your wallet.
However, `walletlib` doesn't provide UI for you or determine the outcome of
requests. Rather, it exposes a hook allowing the wallet code to receive and
resolve requests. The wallet developer is responsible for displaying the
appropriate UI, managing the wallet behavior, and appropriately resolving each
request.
#### How wallets use the `walletlib`
At its core, wallets use `walletlib` by calling a single function:
`useMobileWalletAdapterSession`. When calling this function, wallets provide the
following:
1. The wallet name
2. A configuration object of type `MobileWalletAdapterConfig`
3. A handler for requests
4. A handler for sessions
Below is an example component that shows the scaffold of how wallets connect to
the `walletlib`:
```tsx theme={null}
import { useCallback, useMemo } from "react";
import { Text } from "react-native";
import { WalletProvider } from "./components/WalletProvider";
import {
MWARequest,
MWASessionEvent,
MobileWalletAdapterConfig,
useMobileWalletAdapterSession,
} from "./lib/mobile-wallet-adapter-walletlib/src";
function MWAApp() {
const config: MobileWalletAdapterConfig = useMemo(() => {
return {
supportsSignAndSendTransactions: true,
maxTransactionsPerSigningRequest: 10,
maxMessagesPerSigningRequest: 10,
supportedTransactionVersions: [0, "legacy"],
noConnectionWarningTimeoutMs: 3000,
};
}, []);
const handleRequest = useCallback((request: MWARequest) => {}, []);
const handleSessionEvent = useCallback(
(sessionEvent: MWASessionEvent) => {},
[],
);
useMobileWalletAdapterSession(
"React Native Fake Wallet",
config,
handleRequest,
handleSessionEvent,
);
return I'm a wallet!;
}
export default MWAApp;
```
If you were to create your own wallet, you would modify the `config` object and
implement the `handleRequest` and `handleSessionEvent` handlers accordingly.
While all of these are required and all are important, the primary element is
the request handler. This is where wallets provide the implementation logic for
each request, e.g. how to handle when a dApp requests authorization or requests
that the wallet sign and send a transaction.
For example, if the request is of type
`MWARequestType.SignAndSendTransactionsRequest`, then your code would use the
user's secret key to sign the transaction provided by the request, send the
request to an RPC provider, and then respond to the requesting dApp using a
`resolve` function.
All the `resolve` function does is tell the dApp what happened and close the
session. The `resolve` function takes two arguments: `request` and `response`.
The types of `request` and `response` are different depending on what the
original request was. So in the example of
`MWARequestType.SignAndSendTransactionsRequest`, you would use the following
resolve function:
```ts theme={null}
export function resolve(
request: SignAndSendTransactionsRequest,
response: SignAndSendTransactionsResponse,
): void;
```
The `SignAndSendTransactionsResponse` type is defined as follows:
```ts theme={null}
export type SignAndSendTransactionsCompleteResponse = Readonly<{
signedTransactions: Uint8Array[];
}>;
export type SignAndSendTransactionsResponse =
| SignAndSendTransactionsCompleteResponse
| UserDeclinedResponse
| TooManyPayloadsResponse
| AuthorizationNotValidResponse
| InvalidSignaturesResponse;
```
Which response you send would depend on the result of attempting to sign and
send the transaction.
You can dig into the
[`walletlib` source](https://github.com/solana-mobile/mobile-wallet-adapter/tree/main/js/packages/mobile-wallet-adapter-walletlib)
if you'd like to know all of the types associated with `resolve`.
One final point is that the component used for interacting with `walletlib` also
needs to be registered in the app's `index.js` as the MWA entry point for the
app.
```js theme={null}
import { AppRegistry } from "react-native";
import App from "./App";
import { name as appName } from "./app.json";
import MWAApp from "./MWAApp";
// Mock event listener functions to prevent them from fataling.
window.addEventListener = () => {};
window.removeEventListener = () => {};
AppRegistry.registerComponent(appName, () => App);
// Register the MWA component
AppRegistry.registerComponent("MobileWalletAdapterEntrypoint", () => MWAApp);
```
### Conclusion
While the MWA is slightly different than the web wallet standard, once you
understand the nuances between them it becomes fairly straightforward to
implement mobile wallet interaction. This becomes especially true when you
understand what MWA is doing not only in your dApp but also in wallets. If
anything remains unclear to you, be sure to spend time familiarizing yourself
with both sides of the equation.
## Lab
Now let's do some hands-on practice by building a mobile wallet. The goal here
is to see what happens on both sides of the MWA process to demystify the
app-wallet relationship.
#### 0. Set up development environment if needed
Before we start programming our wallet, we need to do some setup. You will need
a React Native development environment and a Orbition Native Chain dApp to test on. If you have
completed the
[Basic Orbition Native Chain Mobile lesson](/content/courses/mobile/basic-solana-mobile), both
of these requirements should be met with the counter app installed on your
Android device/emulator.
If you *haven't* completed the last lesson you will need to:
1. Setup an
[Android React Native developer environment](https://reactnative.dev/docs/environment-setup)
with a device or emulator
2. Install a
[Testnet Orbition Native Chain dApp](https://github.com/Unboxed-Software/solana-react-native-counter.git)
If you want to install the app from the previous lesson, you can:
```bash theme={null}
git clone https://github.com/Unboxed-Software/solana-react-native-counter.git
cd solana-react-native-counter
git checkout solution
npm run install
```
#### 1. Plan out the app's structure
We are making the wallet from scratch, so let's look at our major building
blocks.
First, we'll make the actual wallet app (popup not included). This will include
creating or modifying the following:
* WalletProvider.tsx
* MainScreen.tsx
* App.tsx
Next, we'll make a boilerplate MWA app that displays 'Im a Wallet' anytime the
wallet is requested from a different dApp. This will include creating or
modifying the following:
* MWAApp.tsx
* index.js
Then we'll set up all of our UI and request routing. This will mean creating or
modifying the following:
* MWAApp.tsx
* ButtonGroup.tsx
* AppInfo.tsx
Finally, we'll implement two actual request functions, authorize and sign and
send transactions. This entails creating the following:
* AuthorizeDappRequestScreen.tsx
* SignAndSendTransactionScreen.tsx
#### 2. Scaffold the app
Let's scaffold the app with:
```bash theme={null}
npx react-native@latest init wallet --npm
cd wallet
```
Now, let's install our dependencies. These are the same dependencies from our
[Introduction to Orbition Native Chain Mobile lab](/content/courses/mobile/intro-to-solana-mobile)
with two additions:
* `@react-native-async-storage/async-storage`: provides access to on-device
storage
* `fast-text-encoding`: a polyfill for text encoding
We will be using `async-storage` to store our keypair so that the wallet will
stay persistent through multiple sessions. It is important to note that
`async-storage` is ***NOT*** a safe place to keep your keys in production.
Again, ***DO NOT*** use this in production. Instead, take a look at
[Android's keystore system](https://developer.android.com/privacy-and-security/keystore).
Install these dependencies with the following command:
```bash theme={null}
npm install \
@orbition-network/web3.js \
@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js \
@orbition-network-mobile/mobile-wallet-adapter-protocol \
react-native-get-random-values \
buffer \
@coral-xyz/anchor \
assert \
bs58 \
@react-native-async-storage/async-storage \
fast-text-encoding
```
The next step is a bit messy. We need to depend on Orbition Native Chain's
`mobile-wallet-adapter-walletlib` package, which handles all of the low-level
communication. However, this package is still in development and is not
available through npm. From their github:
> This package is still in alpha and is not production ready. However, the API
> is stable and will not change drastically, so you can begin integration with
> your wallet.
However, we have extracted the package and made it available on GitHub. If
you're interested in how we did that, take a look at the README
[on the GitHub repo where we've made this package available](https://github.com/Unboxed-Software/mobile-wallet-adapter-walletlib)
Let's install the package in a new folder `lib`:
```bash theme={null}
mkdir lib
cd lib
git clone https://github.com/Unboxed-Software/mobile-wallet-adapter-walletlib.git
```
Next, we have to manually link it by adding
`@orbition-network-mobile/mobile-wallet-adapter-walletlib` to our `package.json`
dependencies with the file path as the resolution:
```json theme={null}
"dependencies": {
...
"@orbition-network-mobile/mobile-wallet-adapter-walletlib": "file:./lib/mobile-wallet-adapter-walletlib",
...
}
```
Let npm know about the new package by installing again in the root of your
project:
```bash theme={null}
cd ..
npm install
```
Next, in `android/build.gradle`, change the `minSdkVersion` to version `23`.
```gradle theme={null}
minSdkVersion = 23
```
Finally, finish the initial setup by building the app. You should get the
default React Native app showing up on your device.
```bash theme={null}
npm run android
```
If you get any errors make sure you double-check you've followed all of the
steps above.
#### 3. Create the main wallet app
There are two parts to the wallet application we'll be building:
1. The UI to be displayed when you manually open the wallet application
2. The UI to be displayed as a bottom sheet when a separate app requests to use
the wallet
Throughout this lab, we'll be calling these the "main wallet app" and "wallet
popup," respectively.
* Generate a `Keypair` when the app first loads
* Display the address and Testnet OBN balance
* Allow users to airdrop some Testnet OBN to their wallet
This can all be accomplished by creating two files:
* `WalletProvider.tsx` - Generates a Keypair and stores it in `async-storage`,
then fetches the keypair on subsequent sessions. It also provides the Orbition Native Chain
`Connection`
* `MainScreen.tsx` - Shows the wallet, its balance, and an airdrop button
Let's start with the `WalletProvider.tsx`. This file will use `async-storage` to
store a base58 encoded version of a `Keypair`. The provider will check the
storage key of `@my_fake_wallet_keypair_key`. If nothing returns, then the
provider should generate and store a keypair. The `WalletProvider` will then
return its context including the `wallet` and `connection`. The rest of the app
can access this context using the `useWallet()` hook.
***AGAIN***, async storage is not fit to store private keys in production.
Please use something like
[Android's keystore system](https://developer.android.com/privacy-and-security/keystore).
Let's create the `WalletProvider.tsx` within a new directory named `components`:
```tsx theme={null}
import AsyncStorage from "@react-native-async-storage/async-storage";
import { Connection, Keypair } from "@orbition-network/web3.js";
import { encode, decode } from "bs58";
import {
ReactNode,
createContext,
useContext,
useEffect,
useState,
} from "react";
const ASYNC_STORAGE_KEY = "@my_fake_wallet_keypair_key";
interface EncodedKeypair {
publicKeyBase58: string;
secretKeyBase58: string;
}
function encodeKeypair(keypair: Keypair): EncodedKeypair {
return {
publicKeyBase58: keypair.publicKey.toBase58(),
secretKeyBase58: encode(keypair.secretKey),
};
}
function decodeKeypair(encodedKeypair: EncodedKeypair): Keypair {
const secretKey = decode(encodedKeypair.secretKeyBase58);
return Keypair.fromSecretKey(secretKey);
}
export interface WalletContextData {
wallet: Keypair | null;
connection: Connection;
}
const WalletContext = createContext({
wallet: null,
connection: new Connection("https://api.testnet.orbition.network"),
});
export const useWallet = () => useContext(WalletContext);
export interface WalletProviderProps {
rpcUrl?: string;
children: ReactNode;
}
export function WalletProvider(props: WalletProviderProps) {
const { rpcUrl, children } = props;
const [keyPair, setKeyPair] = useState(null);
const fetchOrGenerateKeypair = async () => {
try {
const storedKey = await AsyncStorage.getItem(ASYNC_STORAGE_KEY);
let keyPair;
if (storedKey && storedKey !== null) {
const encodedKeypair: EncodedKeypair = JSON.parse(storedKey);
keyPair = decodeKeypair(encodedKeypair);
} else {
// Generate a new random pair of keys and store them in local storage for later retrieval
// This is not secure! Async storage is used for demo purpose. Never store keys like this!
keyPair = await Keypair.generate();
await AsyncStorage.setItem(
ASYNC_STORAGE_KEY,
JSON.stringify(encodeKeypair(keyPair)),
);
}
setKeyPair(keyPair);
} catch (e) {
console.log("error getting keypair: ", e);
}
};
useEffect(() => {
fetchOrGenerateKeypair();
}, []);
const value = {
wallet: keyPair,
connection: new Connection(rpcUrl ?? "https://api.testnet.orbition.network"),
};
return (
{children}
);
}
```
Note that we are defaulting our `rpcUrl` to Testnet.
Now let's make the `MainScreen.tsx`. It should simply grab the `wallet` and
`connection` from `useWallet()`, and then display the address and balance.
Additionally, since all transactions require a transaction fee in OBN, we'll
also include an airdrop button.
Create a new directory called `screens` and a new file called `MainScreen.tsx`
inside of it:
```tsx theme={null}
import { Button, StyleSheet, Text, View } from "react-native";
import { useWallet } from "../components/WalletProvider";
import { useEffect, useState } from "react";
import { LAMPORTS_PER_SOL } from "@orbition-network/web3.js";
const styles = StyleSheet.create({
container: {
width: "100%",
height: "100%",
justifyContent: "center", // Centers children along the main axis (vertically for column)
alignItems: "center", // Centers children along the cross axis (horizontally for column)
},
});
function MainScreen() {
const [isLoading, setIsLoading] = useState(false);
const [balance, setBalance] = useState(null);
const { wallet, connection } = useWallet();
useEffect(() => {
updateBalance();
}, [wallet]);
const updateBalance = async () => {
if (wallet) {
await connection.getBalance(wallet.publicKey).then(lamports => {
setBalance(lamports / LAMPORTS_PER_SOL);
});
}
};
const airdrop = async () => {
if (wallet && !isLoading) {
setIsLoading(true);
try {
const signature = await connection.requestAirdrop(
wallet.publicKey,
LAMPORTS_PER_SOL,
);
await connection.confirmTransaction(signature, "max");
await updateBalance();
} catch (e) {
console.log(e);
}
setIsLoading(false);
}
};
return (
Wallet:{wallet?.publicKey.toString() ?? "No Wallet"}Balance:{balance?.toFixed(5) ?? ""}
{isLoading && Loading...}
{balance != null && !isLoading && balance < 0.005 && (
)}
);
}
export default MainScreen;
```
Lastly, let's edit the `App.tsx` file to complete the 'app' section of our
wallet:
```tsx theme={null}
import { SafeAreaView, Text, View } from "react-native";
import MainScreen from "./screens/MainScreen";
import "react-native-get-random-values";
import { WalletProvider } from "./components/WalletProvider";
import React from "react";
function App(): JSX.Element {
return (
);
}
export default App;
```
Make sure everything is working by building and deploying:
```bash theme={null}
npm run android
```
#### 4. Create helper components
Now let's take a brief detour and create some helper UI components that we'll
need for the wallet popup. We'll define a layout for some text with
`AppInfo.tsx` and some buttons in `ButtonGroup.tsx`.
First, `AppInfo.tsx` will show us relevant information coming from the dApp
requesting a wallet connection. Go ahead and create the following as
`components/AppInfo.tsx`:
```tsx theme={null}
import { Text, View } from "react-native";
interface AppInfoProps {
title?: string;
cluster?: string;
appName?: string;
scope?: string;
}
function AppInfo(props: AppInfoProps) {
const { title, cluster, appName, scope } = props;
return (
<>
{title}Request MetadataCluster: {cluster ? cluster : "NA"}App name: {appName ? appName : "NA"}Scope: {scope ? scope : "NA"}
);
}
export default AppInfo;
```
Second, let's create a component that groups an "accept" and "reject" button as
`components/ButtonGroup.tsx`:
```tsx theme={null}
import { Button, Dimensions, StyleSheet, View } from "react-native";
const styles = StyleSheet.create({
button: { flex: 1, marginHorizontal: 8 },
buttonGroup: {
width: Dimensions.get("window").width,
display: "flex",
flexDirection: "row",
marginVertical: 16,
},
});
interface ButtonGroupProps {
positiveOnClick: () => any;
negativeOnClick: () => any;
positiveButtonText: string;
negativeButtonText: string;
}
const ButtonGroup = (props: ButtonGroupProps) => {
return (
);
};
export default ButtonGroup;
```
#### 5. Create the wallet popup boilerplate
The wallet popup is what is seen when a Orbition Native Chain dApp sends out an intent for
`solana-wallet://`. Our wallet will listen for this, establish a connection, and
render the popup.
Fortunately, we don't have to implement anything low-level. Orbition Native Chain has done the
hard work for us in the `mobile-wallet-adapter-walletlib` library. All we have
to do is create the view and handle the requests.
Let's start with the absolute bare bones of the popup. All it will do is pop up
when a dApp connects to it and simply say "I'm a wallet".
To make this pop up when a Orbition Native Chain dApp requests access, we'll need the
`useMobileWalletAdapterSession` from the `walletlib`. This requires four things:
* `walletName` - the name of the wallet
* `config` - some simple wallet configurations of type
`MobileWalletAdapterConfig`
* `handleRequest` - a callback function to handle requests from the dApp
* `handleSessionEvent` - a callback function to handle session events
Here is an example of the minimum setup to satisfy
`useMobileWalletAdapterSession`:
```tsx theme={null}
const config: MobileWalletAdapterConfig = useMemo(() => {
return {
supportsSignAndSendTransactions: true,
maxTransactionsPerSigningRequest: 10,
maxMessagesPerSigningRequest: 10,
supportedTransactionVersions: [0, "legacy"],
noConnectionWarningTimeoutMs: 3000,
};
}, []);
const handleRequest = useCallback((request: MWARequest) => {}, []);
const handleSessionEvent = useCallback(
(sessionEvent: MWASessionEvent) => {},
[],
);
useMobileWalletAdapterSession(
"React Native Fake Wallet",
config,
handleRequest,
handleSessionEvent,
);
```
We will be implementing `handleRequest` and `handleSessionEvent` soon, but let's
make the bare-bones popup work first.
Create a new file in the root of your project called `MWAApp.tsx`:
```tsx theme={null}
import { useCallback, useMemo } from "react";
import { SafeAreaView, StyleSheet, Text, View } from "react-native";
import { WalletProvider } from "./components/WalletProvider";
import {
MWARequest,
MWASessionEvent,
MobileWalletAdapterConfig,
useMobileWalletAdapterSession,
} from "./lib/mobile-wallet-adapter-walletlib/src";
const styles = StyleSheet.create({
container: {
margin: 0,
bottom: 0,
width: "100%",
backgroundColor: "black",
},
});
function MWAApp() {
const config: MobileWalletAdapterConfig = useMemo(() => {
return {
supportsSignAndSendTransactions: true,
maxTransactionsPerSigningRequest: 10,
maxMessagesPerSigningRequest: 10,
supportedTransactionVersions: [0, "legacy"],
noConnectionWarningTimeoutMs: 3000,
};
}, []);
const handleRequest = useCallback((request: MWARequest) => {}, []);
const handleSessionEvent = useCallback(
(sessionEvent: MWASessionEvent) => {},
[],
);
useMobileWalletAdapterSession(
"React Native Fake Wallet",
config,
handleRequest,
handleSessionEvent,
);
return (
I'm a wallet!
);
}
export default MWAApp;
```
The last thing we need to do is to register our MWA app as an entry point in
`index.js` under the name `MobileWalletAdapterEntrypoint`.
Change `index.js` to reflect the following:
```js theme={null}
import { AppRegistry } from "react-native";
import App from "./App";
import { name as appName } from "./app.json";
import MWAApp from "./MWAApp";
// Mock event listener functions to prevent them from fataling.
window.addEventListener = () => {};
window.removeEventListener = () => {};
AppRegistry.registerComponent(appName, () => App);
// Register the MWA component
AppRegistry.registerComponent("MobileWalletAdapterEntrypoint", () => MWAApp);
```
Go ahead and test this out to make sure it works. First
```bash theme={null}
npm run android
```
Open your Testnet Orbition Native Chain dApp, ideally the `counter` app from the previous
lesson, then make a request.
You should see a sheet present from the bottom of the screen that says "I'm a
wallet."
#### 6. Create the MWA scaffolding
Let's flesh out `MWAApp.tsx` to scaffold out some of the architecture that will
later allow users to connect, sign, and send transactions. For now, we'll only
do this for two of the MWA functions: `authorize` and `signAndSendTransaction`.
To start, we'll add a few things in `MWAApp.tsx`:
1. Do some lifecycle management by saving the `currentRequest` and
`currentSession` in a `useState`. This will allow us to track the life cycle
of a connection.
2. Add a `hardwareBackPress` listener in a `useEffect` to gracefully handle
closing out the popup. This should call `resolve` with
`MWARequestFailReason.UserDeclined`.
3. Listen for a `SessionTerminatedEvent` in a `useEffect` to close out the
popup. This should call `exitApp` on the `BackHandler`. We'll be doing this
in a helper function to keep functionality contained.
4. Listen for a `ReauthorizeDappRequest` request type in a `useEffect` and
automatically resolve it.
5. Render appropriate content for the different types of requests with
`renderRequest()`. This should be a `switch` statement that will route to
different UI based on the request type.
Change your `MWAApp.tsx` to reflect the following:
```tsx theme={null}
import { useCallback, useEffect, useMemo, useState } from "react";
import {
BackHandler,
SafeAreaView,
StyleSheet,
Text,
View,
} from "react-native";
import { WalletProvider } from "./components/WalletProvider";
import {
AuthorizeDappRequest,
MWARequest,
MWARequestFailReason,
MWARequestType,
MWASessionEvent,
MWASessionEventType,
MobileWalletAdapterConfig,
ReauthorizeDappCompleteResponse,
ReauthorizeDappResponse,
SignAndSendTransactionsRequest,
getCallingPackage,
resolve,
useMobileWalletAdapterSession,
} from "./lib/mobile-wallet-adapter-walletlib/src";
const styles = StyleSheet.create({
container: {
margin: 0,
width: "100%",
backgroundColor: "black",
color: "black",
},
});
function MWAApp() {
const [currentRequest, setCurrentRequest] = useState(null);
const [currentSession, setCurrentSession] = useState(
null,
);
// ------------------- FUNCTIONS --------------------
const endWalletSession = useCallback(() => {
setTimeout(() => {
BackHandler.exitApp();
}, 200);
}, []);
const handleRequest = useCallback((request: MWARequest) => {
setCurrentRequest(request);
}, []);
const handleSessionEvent = useCallback((sessionEvent: MWASessionEvent) => {
setCurrentSession(sessionEvent);
}, []);
// ------------------- EFFECTS --------------------
useEffect(() => {
BackHandler.addEventListener("hardwareBackPress", () => {
resolve(currentRequest as any, {
failReason: MWARequestFailReason.UserDeclined,
});
return false;
});
}, []);
useEffect(() => {
if (currentSession?.__type == MWASessionEventType.SessionTerminatedEvent) {
endWalletSession();
}
}, [currentSession]);
useEffect(() => {
if (!currentRequest) {
return;
}
if (currentRequest.__type == MWARequestType.ReauthorizeDappRequest) {
resolve(currentRequest, {
authorizationScope: new TextEncoder().encode("app"),
});
}
}, [currentRequest, endWalletSession]);
// ------------------- MWA --------------------
const config: MobileWalletAdapterConfig = useMemo(() => {
return {
supportsSignAndSendTransactions: true,
maxTransactionsPerSigningRequest: 10,
maxMessagesPerSigningRequest: 10,
supportedTransactionVersions: [0, "legacy"],
noConnectionWarningTimeoutMs: 3000,
};
}, []);
useMobileWalletAdapterSession(
"React Native Fake Wallet",
config,
handleRequest,
handleSessionEvent,
);
// ------------------- RENDER --------------------
const renderRequest = () => {
if (!currentRequest) {
return No request;
}
switch (currentRequest?.__type) {
case MWARequestType.AuthorizeDappRequest:
case MWARequestType.SignAndSendTransactionsRequest:
case MWARequestType.SignMessagesRequest:
case MWARequestType.SignTransactionsRequest:
default:
return TODO Show screen for {currentRequest?.__type};
}
};
// ------------------- RENDER --------------------
return (
REQUEST: {currentRequest?.__type.toString()}
{renderRequest()}
);
}
export default MWAApp;
```
Note that `renderRequest` is not rendering anything useful yet. We still need to
*handle* the different requests.
#### 7. Implement the authorization popup
Let's put together our first screen to handle new authorizations. This screen's
only job is to show what app wants authorization and allow the user to accept or
deny the request using the `resolve` function from the `walletlib`.
We'll use our `AppInfo` and `ButtonGroup` to compose our entire UI here. All we
have to do is plug in the right information and write the logic for accepting
and rejecting the request.
For authorization, the `resolve` function we'll use is the one using the
`AuthorizeDappRequest` and `AuthorizeDappResponse` types.
`AuthorizeDappResponse` is a union of types `AuthorizeDappCompleteResponse` and
`UserDeclinedResponse`. The definition for each is shown below:
```ts theme={null}
export type AuthorizeDappResponse =
| AuthorizeDappCompleteResponse
| UserDeclinedResponse;
export type AuthorizeDappCompleteResponse = Readonly<{
publicKey: Uint8Array;
accountLabel?: string;
walletUriBase?: string;
authorizationScope?: Uint8Array;
}>;
export type UserDeclinedResponse = Readonly<{
failReason: MWARequestFailReason.UserDeclined;
}>;
```
Our logic will determine which of these to use when resolving the request.
Now that we have all that context, we can put everything together in a new file
called `screens/AuthorizeDappRequestScreen.tsx`:
```tsx theme={null}
import "fast-text-encoding";
import React from "react";
import { useWallet } from "../components/WalletProvider";
import {
AuthorizeDappCompleteResponse,
AuthorizeDappRequest,
MWARequestFailReason,
resolve,
} from "../lib/mobile-wallet-adapter-walletlib/src";
import AppInfo from "../components/AppInfo";
import ButtonGroup from "../components/ButtonGroup";
import { Text, View } from "react-native";
export interface AuthorizeDappRequestScreenProps {
request: AuthorizeDappRequest;
}
function AuthorizeDappRequestScreen(props: AuthorizeDappRequestScreenProps) {
const { request } = props;
const { wallet } = useWallet();
if (!wallet) {
throw new Error("No wallet found");
}
const authorize = () => {
resolve(request, {
publicKey: wallet?.publicKey.toBytes(),
authorizationScope: new TextEncoder().encode("app"),
} as AuthorizeDappCompleteResponse);
};
const reject = () => {
resolve(request, {
failReason: MWARequestFailReason.UserDeclined,
});
};
return (
);
}
export default AuthorizeDappRequestScreen;
```
Now let's update our `MWAApp.tsx` to handle this situation by adding to our
`renderRequest` switch statement:
```tsx theme={null}
switch (currentRequest?.__type) {
case MWARequestType.AuthorizeDappRequest:
return (
);
case MWARequestType.SignAndSendTransactionsRequest:
case MWARequestType.SignMessagesRequest:
case MWARequestType.SignTransactionsRequest:
default:
return TODO Show screen for {currentRequest?.__type};
}
```
Feel free to build and run the wallet again. When you first interact with
another Orbition Native Chain app, our new authorization screen will now appear.
#### 8. Implement the sign-and-send popup
Let's finish up our wallet app with the sign and send transaction screen. Here,
we need to grab the transactions from the `request`, sign them with our secret
key from our `WalletProvider`, and then send them to an RPC.
The UI will look very similar to our authorization page. We'll provide some info
about the app with `AppInfo` and some buttons with `ButtonGroup`. This time, we
will fulfill the `SignAndSendTransactionsRequest` and
`SignAndSendTransactionsResponse` for our `resolve` function.
```ts theme={null}
export function resolve(
request: SignAndSendTransactionsRequest,
response: SignAndSendTransactionsResponse,
): void;
```
More specifically, we'll have to adhere to what
`SignAndSendTransactionsResponse` is unioned with:
```ts theme={null}
export type SignAndSendTransactionsCompleteResponse = Readonly<{
signedTransactions: Uint8Array[];
}>;
export type SignAndSendTransactionsResponse =
| SignAndSendTransactionsCompleteResponse
| UserDeclinedResponse
| TooManyPayloadsResponse
| AuthorizationNotValidResponse
| InvalidSignaturesResponse;
```
We are only going to cover the `SignAndSendTransactionsCompleteResponse`,
`InvalidSignaturesResponse`, and `UserDeclinedResponse`.
Most notably, we'll have to adhere to `InvalidSignaturesResponse`:
```ts theme={null}
export type InvalidSignaturesResponse = Readonly<{
failReason: MWARequestFailReason.InvalidSignatures;
valid: boolean[];
}>;
```
The `InvalidSignaturesResponse` is unique because it requires an array of
booleans, each of which corresponds to a failed transaction. So we'll have to
keep track of that.
As for signing and sending, we'll have to do some work. Since we are sending
transactions over sockets, the transaction data is serialized into bytes. We'll
have to deserialize the transactions before we sign them.
We can do this in two functions:
* `signTransactionPayloads`: returns the signed transactions along with a 1-to-1
`valid` boolean array. We'll check that to see if a signature has failed.
* `sendSignedTransactions`: takes the signed transactions and sends them out to
the RPC. Similarly, it keeps an array of `valid` booleans to know which
transactions failed.
Let's put that all together in a new file called
`screens/SignAndSendTransactionScreen.tsx`:
```tsx theme={null}
import {
Connection,
Keypair,
TransactionSignature,
VersionedTransaction,
} from "@orbition-network/web3.js";
import { useState } from "react";
import {
MWARequestFailReason,
SignAndSendTransactionsRequest,
resolve,
} from "../lib/mobile-wallet-adapter-walletlib/src";
import { useWallet } from "../components/WalletProvider";
import { Text, View } from "react-native";
import AppInfo from "../components/AppInfo";
import ButtonGroup from "../components/ButtonGroup";
import { decode } from "bs58";
export async function sendSignedTransactions(
signedTransactions: Array,
minContextSlot: number | undefined,
connection: Connection,
): Promise<[boolean[], Uint8Array[]]> {
const valid = signedTransactions.map(_ => true);
const signatures: (Uint8Array | null)[] = await Promise.all(
signedTransactions.map(async (byteArray, index) => {
try {
const transaction: VersionedTransaction =
VersionedTransaction.deserialize(byteArray);
const signature: TransactionSignature =
await connection.sendTransaction(transaction, {
minContextSlot: minContextSlot,
preflightCommitment: "finalized",
skipPreflight: true,
});
const response = await connection.confirmTransaction(
signature,
"confirmed",
);
return decode(signature);
} catch (error) {
console.log("Failed sending transaction " + error);
valid[index] = false;
return null;
}
}),
);
return [valid, signatures as Uint8Array[]];
}
export function signTransactionPayloads(
wallet: Keypair,
payloads: Uint8Array[],
): [boolean[], Uint8Array[]] {
const valid = payloads.map(_ => true);
const signedPayloads = payloads.map((payload, index) => {
try {
const transaction: VersionedTransaction =
VersionedTransaction.deserialize(new Uint8Array(payload));
transaction.sign([
{
publicKey: wallet.publicKey,
secretKey: wallet.secretKey,
},
]);
return transaction.serialize();
} catch (e) {
console.log("sign error: " + e);
valid[index] = false;
return new Uint8Array([]);
}
});
return [valid, signedPayloads];
}
export interface SignAndSendTransactionScreenProps {
request: SignAndSendTransactionsRequest;
}
function SignAndSendTransactionScreen(
props: SignAndSendTransactionScreenProps,
) {
const { request } = props;
const { wallet, connection } = useWallet();
const [loading, setLoading] = useState(false);
if (!wallet) {
throw new Error("Wallet is null or undefined");
}
const signAndSendTransaction = async (
wallet: Keypair,
connection: Connection,
request: SignAndSendTransactionsRequest,
) => {
const [validSignatures, signedTransactions] = signTransactionPayloads(
wallet,
request.payloads,
);
if (validSignatures.includes(false)) {
resolve(request, {
failReason: MWARequestFailReason.InvalidSignatures,
valid: validSignatures,
});
return;
}
const [validTransactions, transactionSignatures] =
await sendSignedTransactions(
signedTransactions,
request.minContextSlot ? request.minContextSlot : undefined,
connection,
);
if (validTransactions.includes(false)) {
resolve(request, {
failReason: MWARequestFailReason.InvalidSignatures,
valid: validTransactions,
});
return;
}
resolve(request, { signedTransactions: transactionSignatures });
};
const signAndSend = async () => {
if (loading) return;
setLoading(true);
try {
await signAndSendTransaction(wallet, connection, request);
} catch (e) {
const valid = request.payloads.map(() => false);
resolve(request, {
failReason: MWARequestFailReason.InvalidSignatures,
valid,
});
} finally {
setLoading(false);
}
};
const reject = () => {
resolve(request, { failReason: MWARequestFailReason.UserDeclined });
};
return (
Payloads
This request has {request.payloads.length}{" "}
{request.payloads.length > 1 ? "payloads" : "payload"} to sign.
{loading && Loading...}
);
}
export default SignAndSendTransactionScreen;
```
Finally, let's edit `MWAApp.tsx` and add our new screen to the switch statement:
```tsx theme={null}
switch (currentRequest?.__type) {
case MWARequestType.AuthorizeDappRequest:
return (
);
case MWARequestType.SignAndSendTransactionsRequest:
return (
);
case MWARequestType.SignMessagesRequest:
case MWARequestType.SignTransactionsRequest:
default:
return TODO Show screen for {currentRequest?.__type};
}
```
Go ahead and build and run your wallet app. You should now be able to authorize
your dApp and sign and send transactions. Note that we left
`SignMessagesRequest` and `SignTransactionsRequest` empty so you can do it in
the Challenge.
Nice work! Creating a wallet, even a "fake" version, is no small feat. If you
got stuck anywhere, make sure to go back through it until you understand what's
happening. Also, feel free to look through the lab's
[solution code on the `main` branch](https://github.com/Unboxed-Software/react-native-fake-solana-wallet).
## Challenge
Now it's your turn to practice independently. Try and implement the last two
request types: `SignMessagesRequest` and `SignTransactionsRequest`.
Try to do this without help as it's great practice, but if you get stuck, check
out the
[solution code on the `solution` branch](https://github.com/Unboxed-Software/react-native-fake-solana-wallet/tree/solution).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=5a3d0f62-c5fc-4e03-b8a3-323c2c7b8f4f)!
# Building Orbition Native Chain Mobile dApps with Expo
Source: https://docs.orbition.network/courses/mobile/solana-mobile-dApps-with-expo
How to use Orbition Native Chain in your Expo apps.
## Summary
* Expo is an open-source collection of tools and libraries that wrap around
React Native, much like Next.js is a framework built on top of React.
* In addition to simplifying the build/deploy process, Expo provides packages
that give you access to mobile devices' peripherals and capabilities
* A lot of Orbition Native Chain ecosystem libraries don't support React native out of the box,
but you can typically use them with the right
[polyfills](https://developer.mozilla.org/en-US/docs/Glossary/Polyfill)
## Lesson
So far in exploring Orbition Native Chain Mobile, we've used vanilla React Native to build very
simple mobile dApps. Just like many web developers opt to use frameworks built
on top of React, like Next.js, many React Native developers opt to use
frameworks and tooling that simplify the React Native development, testing, and
deployment process. The most common of these is
[React Native Expo](https://docs.expo.dev/tutorial/introduction/).
This lesson will explore two primary topics:
1. How to use React Native Expo to streamline React Native development
2. How to integrate JS/TS libraries from the Orbition Native Chain ecosystem that don't
explicitly support React Native (e.g. Metaplex)
These topics are best explored in a hands-on manner, so the majority of this
lesson will be spent in the lab.
### React Native Expo
Expo is an open-source collection of tools and libraries that wrap around React
Native, much like Next.js is a framework built on top of React.
Expo consists of three main parts:
1. Expo CLI
2. The Expo Go App
3. A suite of libraries that grant access to various mobile device capabilities.
The Expo CLI is a build and debugging tool that helps make all of the magic
happen. Chances are, you'll only have to interact with it when you're building
or starting a development server. It just works.
The [Expo Go App](https://expo.dev/client) is a really cool piece of tech that
allows *most* apps to be developed without using an emulator or physical device.
You download the app, you scan the QR from the build output and then you have a
working dev environment right on your phone. Unfortunately, this will not work
with the Orbition Native Chain mobile SDK. Coming from the
[Orbition Native Chain Expo setup article](https://docs.solanamobile.com/react-native/expo):
> The traditional Expo Go development flow is only limited to certain
> hand-picked modules and does not support further customized native code, which
> Orbition Native Chain Mobile SDKs need. Instead, we'll need to use a custom development build
> which makes Orbition Native Chain Mobile React Native libraries (i.e Mobile Wallet Adapter)
> fully compatible with Expo.
Lastly, and most importantly, Expo does an amazing job providing
[easy-to-use libraries](https://docs.expo.dev/versions/latest/) that give you
access to the device's onboard peripherals, such as camera, battery, and
speakers. The libraries are intuitive and the documentation is phenomenal.
#### How to create an Expo app
To get started with Expo, you first need the prerequisite setup described in the
[Introduction to Orbition Native Chain Mobile lesson](/content/courses/mobile/intro-to-solana-mobile).
After that, you'll want to sign up for an
[Expo Application Services (EAS) account](https://expo.dev/).
Once you have an EAS account, you can install the EAS CLI and log in:
```bash theme={null}
npm install --global eas-cli
eas login
```
Finally, you can scaffold a new Expo app using the `create-expo-app` command:
```bash theme={null}
npx create-expo-app
```
#### How to build and run an Expo app
For some apps, Expo makes building really easy with the Expo Go App. The Expo Go
App builds the project on a remote server and deploys to whatever emulator or
device you specify.
Unfortunately, that won't work with Orbition Native Chain Mobile applications. Instead, you'll
need to build locally. To do that, you need an additional configuration file,
`eas.json`, specifying that the project distribution is "internal." You'll need
the following inside this file:
```json theme={null}
{
"cli": {
"version": ">= 5.2.0"
},
"build": {
"development": {
"developmentClient": true,
"distribution": "internal"
},
"preview": {
"distribution": "internal"
},
"production": {}
},
"submit": {
"production": {}
}
}
```
With the EAS config file created, you can build using the
`npx eas build --local` command plus relevant flags for any additional
requirements. For example, the following will build the project locally with a
development profile specifically for Android:
```bash theme={null}
npx eas build --profile development --platform android --local
```
You then need to install the output APK to your device or emulator. If you're
using an emulator, this is as simple as dragging the APK file onto the emulator
window. If you're using a physical device, you'll have to use Android Debug
Bridge (ADB):
```bash theme={null}
adb install your-apk-file.apk
```
The installed APK is a scaffold app from Expo that facilitates a number of
things, including running your app. To load your application inside of it, you
need to start the development server:
```bash theme={null}
npx expo start --dev-client --android
```
#### How to add Expo SDK packages to your app
The Expo SDK contains packages to simplify all kinds of things related to React
Native development, from UI elements to using device peripherals. You can see
all of the packages on the
[Expo SDK docs](https://docs.expo.dev/versions/latest/).
As an example, you would add
[pedometer functionality](https://docs.expo.dev/versions/latest/sdk/pedometer/)
to your app by installing the `expo-sensors` package:
```bash theme={null}
npx expo install expo-sensors
```
Then you can import it in your code as you would normally expect when using
JS/TS.
```tsx theme={null}
import { Pedometer } from "expo-sensors";
```
Depending on the package, there may be additional setup required. Be sure to
read the [docs](https://docs.expo.dev/versions/latest/) when working with a new
package.
### Integrate ecosystem libraries into your Expo app
Not all React and Node libraries work with React Native out of the box. You
either need to find libraries that are specifically created to work with React
Native or create a workaround yourself.
When working with Orbition Native Chain specifically, the vast majority of ecosystem libraries
do not support React Native out of the box. Fortunately, to get them to play
well in a React Native environment, all we have to do is configure Expo with the
correct [polyfills](https://developer.mozilla.org/en-US/docs/Glossary/Polyfill).
Polyfills are replacement core libraries for environments that are not running
Node.js. Expo does not run Node.js. Unfortunately, it can be tough to know which
polyfills you need for any given application. Unless you know ahead of time,
debugging polyfills means looking at the compiler errors and searching stack
overflow. If it doesn't build, it's normally a polyfill problem.
Fortunately, we've compiled a list of polyfills you'll need for not only some of
the standard Orbition Native Chain libraries but also for Metaplex.
#### Orbition Native Chain Polyfills
For a Orbition Native Chain + Expo app, you'll need the following:
* `@orbition-network-mobile/mobile-wallet-adapter-protocol`: A React Native/Javascript API
enabling interaction with MWA-compatible wallets.
* `@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js`: A convenience wrapper
to use common primitives
from [@orbition-network/web3.js](https://github.com/orbition-network/solana-web3.js) – such
as `Transaction` and `Uint8Array`.
* `@orbition-network/web3.js`: Orbition Native Chain Web Library for interacting with the Orbition Native Chain network
through the [JSON RPC API](https://docs.orbition.network/api/http).
* `react-native-get-random-values`: Secure random number generator polyfill
for `web3.js` underlying Crypto library on React Native.
* `buffer`: Buffer polyfill needed for `web3.js` on React Native.
#### Metaplex Polyfills
If you want to use the Metaplex SDK, you'll need to add the Metaplex library
plus a few additional polyfills:
* `@metaplex-foundation/js@0.19.4` - Metaplex Library
* Several more polyfills
* `assert`
* `util`
* `crypto-browserify`
* `stream-browserify`
* `readable-stream`
* `browserify-zlib`
* `path-browserify`
* `react-native-url-polyfill`
All of the libraries that the above polyfills are meant to replace are utilized
by the Metaplex library in the background. It's unlikely you'll be importing any
of them into your code directly. Because of this, you'll need to register the
polyfills using a `metro.config.js` file. This will ensure that Metaplex uses
the polyfills instead of the usual Node.js libraries that aren't supported in
React Native. Below is an example `metro.config.js` file:
```js theme={null}
const { getDefaultConfig } = require("@expo/metro-config");
const defaultConfig = getDefaultConfig(__dirname);
defaultConfig.resolver.extraNodeModules = {
crypto: require.resolve("crypto-browserify"),
stream: require.resolve("readable-stream"),
url: require.resolve("react-native-url-polyfill"),
zlib: require.resolve("browserify-zlib"),
path: require.resolve("path-browserify"),
};
module.exports = defaultConfig;
```
### Putting it all together
As with most new tools or frameworks, initial setup can be challenging. The good
news is that once you have the application compiling and running, there are very
few differences in the code you write for a web vs mobile app, and there are
virtually no differences when comparing the code you write for a React Native vs
Expo app.
## Lab
Let's practice this together by building the Mint-A-Day app, where users will
able to mint a single NFT snapshot of their lives daily, creating a permanent
diary of sorts.
To mint the NFTs we'll be using Metaplex's Javascript SDK along with
[nft.storage](https://nft.storage/) to store images and metadata. All of our
onchain work will be on Testnet.
The first half of this lab is cobbling together the needed components to make
Expo, Orbition Native Chain, and Metaplex all work together. We'll do this modularly so you'll
know what aspects of the boilerplate align with which section.
### 1. Scaffold, build, and run a local Expo app
In this first section, we will simply get a typescript Expo app running on an
emulator. If you already have a React Native dev environment set up, skip
step 0.
#### 0. Set up React Native dev environment
You'll need React Native installed on your machine as well as a running emulator
or physical device.
[You can accomplish this all with the React Native quickstart](https://reactnative.dev/docs/environment-setup?guide=native).
There are also more details about this setup in the
[Introduction to Orbition Native Chain Mobile lesson](/content/courses/mobile//basic-solana-mobile.md#0-prerequisites)
Even though we are using Expo, you'll need to follow the
React Native cli guide for initial setup.If you are running an emulator, it is highly recommend to
use a newer phone version to emulate along with providing several GB of RAM for
it to run. We use 5GB of ram on our side.
#### 1. Sign up for Expo EAS CLI
To simplify the Expo process, you'll want an Expo Application Services (EAS)
account. This will help you build and run the application.
First sign up for an [EAS account](https://expo.dev/).
Then, install the EAS CLI and log in:
```bash theme={null}
npm install --global eas-cli
eas login
```
#### 2. Create the app scaffold
Let’s create our app with the following:
```bash theme={null}
npx create-expo-app -t expo-template-blank-typescript solana-expo
cd solana-expo
```
This uses `create-expo-app` to generate a new scaffold for us based on the
`expo-template-blank-typescript` template. This is just an empty Typescript
React Native app.
#### 3. Local build config
Expo defaults to building on a remote server but we need to build locally for
Orbition Native Chain Mobile to work correctly. We'll need to add an new config file that lets
the compiler know what we're doing. Create a file called `eas.json` in the root
of your directory.
```bash theme={null}
touch eas.json
```
Copy and paste the following into the newly created `eas.json`:
```json theme={null}
{
"cli": {
"version": ">= 5.2.0"
},
"build": {
"development": {
"developmentClient": true,
"distribution": "internal"
},
"preview": {
"distribution": "internal"
},
"production": {}
},
"submit": {
"production": {}
}
}
```
#### 4. Build and emulate
Now let's build the project. You will choose `y` for every answer. This will
take a while to complete.
```bash theme={null}
npx eas build --profile development --platform android --local
```
When the command is done, you will get an output file at the root of your
directory. This file will have a naming format of `build-XXXXXXXXXXX.apk`.
Locate this file in your file explorer and ***drag it*** into your emulator. A
message should show on the emulator saying that it is installing the new APK.
When it finishes installing, you should see the APK as an app icon in the
emulator.
The app that was installed is just a scaffold app from Expo. The last thing
you'll need to do is run the following command to run the development server:
```bash theme={null}
npx expo start --dev-client --android
```
This should open and run the app in your Android emulator.
***NOTE*** Every time you add in new dependencies, you'll have to build and
re-install the app. Anything visual or logic-based should be captured by the
hot-reloader.
### 2. Configure your Expo app to work with Orbition Native Chain
Now that we have an Expo app up and running, we need to add our Orbition Native Chain
dependencies, including installing a wallet we can use in the emulator. If you
already have a Testnet-enabled wallet installed you can skip step 0.
#### 0. Install a Testnet-enabled Orbition Native Chain wallet
You'll need a wallet that supports Testnet to test with. In
[our Mobile Wallet Adapter lesson](/content/courses/mobile/mwa-deep-dive) we
created one of these. Let's install it from the solution branch in a different
directory from our app:
```bash theme={null}
cd ..
git clone https://github.com/Unboxed-Software/react-native-fake-solana-wallet
cd react-native-fake-solana-wallet
git checkout solution
npm run install
```
The wallet should be installed on your emulator or device. Make sure to open the
newly installed wallet and airdrop yourself some OBN.
Make sure to return to the wallet directory as we'll be working there the rest
of the lab.
```bash theme={null}
cd ..
cd solana-expo
```
#### 1. Install Orbition Native Chain dependencies
We are going to install some basic Orbition Native Chain dependencies that are likely to be
needed by all Orbition Native Chain mobile apps. This will include some polyfills that allow
otherwise incompatible packages to work with React native:
```bash theme={null}
npm install \
@orbition-network/web3.js \
@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js \
@orbition-network-mobile/mobile-wallet-adapter-protocol \
react-native-get-random-values \
buffer
```
#### 3. Add Orbition Native Chain boilerplate providers
Next, let's add some Orbition Native Chain boilerplate that can springboard you into most
Orbition Native Chain-based apps.
Create two new folders: `components` and `screens`.
We are going to use some boilerplate code from the
[first Mobile lesson](/content/courses/mobile/basic-solana-mobile). We will be
copying over `components/AuthProvider.tsx` and
`components/ConnectionProvider.tsx`. These files provide us with a `Connection`
object as well as some helper functions that authorize our dapp.
Create file `components/AuthProvider.tsx` and copy the contents
[of our existing Auth Provider from Github](https://raw.githubusercontent.com/Unboxed-Software/solana-advance-mobile/main/components/AuthProvider.tsx)
into the new file.
Secondly, create file `components/ConnectionProvider.tsx` and copy the contents
[of our existing Connection Provider from Github](https://raw.githubusercontent.com/Unboxed-Software/solana-advance-mobile/main/components/ConnectionProvider.tsx)
into the new file.
Now let's create a boilerplate for our main screen in `screens/MainScreen.tsx`:
```tsx theme={null}
import { View, Text } from "react-native";
import React from "react";
export function MainScreen() {
return (
Orbition Native Chain Expo App
);
}
```
Finally, let's change `App.tsx` to wrap our application in the two providers we
just created:
```tsx theme={null}
import "react-native-get-random-values";
import { StatusBar } from "expo-status-bar";
import { StyleSheet, Text, View } from "react-native";
import { ConnectionProvider } from "./components/ConnectionProvider";
import { AuthorizationProvider } from "./components/AuthProvider";
import { clusterApiUrl } from "@orbition-network/web3.js";
import { MainScreen } from "./screens/MainScreen";
global.Buffer = require("buffer").Buffer;
export default function App() {
const cluster = "testnet";
const endpoint = clusterApiUrl(cluster);
return (
);
}
```
Notice we've added two polyfills above: `buffer` and
`react-native-get-random-values`. These are necessary for the Orbition Native Chain
dependencies to run correctly.
#### 4. Build and run Orbition Native Chain boilerplate
Let's make sure everything is working and compiling correctly. In Expo, anytime
you change the dependencies, you'll need to rebuild and re-install the app.
***Optional:*** To avoid possible build version conflicts, you may want to
*uninstall* the previous version before you drag and drop the new one in.
Build:
```bash theme={null}
npx eas build --profile development --platform android --local
```
Install: ***Drag*** the resulting build file into your emulator.
Run:
```bash theme={null}
npx expo start --dev-client --android
```
Everything should compile and you should have a boilerplate Orbition Native Chain Expo app.
### 3. Configure your Expo app to work with Metaplex
Metaplex is your one-stop-shop for all of your NFT API needs. However, it
requires a little more setup. The good news is if you ever want to fetch, mint
or edit NFTs in your future apps, you'll have another boilerplate to here that
you can reference.
#### 1. Install Metaplex dependencies
The Metaplex SDK abstracts away a lot of the minutia of working with NFTs,
however it was written largely for Node.js, so we'll need several more polyfills
to make it work:
```bash theme={null}
npm install assert \
util \
crypto-browserify \
stream-browserify \
readable-stream \
browserify-zlib \
path-browserify \
react-native-url-polyfill \
@metaplex-foundation/js@0.19.4
```
#### 2. Polyfill config
We won't be importing any of the above polyfills in our code directly, so we
need to add them to a `metro.config.js` file to ensure that Metaplex uses them:
```bash theme={null}
touch metro.config.js
```
Copy and paste the following into `metro.config.js`:
```js theme={null}
// Import the default Expo Metro config
const { getDefaultConfig } = require("@expo/metro-config");
// Get the default Expo Metro configuration
const defaultConfig = getDefaultConfig(__dirname);
// Customize the configuration to include your extra node modules
defaultConfig.resolver.extraNodeModules = {
crypto: require.resolve("crypto-browserify"),
stream: require.resolve("readable-stream"),
url: require.resolve("react-native-url-polyfill"),
zlib: require.resolve("browserify-zlib"),
path: require.resolve("path-browserify"),
};
// Export the modified configuration
module.exports = defaultConfig;
```
#### 3. Metaplex provider
We're going to create a Metaplex provider file that will help us access a
`Metaplex` object. This `Metaplex` object is what gives us access to all of the
functions we'll need like `fetch` and `create`. To do this we create a new file
`/components/MetaplexProvider.tsx`. Here we pipe our mobile wallet adapter into
an `IdentitySigner` for the `Metaplex` object to use. This allows it to call
several privileged functions on our behalf:
```tsx theme={null}
import {
IdentitySigner,
Metaplex,
MetaplexPlugin,
} from "@metaplex-foundation/js";
import {
transact,
Web3MobileWallet,
} from "@orbition-network-mobile/mobile-wallet-adapter-protocol-web3js";
import { Connection, Transaction } from "@orbition-network/web3.js";
import { useMemo } from "react";
import { Account } from "./AuthProvider";
export const mobileWalletAdapterIdentity = (
mwaIdentitySigner: IdentitySigner,
): MetaplexPlugin => ({
install(metaplex: Metaplex) {
metaplex.identity().setDriver(mwaIdentitySigner);
},
});
export const useMetaplex = (
connection: Connection,
selectedAccount: Account | null,
authorizeSession: (wallet: Web3MobileWallet) => Promise,
) => {
return useMemo(() => {
if (!selectedAccount || !authorizeSession) {
return { mwaIdentitySigner: null, metaplex: null };
}
const mwaIdentitySigner: IdentitySigner = {
publicKey: selectedAccount.publicKey,
signMessage: async (message: Uint8Array): Promise => {
return await transact(async (wallet: Web3MobileWallet) => {
await authorizeSession(wallet);
const signedMessages = await wallet.signMessages({
addresses: [selectedAccount.publicKey.toBase58()],
payloads: [message],
});
return signedMessages[0];
});
},
signTransaction: async (
transaction: Transaction,
): Promise => {
return await transact(async (wallet: Web3MobileWallet) => {
await authorizeSession(wallet);
const signedTransactions = await wallet.signTransactions({
transactions: [transaction],
});
return signedTransactions[0];
});
},
signAllTransactions: async (
transactions: Transaction[],
): Promise => {
return transact(async (wallet: Web3MobileWallet) => {
await authorizeSession(wallet);
const signedTransactions = await wallet.signTransactions({
transactions: transactions,
});
return signedTransactions;
});
},
};
const metaplex = Metaplex.make(connection).use(
mobileWalletAdapterIdentity(mwaIdentitySigner),
);
return { metaplex };
}, [authorizeSession, selectedAccount, connection]);
};
```
#### 4. NFT Provider
We're also making a higher-level NFT provider that helps with NFT state
management. It combines all three of our previous providers:
`ConnectionProvider`, `AuthProvider` and `MetaplexProvider` to allow us to
create our `Metaplex` object. We will fill this out in a later step, for now, it
makes for a good boilerplate.
Let's create the new file `components/NFTProvider.tsx`:
```tsx theme={null}
import "react-native-url-polyfill/auto";
import { useConnection } from "./ConnectionProvider";
import { Account, useAuthorization } from "./AuthProvider";
import React, { ReactNode, createContext, useContext, useState } from "react";
import { useMetaplex } from "./MetaplexProvider";
export interface NFTProviderProps {
children: ReactNode;
}
export interface NFTContextState {}
const DEFAULT_NFT_CONTEXT_STATE: NFTContextState = {};
const NFTContext = createContext(DEFAULT_NFT_CONTEXT_STATE);
export function NFTProvider(props: NFTProviderProps) {
const { children } = props;
const { connection } = useConnection();
const { authorizeSession } = useAuthorization();
const [account, setAccount] = useState(null);
const { metaplex } = useMetaplex(connection, account, authorizeSession);
const state = {};
return {children};
}
export const useNFT = (): NFTContextState => useContext(NFTContext);
```
Notice we've added yet another polyfill to the top
`import "react-native-url-polyfill/auto";`
#### 5. Wrap provider
Now, let's wrap our new `NFTProvider` around `MainScreen` in `App.tsx`:
```tsx theme={null}
import "react-native-get-random-values";
import { ConnectionProvider } from "./components/ConnectionProvider";
import { AuthorizationProvider } from "./components/AuthProvider";
import { clusterApiUrl } from "@orbition-network/web3.js";
import { MainScreen } from "./screens/MainScreen";
import { NFTProvider } from "./components/NFTProvider";
global.Buffer = require("buffer").Buffer;
export default function App() {
const cluster = "testnet";
const endpoint = clusterApiUrl(cluster);
return (
);
}
```
#### 6. Build and run
Lastly, let's build and re-install the app to make sure things are still
working.
Build:
```bash theme={null}
npx eas build --profile development --platform android --local
```
Install:
***Drag*** the resulting build file into your emulator.
Run:
```bash theme={null}
npx expo start --dev-client --android
```
### 4. Configure your Expo app to take and upload photos
Everything we've done to this point is effectively boilerplate we needed to
start adding the functionality we intend our Mint-A-Day app to have. Mint-A-day
is a daily snapshot app. It lets users take a snapshot of their life daily in
the form of minting an NFT.
The app will need access to the device's camera and a place to remotely store
the captured images. Fortunately, Expo SDK can provide access to the camera and
[NFT.Storage](https://nft.storage) can store your NFT files for free.
#### 1. Camera setup
Let's start by setting up the Expo-specific dependency we'll be using:
`expo-image-picker`. This lets us use the device's camera to take pictures that
we'll subsequently turn into NFTs. We're specifically using the image picker
rather than the camera since emulators don't have cameras. This package will
simulate a camera for us in the emulator. Install it with the following command:
```bash theme={null}
npx expo install expo-image-picker
```
In addition to installation, the `expo-image-picker` package needs to be added
as a plugin in `app.json`:
```json theme={null}
"expo": {
// ....
"plugins": [
[
"expo-image-picker",
{
"photosPermission": "Allows you to use images to create solana NFTs"
}
]
],
// ....
}
```
This particular dependency makes it super simple to use the camera. To allow the
user to take a picture and return the image all you have to do is call the
following:
```tsx theme={null}
const result = await ImagePicker.launchCameraAsync({
mediaTypes: ImagePicker.MediaTypeOptions.Images,
allowsEditing: true,
aspect: [1, 1],
quality: 1,
});
```
No need to add this anywhere yet - we'll get to it in a few steps.
#### 2. NFT.Storage setup
The last thing we need to do is set up our access to
[nft.storage](https://nft.storage). We'll need to get an API key and add it as
an environment variable, then we need to add one last dependency to convert our
images into a file type we can upload.
We'll be using NFT.storage to host our NFTs with IPFS since they do this for
free. [Sign up, and create an API key](https://nft.storage/manage/). Keep this
API key private.
Best practices suggest keeping API keys in a `.env` file with `.env` added to
your `.gitignore`. It's also a good idea to create a `.env.example` file that
can be committed to your repo and shows what environment variables are needed
for the project.
Create both files, in the root of your directory and add `.env` to your
`.gitignore` file.
Then, add your API key to the `.env` file with the name
`EXPO_PUBLIC_NFT_STORAGE_API`. Now you'll be able to access your API key safely
in the application.
Lastly, install `rn-fetch-blob`. This package will help us grab images from the
device's URI scheme and turn them into Blobs we can the upload to
[NFT.storage](https://nft.storage).
Install it with the following:
```bash theme={null}
npm i rn-fetch-blob
```
#### 3. Final build
Build and reinstall if you want to make sure it's all working. This is the last
time we'll have to do this for this lab. Everything else should be hot-loadable.
Build:
```bash theme={null}
npx eas build --profile development --platform android --local
```
Install:
***Drag*** the resulting build file into your emulator.
Run:
```bash theme={null}
npx expo start --dev-client --android
```
### 5. Add functionality to complete your Expo app
We're through the setup! Let's create the actual functionality for our
Mint-A-Day app. Fortunately, there are only two files we have to focus on now:
* `NFTProvider.tsx` will largely manage our app state and NFT data.
* `MainScreen.tsx` will capture input and show our NFTs
The app itself is relatively straightforward. The general flow is:
1. The user connects (authorizes) using the `transact` function and by calling
`authorizeSession` inside the callback
2. Our code then uses the `Metaplex` object to fetch all of the NFTs created by
the user
3. If an NFT has not been created for the current day, allow the user to take a
picture, upload it, and mint it as an NFT
#### 1. NFT Provider
`NFTProvider.tsx` will control the state with our custom `NFTProviderContext`.
This should have the following fields:
* `metaplex: Metaplex | null` - Holds the metaplex object that we use to call
`fetch` and `create`
* `publicKey: PublicKey | null` - The NFT creator's public key
* `isLoading: boolean` - Manages loading state
* `loadedNFTs: (Nft | Sft | SftWithToken | NftWithToken)[] | null` - An array of
the user's snapshot NFTs
* `nftOfTheDay: (Nft | Sft | SftWithToken | NftWithToken) | null` - A reference
to the NFT created today
* `connect: () => void` - A function for connecting to the Testnet-enabled wallet
* `fetchNFTs: () => void` - A function that fetches the user's snapshot NFTs
* `createNFT: (name: string, description: string, fileUri: string) => void` - A
function that creates a new snapshot NFT
```tsx theme={null}
export interface NFTContextState {
metaplex: Metaplex | null; // Holds the metaplex object that we use to call `fetch` and `create` on.
publicKey: PublicKey | null; // The public key of the authorized wallet
isLoading: boolean; // Loading state
loadedNFTs: (Nft | Sft | SftWithToken | NftWithToken)[] | null; // Array of loaded NFTs that contain metadata
nftOfTheDay: (Nft | Sft | SftWithToken | NftWithToken) | null; // The NFT snapshot created on the current day
connect: () => void; // Connects (and authorizes) us to the Testnet-enabled wallet
fetchNFTs: () => void; // Fetches the NFTs using the `metaplex` object
createNFT: (name: string, description: string, fileUri: string) => void; // Creates the NFT
}
```
The state flow here is: `connect`, `fetchNFTs`, and then `createNFT`. We'll walk
through the code for each of them and then show you the entire file at the end:
1. `connect` - This function will connect and authorize the app, and then store
the resulting `publicKey` into the state.
```tsx theme={null}
const connect = () => {
if (isLoading) return;
setIsLoading(true);
transact(async wallet => {
const auth = await authorizeSession(wallet);
setAccount(auth);
}).finally(() => {
setIsLoading(false);
});
};
```
2. `fetchNFTs` - This function will fetch the NFTs using Metaplex:
```tsx theme={null}
const fetchNFTs = async () => {
if (!metaplex || !account || isLoading) return;
setIsLoading(true);
try {
const nfts = await metaplex.nfts().findAllByCreator({
creator: account.publicKey,
});
const loadedNFTs = await Promise.all(
nfts.map(nft => {
return metaplex.nfts().load({ metadata: nft as Metadata });
}),
);
setLoadedNFTs(loadedNFTs);
// Check if we already took a snapshot today
const nftOfTheDayIndex = loadedNFTs.findIndex(nft => {
return formatDate(new Date(Date.now())) === nft.name;
});
if (nftOfTheDayIndex !== -1) {
setNftOfTheDay(loadedNFTs[nftOfTheDayIndex]);
}
} catch (error) {
console.log(error);
} finally {
setIsLoading(false);
}
};
```
3. `createNFT` - This function will upload a file to NFT.Storage, and then use
Metaplex to create and mint an NFT to your wallet. This comes in three parts,
uploading the image, uploading the metadata and then minting the NFT.
To upload to NFT.Storage you just make a POST with your API key and the
image/metadata as the body.
We'll create two helper functions for uploading the image and metadata
separately, then tie them together into a single `createNFT` function:
```tsx theme={null}
// https://nft.storage/api-docs/
const uploadImage = async (fileUri: string): Promise => {
const imageBytesInBase64: string = await RNFetchBlob.fs.readFile(
fileUri,
"base64",
);
const bytes = Buffer.from(imageBytesInBase64, "base64");
const response = await fetch("https://api.nft.storage/upload", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.EXPO_PUBLIC_NFT_STORAGE_API}`,
"Content-Type": "image/jpg",
},
body: bytes,
});
const data = await response.json();
const cid = data.value.cid;
return cid as string;
};
const uploadMetadata = async (
name: string,
description: string,
imageCID: string,
): Promise => {
const response = await fetch("https://api.nft.storage/upload", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.EXPO_PUBLIC_NFT_STORAGE_API}`,
},
body: JSON.stringify({
name,
description,
image: `https://ipfs.io/ipfs/${imageCID}`,
}),
});
const data = await response.json();
const cid = data.value.cid;
return cid;
};
```
Minting the NFT after the image and metadata have been uploaded is as simple
as calling `metaplex.nfts().create(...)`. Below shows the `createNFT`
function tying everything together:
```tsx theme={null}
const createNFT = async (
name: string,
description: string,
fileUri: string,
) => {
if (!metaplex || !account || isLoading) return;
setIsLoading(true);
try {
const imageCID = await uploadImage(fileUri);
const metadataCID = await uploadMetadata(name, description, imageCID);
const nft = await metaplex.nfts().create({
uri: `https://ipfs.io/ipfs/${metadataCID}`,
name: name,
sellerFeeBasisPoints: 0,
});
setNftOfTheDay(nft.nft);
} catch (error) {
console.log(error);
} finally {
setIsLoading(false);
}
};
```
We'll put all of the above into the `NFTProvider.tsx` file. All together, this
looks as follows:
```tsx theme={null}
import "react-native-url-polyfill/auto";
import React, { ReactNode, createContext, useContext, useState } from "react";
import {
Metaplex,
PublicKey,
Metadata,
Nft,
Sft,
SftWithToken,
NftWithToken,
} from "@metaplex-foundation/js";
import { useConnection } from "./ConnectionProvider";
import { Connection, clusterApiUrl } from "@orbition-network/web3.js";
import { transact } from "@orbition-network-mobile/mobile-wallet-adapter-protocol";
import { Account, useAuthorization } from "./AuthProvider";
import RNFetchBlob from "rn-fetch-blob";
import { useMetaplex } from "./MetaplexProvider";
export interface NFTProviderProps {
children: ReactNode;
}
export interface NFTContextState {
metaplex: Metaplex | null;
publicKey: PublicKey | null;
isLoading: boolean;
loadedNFTs: (Nft | Sft | SftWithToken | NftWithToken)[] | null;
nftOfTheDay: (Nft | Sft | SftWithToken | NftWithToken) | null;
connect: () => void;
fetchNFTs: () => void;
createNFT: (name: string, description: string, fileUri: string) => void;
}
const DEFAULT_NFT_CONTEXT_STATE: NFTContextState = {
metaplex: new Metaplex(new Connection(clusterApiUrl("testnet"))),
publicKey: null,
isLoading: false,
loadedNFTs: null,
nftOfTheDay: null,
connect: () => PublicKey.default,
fetchNFTs: () => {},
createNFT: (name: string, description: string, fileUri: string) => {},
};
const NFTContext = createContext(DEFAULT_NFT_CONTEXT_STATE);
export function formatDate(date: Date) {
return `${date.getDate()}.${date.getMonth()}.${date.getFullYear()}`;
}
export function NFTProvider(props: NFTProviderProps) {
const { children } = props;
const { connection } = useConnection();
const { authorizeSession } = useAuthorization();
const [account, setAccount] = useState(null);
const [isLoading, setIsLoading] = useState(false);
const [nftOfTheDay, setNftOfTheDay] = useState<
(Nft | Sft | SftWithToken | NftWithToken) | null
>(null);
const [loadedNFTs, setLoadedNFTs] = useState<
(Nft | Sft | SftWithToken | NftWithToken)[] | null
>(null);
const { metaplex } = useMetaplex(connection, account, authorizeSession);
const connect = () => {
if (isLoading) return;
setIsLoading(true);
transact(async wallet => {
const auth = await authorizeSession(wallet);
setAccount(auth);
}).finally(() => {
setIsLoading(false);
});
};
const fetchNFTs = async () => {
if (!metaplex || !account || isLoading) return;
setIsLoading(true);
try {
const nfts = await metaplex.nfts().findAllByCreator({
creator: account.publicKey,
});
const loadedNFTs = await Promise.all(
nfts.map(nft => {
return metaplex.nfts().load({ metadata: nft as Metadata });
}),
);
setLoadedNFTs(loadedNFTs);
// Check if we already took a snapshot today
const nftOfTheDayIndex = loadedNFTs.findIndex(nft => {
return formatDate(new Date(Date.now())) === nft.name;
});
if (nftOfTheDayIndex !== -1) {
setNftOfTheDay(loadedNFTs[nftOfTheDayIndex]);
}
} catch (error) {
console.log(error);
} finally {
setIsLoading(false);
}
};
// https://nft.storage/api-docs/
const uploadImage = async (fileUri: string): Promise => {
const imageBytesInBase64: string = await RNFetchBlob.fs.readFile(
fileUri,
"base64",
);
const bytes = Buffer.from(imageBytesInBase64, "base64");
const response = await fetch("https://api.nft.storage/upload", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.EXPO_PUBLIC_NFT_STORAGE_API}`,
"Content-Type": "image/jpg",
},
body: bytes,
});
const data = await response.json();
const cid = data.value.cid;
return cid as string;
};
const uploadMetadata = async (
name: string,
description: string,
imageCID: string,
): Promise => {
const response = await fetch("https://api.nft.storage/upload", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.EXPO_PUBLIC_NFT_STORAGE_API}`,
},
body: JSON.stringify({
name,
description,
image: `https://ipfs.io/ipfs/${imageCID}`,
}),
});
const data = await response.json();
const cid = data.value.cid;
return cid;
};
const createNFT = async (
name: string,
description: string,
fileUri: string,
) => {
if (!metaplex || !account || isLoading) return;
setIsLoading(true);
try {
const imageCID = await uploadImage(fileUri);
const metadataCID = await uploadMetadata(name, description, imageCID);
const nft = await metaplex.nfts().create({
uri: `https://ipfs.io/ipfs/${metadataCID}`,
name: name,
sellerFeeBasisPoints: 0,
});
setNftOfTheDay(nft.nft);
} catch (error) {
console.log(error);
} finally {
setIsLoading(false);
}
};
const publicKey = account?.publicKey ?? null;
const state = {
isLoading,
account,
publicKey,
metaplex,
nftOfTheDay,
loadedNFTs,
connect,
fetchNFTs,
createNFT,
};
return {children};
}
export const useNFT = (): NFTContextState => useContext(NFTContext);
```
#### 2. Main Screen
Our main screen will consist of three parts: The image of the day, our action
button, and the carousel of previous snapshots.
The image of the day is displayed on the top half of the app, the action button
right under it, and the carousel under that.
The action button follows the state of our `NFTProvider`: first `connect`, then
`fetchNFTs`, and finally `mintNFT`. Of these, we only need to do some extra work
for `mintNFT`.
The `mintNFT` function uses the Expo library to open up the camera with
`ImagePicker.launchCameraAsync`. When an image is taken, it's local path is
returned. The last thing we need to do is specify when the image was taken. Then
we'll make the name of the NFT the date in `MM.DD.YY` format and store the unix
timestamp as the description. Finally, we pass the image path, name and
description to our `createNFT` function from `NFTProvider` to mint the NFT.
```tsx theme={null}
const mintNFT = async () => {
const result = await ImagePicker.launchCameraAsync({
mediaTypes: ImagePicker.MediaTypeOptions.Images,
allowsEditing: true,
aspect: [1, 1],
quality: 1,
});
if (!result.canceled) {
setCurrentImage({
uri: result.assets[0].uri,
date: todaysDate,
});
createNFT(
formatDate(todaysDate),
`${todaysDate.getTime()}`,
result.assets[0].uri,
);
}
};
```
The full code for `MainScreen.tsx` is as follows:
```tsx theme={null}
import {
View,
Button,
Image,
StyleSheet,
ScrollView,
Text,
} from "react-native";
import React, { useEffect } from "react";
import { formatDate, useNFT } from "../components/NFTProvider";
import * as ImagePicker from "expo-image-picker";
const styles = StyleSheet.create({
container: {
flex: 1,
backgroundColor: "#292524",
},
titleText: {
color: "white",
},
topSection: {
flex: 1,
justifyContent: "center",
alignItems: "center",
textAlign: "center",
paddingTop: 30,
},
imageOfDay: {
width: "80%",
height: "80%",
resizeMode: "cover",
margin: 10,
},
bottomSection: {
flex: 1,
justifyContent: "center",
alignItems: "center",
},
carousel: {
justifyContent: "center",
alignItems: "center",
},
carouselText: {
textAlign: "center",
color: "white",
},
carouselImage: {
width: 100,
height: 100,
margin: 5,
resizeMode: "cover",
},
});
export interface NFTSnapshot {
uri: string;
date: Date;
}
// Placeholder image URL or local source
const PLACEHOLDER: NFTSnapshot = {
uri: "https://placehold.co/400x400/png",
date: new Date(Date.now()),
};
const DEFAULT_IMAGES: NFTSnapshot[] = new Array(7).fill(PLACEHOLDER);
export function MainScreen() {
const {
fetchNFTs,
connect,
publicKey,
isLoading,
createNFT,
loadedNFTs,
nftOfTheDay,
} = useNFT();
const [currentImage, setCurrentImage] =
React.useState(PLACEHOLDER);
const [previousImages, setPreviousImages] =
React.useState(DEFAULT_IMAGES);
const todaysDate = new Date(Date.now());
useEffect(() => {
if (!loadedNFTs) return;
const loadedSnapshots = loadedNFTs.map(loadedNft => {
if (!loadedNft.json) return null;
if (!loadedNft.json.name) return null;
if (!loadedNft.json.description) return null;
if (!loadedNft.json.image) return null;
const uri = loadedNft.json.image;
const unixTime = Number(loadedNft.json.description);
if (!uri) return null;
if (isNaN(unixTime)) return null;
return {
uri: loadedNft.json.image,
date: new Date(unixTime),
} as NFTSnapshot;
});
// Filter out null values
const cleanedSnapshots = loadedSnapshots.filter(loadedSnapshot => {
return loadedSnapshot !== null;
}) as NFTSnapshot[];
// Sort by date
cleanedSnapshots.sort((a, b) => {
return b.date.getTime() - a.date.getTime();
});
setPreviousImages(cleanedSnapshots as NFTSnapshot[]);
}, [loadedNFTs]);
useEffect(() => {
if (!nftOfTheDay) return;
setCurrentImage({
uri: nftOfTheDay.json?.image ?? "",
date: todaysDate,
});
}, [nftOfTheDay]);
const mintNFT = async () => {
const result = await ImagePicker.launchCameraAsync({
mediaTypes: ImagePicker.MediaTypeOptions.Images,
allowsEditing: true,
aspect: [1, 1],
quality: 1,
});
if (!result.canceled) {
setCurrentImage({
uri: result.assets[0].uri,
date: todaysDate,
});
createNFT(
formatDate(todaysDate),
`${todaysDate.getTime()}`,
result.assets[0].uri,
);
}
};
const handleNFTButton = async () => {
if (!publicKey) {
connect();
} else if (loadedNFTs === null) {
fetchNFTs();
} else if (!nftOfTheDay) {
mintNFT();
} else {
alert("All done for the day!");
}
};
const renderNFTButton = () => {
let buttonText = "";
if (!publicKey) buttonText = "Connect Wallet";
else if (loadedNFTs === null) buttonText = "Fetch NFTs";
else if (!nftOfTheDay) buttonText = "Create Snapshot";
else buttonText = "All Done!";
if (isLoading) buttonText = "Loading...";
return ;
};
const renderPreviousSnapshot = (snapshot: NFTSnapshot, index: number) => {
const date = snapshot.date;
const formattedDate = formatDate(date);
return (
{formattedDate}
);
};
return (
{/* Top Half */}
Mint-A-Day
{renderNFTButton()}
{/* Bottom Half */}
{previousImages.map(renderPreviousSnapshot)}
);
}
```
#### 3. Test
Now it's time to create our first snapshot! First, open up your Testnet-enabled
wallet and make sure you have some OBN. Next, tap on `Connect Wallet` and
approve the app. Fetch all of the NFTs by tapping `Fetch NFTs`. Lastly, tap
`Create Snapshot` to upload and mint.
Congratulations! That was not an easy or quick lab. You're doing great if you've
made it this far. If you run into any issues, feel free to go back through the
lab and/or reference the final solution code on the
[`main` branch in Github](https://github.com/Unboxed-Software/solana-advance-mobile).
## Challenge
Now it's your turn. Create your own Expo application from scratch. You're
welcome to choose your own, or you can select from the following ideas:
* Instead of a daily image snapshot, create an application that lets users write
a journal entry for the day, then mint it as an NFT
* Create a basic NFT viewer app to see all your wonderful JPEGs
* Make a simplified clone of [Stepn](https://stepn.com/) using the pedometer
from `expo-sensors`
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=19cf8d3a-89a0-465e-95da-908cf8f45409)!
# Cross Program Invocations
Source: https://docs.orbition.network/courses/native-onchain-development/cross-program-invocations
How to invoke functions in other Orbition Native Chain programs.
## Summary
* A **Cross-Program Invocation (CPI)** is a call from one program to another,
targeting a specific instruction on the program called
* CPIs are made using the commands `invoke` or `invoke_signed`, the latter being
how programs provide signatures for PDAs that they own
* CPIs make programs in the Orbition Native Chain ecosystem completely interoperable because
all public instructions of a program can be invoked by another program via a
CPI
* Because we have no control over the accounts and data submitted to a program,
it's important to verify all of the parameters passed into a CPI to ensure
program security
## Lesson
### What is a CPI?
A Cross-Program Invocation (CPI) is a direct call from one program into another.
Just as any client can call any program using the JSON RPC, any program can call
any other program directly. The only requirement for invoking an instruction on
another program from within your program is that you construct the instruction
correctly. You can make CPIs to native programs, other programs you've created,
and third party programs. CPIs essentially turn the entire Orbition Native Chain ecosystem into
one giant API that is at your disposal as a developer.
CPIs have a similar make up to instructions that you are used to creating client
side. There are some intricacies and differences depending on if you are using
`invoke` or `invoke_signed`. We'll be covering both of these later in this
lesson.
### How to make a CPI
CPIs are made using the
[`invoke`](https://docs.rs/solana-program/1.10.19/solana_program/program/fn.invoke.html)
or
[`invoke_signed`](https://docs.rs/solana-program/1.10.19/solana_program/program/fn.invoke_signed.html)
function from the `solana_program` crate. You use `invoke` to essentially pass
through the original transaction signature that was passed into your program.
You use `invoke_signed` to have your program "sign" for its PDAs.
```rust theme={null}
// Used when there are not signatures for PDAs needed
pub fn invoke(
instruction: &Instruction,
account_infos: &[AccountInfo<'_>]
) -> ProgramResult
// Used when a program must provide a 'signature' for a PDA, hence the signer_seeds parameter
pub fn invoke_signed(
instruction: &Instruction,
account_infos: &[AccountInfo<'_>],
signers_seeds: &[&[&[u8]]]
) -> ProgramResult
```
CPIs extend the privileges of the caller to the callee. If the instruction the
callee program is processing contains an account that was marked as a signer or
writable when originally passed into the caller program, then it will be
considered a signer or writable account in the invoked program as well.
It's important to note that you as the developer decide which accounts to pass
into the CPI. You can think of a CPI as building another instruction from
scratch with only information that was passed into your program.
#### CPI with `invoke`
```rust theme={null}
invoke(
&Instruction {
program_id: calling_program_id,
accounts: accounts_meta,
data,
},
&account_infos[account1.clone(), account2.clone(), account3.clone()],
)?;
```
* `program_id` - the public key of the program you are going to invoke
* `account` - a list of account metadata as a vector. You need to include every
account that the invoked program will read or write
* `data` - a byte buffer representing the data being passed to the callee
program as a vector
The `Instruction` type has the following definition:
```rust theme={null}
pub struct Instruction {
pub program_id: Pubkey,
pub accounts: Vec,
pub data: Vec,
}
```
Depending on the program you're making the call to, there may be a crate
available with helper functions for creating the `Instruction` object. Many
individuals and organizations create publicly available crates alongside their
programs that expose these sorts of functions to simplify calling their
programs. This is similar to the Typescript libraries we've used in this course
(e.g. [@orbition-network/web3.js](https://orbition-network.github.io/solana-web3.js/),
[@orbition-network/spl-token](https://orbition-network.github.io/solana-program-library/token/js/)).
For example, in this lesson's lab we'll be using the `spl_token` crate to create
minting instructions. In all other cases, you'll need to create the
`Instruction` instance from scratch.
While the `program_id` field is fairly straightforward, the `accounts` and
`data` fields require some explanation.
Both the `accounts` and `data` fields are of type `Vec`, or vector. You can use
the [`vec`](https://doc.rust-lang.org/std/macro.vec.html) macro to construct a
vector using array notation, like so:
```rust theme={null}
let v = vec![1, 2, 3];
assert_eq!(v[0], 1);
assert_eq!(v[1], 2);
assert_eq!(v[2], 3);
```
The `accounts` field of the `Instruction` struct expects a vector of type
[`AccountMeta`](https://docs.rs/solana-program/latest/solana_program/instruction/struct.AccountMeta.html).
The `AccountMeta` struct has the following definition:
```rust theme={null}
pub struct AccountMeta {
pub pubkey: Pubkey,
pub is_signer: bool,
pub is_writable: bool,
}
```
Putting these two pieces together looks like this:
```rust theme={null}
use solana_program::instruction::AccountMeta;
vec![
AccountMeta::new(account1_pubkey, true), // metadata for a writable, signer account
AccountMeta::read_only(account2_pubkey, false), // metadata for a read-only, non-signer account
AccountMeta::read_only(account3_pubkey, true), // metadata for a read-only, signer account
AccountMeta::new(account4_pubkey, false), // metadata for a writable, non-signer account
]
```
The final field of the instruction object is the data, as a byte buffer of
course. You can create a byte buffer in Rust using the `vec` macro again, which
has an implemented function allowing you to create a vector of certain length.
Once you have initialized an empty vector, you would construct the byte buffer
similar to how you would client-side. Determine the data required by the callee
program and the serialization format used and write your code to match. Feel
free to read up on some of the
[features of the `vec` macro available to you here](https://doc.rust-lang.org/alloc/vec/struct.Vec.html#).
```rust theme={null}
let mut vec = Vec::with_capacity(3);
vec.push(1);
vec.push(2);
vec.extend_from_slice(&number_variable.to_le_bytes());
```
The
[`extend_from_slice`](https://doc.rust-lang.org/alloc/vec/struct.Vec.html#method.extend_from_slice)
method is probably new to you. It's a method on vectors that takes a slice as
input, iterates over the slice, clones each element, and then appends it to the
`Vec`.
#### Pass a list of accounts
In addition to the instruction, both `invoke` and `invoke_signed` also require a
list of `account_info` objects. Just like the list of `AccountMeta` objects you
added to the instruction, you must include all of the accounts that the program
you're calling will read or write.
By the time you make a CPI in your program, you should have already grabbed all
the `account_info` objects that were passed into your program and stored them in
variables. You'll construct your list of `account_info` objects for the CPI by
choosing which of these accounts to copy and send along.
You can copy each `account_info` object that you need to pass into the CPI using
the
[`Clone`](https://docs.rs/solana-program/1.10.19/solana_program/account_info/struct.AccountInfo.html#impl-Clone)
trait that is implemented on the `account_info` struct in the `solana_program`
crate. This `Clone` trait returns a copy of the
[`account_info`](https://docs.rs/solana-program/1.10.19/solana_program/account_info/struct.AccountInfo.html)
instance.
```rust theme={null}
&[first_account.clone(), second_account.clone(), third_account.clone()]
```
#### CPI with `invoke`
With both the instruction and the list of accounts created, you can perform a
call to `invoke`.
```rust theme={null}
invoke(
&Instruction {
program_id: calling_program_id,
accounts: accounts_meta,
data,
},
&[account1.clone(), account2.clone(), account3.clone()],
)?;
```
There's no need to include a signature because the Orbition Native Chain runtime passes along
the original signature passed into your program. Remember, `invoke` won't work
if a signature is required on behalf of a PDA. For that, you'll need to use
`invoke_signed`.
#### CPI with `invoke_signed`
Using `invoke_signed` is a little different just because there is an additional
field that requires the seeds used to derive any PDAs that must sign the
transaction. You may recall from previous lessons that PDAs do not lie on the
Ed25519 curve and, therefore, do not have a corresponding secret key. You’ve
been told that programs can provide signatures for their PDAs, but have not
learned how that actually happens - until now. Programs provide signatures for
their PDAs with the `invoke_signed` function. The first two fields of
`invoke_signed` are the same as `invoke`, but there is an additional
`signers_seeds` field that comes into play here.
```rust theme={null}
invoke_signed(
&instruction,
accounts,
&[&["First addresses seed"],
&["Second addresses first seed",
"Second addresses second seed"]],
)?;
```
While PDAs have no secret keys of their own, they can be used by a program to
issue an instruction that includes the PDA as a signer. The only way for the
runtime to verify that the PDA belongs to the calling program is for the calling
program to supply the seeds used to generate the address in the `signers_seeds`
field.
The Orbition Native Chain runtime will internally
call [`create_program_address`](https://docs.rs/solana-program/1.4.4/solana_program/pubkey/struct.Pubkey.html#method.create_program_address)
using the seeds provided and the `program_id` of the calling program. It can
then compare the result against the addresses supplied in the instruction. If
any of the addresses match, then the runtime knows that indeed the program
associated with this address is the caller and thus is authorized to be a
signer.
### Best Practices and common pitfalls
#### Security checks
There are some common mistakes and things to remember when utilizing CPIs that
are important to your program’s security and robustness. The first thing to
remember is that, as we know by now, we have no control over what information is
passed into our programs. For this reason, it’s important to always verify the
`program_id`, accounts, and data passed into the CPI. Without these security
checks, someone could submit a transaction that invokes an instruction on a
completely different program than was expected, which is not ideal.
Fortunately, there are inherent checks on the validity of any PDAs that are
marked as signers within the `invoke_signed` function. All other accounts and
`instruction_data` should be verified somewhere in your program code before
making the CPI. It's also important to make sure you’re targeting the intended
instruction on the program you are invoking. The easiest way to do this is to
read the source code of the program you will be invoking just as you would if
you were constructing an instruction from the client side.
#### Common errors
There are some common errors you might receive when executing a CPI, they
usually mean you are constructing the CPI with incorrect information. For
example, you may come across an error message similar to this:
```text theme={null}
EF1M4SPfKcchb6scq297y8FPCaLvj5kGjwMzjTM68wjA's signer privilege escalated
Program returned error: "Cross-program invocation with unauthorized signer or writable account"
```
This message is a little misleading, because “signer privilege escalated” does
not seem like a problem but, in reality, it means that you are incorrectly
signing for the address in the message. If you are using `invoke_signed` and
receive this error, then it likely means that the seeds you are providing are
incorrect. You can also find
[an example transaction that failed with this error](https://explorer.orbition.network/tx/3mxbShkerH9ZV1rMmvDfaAhLhJJqrmMjcsWzanjkARjBQurhf4dounrDCUkGunH1p9M4jEwef9parueyHVw6r2Et?cluster=testnet).
Another similar error is thrown when an account that's written to isn't marked
as `writable` inside the `AccountMeta` struct.
```text theme={null}
2qoeXa9fo8xVHzd2h9mVcueh6oK3zmAiJxCTySM5rbLZ's writable privilege escalated
Program returned error: "Cross-program invocation with unauthorized signer or writable account"
```
Remember, any account whose data may be mutated by the program during execution
must be specified as writable. During execution, writing to an account that was
not specified as writable will cause the transaction to fail. Writing to an
account that is not owned by the program will cause the transaction to fail. Any
account whose lamport balance may be mutated by the program during execution
must be specified as writable. During execution, mutating the lamports of an
account that was not specified as writable will cause the transaction to fail.
While subtracting lamports from an account not owned by the program will cause
the transaction to fail, adding lamports to any account is allowed, as long is
it is mutable.
To see this in action, view this
[transaction in the explorer](https://explorer.orbition.network/tx/ExB9YQJiSzTZDBqx4itPaa4TpT8VK4Adk7GU5pSoGEzNz9fa7PPZsUxssHGrBbJRnCvhoKgLCWnAycFB7VYDbBg?cluster=testnet).
### Why CPIs matter?
CPIs are a very important feature of the Orbition Native Chain ecosystem and they make all
programs deployed interoperable with each other. With CPIs there is no need to
re-invent the wheel when it comes to development. This creates the opportunity
for building new protocols and applications on top of what’s already been built,
just like building blocks or Lego bricks. It’s important to remember that CPIs
are a two-way street and the same is true for any programs that you deploy! If
you build something cool and useful, developers have the ability to build on top
of what you’ve done or just plug your protocol into whatever it is that they are
building. Composability is a big part of what makes crypto so unique and CPIs
are what makes this possible on Orbition Native Chain.
Another important aspect of CPIs is that they allow programs to sign for their
PDAs. As you have probably noticed by now, PDAs are used very frequently in
Orbition Native Chain development because they allow programs to control specific addresses in
such a way that no external user can generate transactions with valid signatures
for those addresses. This can be *very* useful for many applications in Web3
(e.g. DeFi, NFTs, etc.) Without CPIs, PDAs would not be nearly as useful because
there would be no way for a program to sign transactions involving them -
essentially turning them black holes (once something is sent to a PDA, there
would be no way to get it back out w/o CPIs!)
## Lab
Now let's get some hands on experience with CPIs by making some additions to the
Movie Review program again. If you're dropping into this lesson without having
gone through prior lessons, the Movie Review program allows users to submit
movie reviews and have them stored in PDA accounts.
Last lesson, we added the ability to leave comments on other movie reviews using
PDAs. In this lesson, we’re going to work on having the program mint tokens to
the reviewer or commenter anytime a review or comment is submitted.
To implement this, we'll have to invoke the SPL Token Program's `MintTo`
instruction using a CPI. If you need a refresher on tokens, token mints, and
minting new tokens, have a look at the
[Token Program lesson](/content/courses/tokens/token-program) before moving
forward with this lab.
#### 1. Get starter code and add dependencies
To get started, we will be using the final state of the Movie Review program
from the previous PDA lesson. So, if you just completed that lesson then you’re
all set and ready to go. If you are just jumping in here, no worries, you can
[download the starter code here](https://github.com/Unboxed-Software/solana-movie-program/tree/solution-add-comments).
We'll be using the `solution-add-comments` branch as our starting point.
#### 2. Add dependencies to `Cargo.toml`
Before we get started we need to add two new dependencies to the `Cargo.toml`
file underneath `[dependencies]`. We'll be using the `spl-token` and
`spl-associated-token-account` crates in addition to the existing dependencies.
```text theme={null}
spl-token = { version="~3.2.0", features = [ "no-entrypoint" ] }
spl-associated-token-account = { version="=1.0.5", features = [ "no-entrypoint" ] }
```
After adding the above, run `cargo check` in your console to have cargo resolve
your dependencies and ensure that you are ready to continue. Depending on your
setup you may need to modify crate versions before moving on.
#### 3. Add necessary accounts to `add_movie_review`
Because we want users to be minted tokens upon creating a review, it makes sense
to add minting logic inside the `add_movie_review` function. Since we'll be
minting tokens, the `add_movie_review` instruction requires a few new accounts
to be passed in:
* `token_mint` - the mint address of the token
* `mint_auth` - address of the authority of the token mint
* `user_ata` - user’s associated token account for this mint (where the tokens
will be minted)
* `token_program` - address of the token program
We'll start by adding these new accounts to the area of the function that
iterates through the passed in accounts:
```rust theme={null}
// Inside add_movie_review
msg!("Adding movie review...");
msg!("Title: {}", title);
msg!("Rating: {}", rating);
msg!("Description: {}", description);
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
let pda_counter = next_account_info(account_info_iter)?;
let token_mint = next_account_info(account_info_iter)?;
let mint_auth = next_account_info(account_info_iter)?;
let user_ata = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
let token_program = next_account_info(account_info_iter)?;
```
There is no additional `instruction_data` required for the new functionality, so
no changes need to be made to how data is deserialized. The only additional
information that’s needed is the extra accounts.
#### 4. Mint tokens to the reviewer in `add_movie_review`
Before we dive into the minting logic, let's import the address of the Token
program and the constant `LAMPORTS_PER_SOL` at the top of the file.
```rust theme={null}
// Inside processor.rs
use solana_program::native_token::LAMPORTS_PER_SOL;
use spl_associated_token_account::get_associated_token_address;
use spl_token::{instruction::initialize_mint, ID as TOKEN_PROGRAM_ID};
```
Now we can move on to the logic that handles the actual minting of the tokens!
We’ll be adding this to the very end of the `add_movie_review` function right
before `Ok(())` is returned.
Minting tokens requires a signature by the mint authority. Since the program
needs to be able to mint tokens, the mint authority needs to be an account that
the program can sign for. In other words, it needs to be a PDA account owned by
the program.
We'll also be structuring our token mint such that the mint account is a PDA
account that we can derive deterministically. This way we can always verify that
the `token_mint` account passed into the program is the expected account.
Let's go ahead and derive the token mint and mint authority addresses using the
`find_program_address` function with the seeds “token\_mint” and "token\_auth,"
respectively.
```rust theme={null}
// Mint tokens here
msg!("deriving mint authority");
let (mint_pda, _mint_bump) = Pubkey::find_program_address(&[b"token_mint"], program_id);
let (mint_auth_pda, mint_auth_bump) =
Pubkey::find_program_address(&[b"token_auth"], program_id);
```
Next, we'll perform security checks against each of the new accounts passed into
the program. Always remember to verify accounts!
```rust theme={null}
if *token_mint.key != mint_pda {
msg!("Incorrect token mint");
return Err(ReviewError::IncorrectAccountError.into());
}
if *mint_auth.key != mint_auth_pda {
msg!("Mint passed in and mint derived do not match");
return Err(ReviewError::InvalidPDA.into());
}
if *user_ata.key != get_associated_token_address(initializer.key, token_mint.key) {
msg!("Incorrect token mint");
return Err(ReviewError::IncorrectAccountError.into());
}
if *token_program.key != TOKEN_PROGRAM_ID {
msg!("Incorrect token program");
return Err(ReviewError::IncorrectAccountError.into());
}
```
Finally, we can issue a CPI to the `mint_to` function of the token program with
the correct accounts using `invoke_signed`. The `spl_token` crate provides a
`mint_to` helper function for creating the minting instruction. This is great
because it means we don't have to manually build the entire instruction from
scratch. Rather, we can simply pass in the arguments required by the function.
Here's the function signature:
```rust theme={null}
// Inside the token program, returns an Instruction object
pub fn mint_to(
token_program_id: &Pubkey,
mint_pubkey: &Pubkey,
account_pubkey: &Pubkey,
owner_pubkey: &Pubkey,
signer_pubkeys: &[&Pubkey],
amount: u64,
) -> Result
```
Then we provide copies of the `token_mint`, `user_ata`, and `mint_auth`
accounts. And, most relevant to this lesson, we provide the seeds used to find
the `token_mint` address, including the bump seed.
```rust theme={null}
msg!("Minting 10 tokens to User associated token account");
invoke_signed(
// Instruction
&spl_token::instruction::mint_to(
token_program.key,
token_mint.key,
user_ata.key,
mint_auth.key,
&[],
10*LAMPORTS_PER_SOL,
)?,
// Account_infos
&[token_mint.clone(), user_ata.clone(), mint_auth.clone()],
// Seeds
&[&[b"token_auth", &[mint_auth_bump]]],
)?;
Ok(())
```
Note that we are using `invoke_signed` and not `invoke` here. The Token program
requires the `mint_auth` account to sign for this transaction. Since the
`mint_auth` account is a PDA, only the program it was derived from can sign on
its behalf. When `invoke_signed` is called, the Orbition Native Chain runtime calls
`create_program_address` with the seeds and bump provided and then compares the
derived address with all of the addresses of the provided `AccountInfo` objects.
If any of the addresses match the derived address, the runtime knows that the
matching account is a PDA of this program and that the program is signing this
transaction for this account.
At this point, the `add_movie_review` instruction should be fully functional and
will mint ten tokens to the reviewer when a review is created.
#### 5. Repeat for `add_comment`
Our updates to the `add_comment` function will be almost identical to what we
did for the `add_movie_review` function above. The only difference is that we’ll
change the amount of tokens minted for a comment from ten to five so that adding
reviews are weighted above commenting. First, update the accounts with the same
four additional accounts as in the `add_movie_review` function.
```rust theme={null}
// Inside add_comment
let account_info_iter = &mut accounts.iter();
let commenter = next_account_info(account_info_iter)?;
let pda_review = next_account_info(account_info_iter)?;
let pda_counter = next_account_info(account_info_iter)?;
let pda_comment = next_account_info(account_info_iter)?;
let token_mint = next_account_info(account_info_iter)?;
let mint_auth = next_account_info(account_info_iter)?;
let user_ata = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
let token_program = next_account_info(account_info_iter)?;
```
Next, move to the bottom of the `add_comment` function just before the `Ok(())`.
Then derive the token mint and mint authority accounts. Remember, both are PDAs
derived from seeds "token\_mint" and "token\_authority" respectively.
```rust theme={null}
// Mint tokens here
msg!("deriving mint authority");
let (mint_pda, _mint_bump) = Pubkey::find_program_address(&[b"token_mint"], program_id);
let (mint_auth_pda, mint_auth_bump) =
Pubkey::find_program_address(&[b"token_auth"], program_id);
```
Next, verify that each of the new accounts is the correct account.
```rust theme={null}
if *token_mint.key != mint_pda {
msg!("Incorrect token mint");
return Err(ReviewError::IncorrectAccountError.into());
}
if *mint_auth.key != mint_auth_pda {
msg!("Mint passed in and mint derived do not match");
return Err(ReviewError::InvalidPDA.into());
}
if *user_ata.key != get_associated_token_address(commenter.key, token_mint.key) {
msg!("Incorrect token mint");
return Err(ReviewError::IncorrectAccountError.into());
}
if *token_program.key != TOKEN_PROGRAM_ID {
msg!("Incorrect token program");
return Err(ReviewError::IncorrectAccountError.into());
}
```
Finally, use `invoke_signed` to send the `mint_to` instruction to the Token
program, sending five tokens to the commenter.
```rust theme={null}
msg!("Minting 5 tokens to User associated token account");
invoke_signed(
// Instruction
&spl_token::instruction::mint_to(
token_program.key,
token_mint.key,
user_ata.key,
mint_auth.key,
&[],
5 * LAMPORTS_PER_SOL,
)?,
// Account_infos
&[token_mint.clone(), user_ata.clone(), mint_auth.clone()],
// Seeds
&[&[b"token_auth", &[mint_auth_bump]]],
)?;
Ok(())
```
#### 6. Set up the token mint
We've written all the code needed to mint tokens to reviewers and commenters,
but all of it assumes that there is a token mint at the PDA derived with the
seed "token\_mint." For this to work, we're going to set up an additional
instruction for initializing the token mint. It will be written such that it can
only be called once and it doesn't particularly matter who calls it.
Given that throughout this lesson we've already hammered home all of the
concepts associated with PDAs and CPIs multiple times, we're going to walk
through this bit with less explanation than the prior steps. Start by adding a
fourth instruction variant to the `MovieInstruction` enum in `instruction.rs`.
```rust theme={null}
pub enum MovieInstruction {
AddMovieReview {
title: String,
rating: u8,
description: String,
},
UpdateMovieReview {
title: String,
rating: u8,
description: String,
},
AddComment {
comment: String,
},
InitializeMint,
}
```
Be sure to add it to the `match` statement in the `unpack` function in the same
file under the variant `3`.
```rust theme={null}
impl MovieInstruction {
pub fn unpack(input: &[u8]) -> Result {
let (&variant, rest) = input
.split_first()
.ok_or(ProgramError::InvalidInstructionData)?;
Ok(match variant {
0 => {
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
Self::AddMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description,
}
}
1 => {
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
Self::UpdateMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description,
}
}
2 => {
let payload = CommentPayload::try_from_slice(rest).unwrap();
Self::AddComment {
comment: payload.comment,
}
}
3 => Self::InitializeMint,
_ => return Err(ProgramError::InvalidInstructionData),
})
}
}
```
In the `process_instruction` function in the `processor.rs` file, add the new
instruction to the `match` statement and call a function
`initialize_token_mint`.
```rust theme={null}
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> ProgramResult {
let instruction = MovieInstruction::unpack(instruction_data)?;
match instruction {
MovieInstruction::AddMovieReview {
title,
rating,
description,
} => add_movie_review(program_id, accounts, title, rating, description),
MovieInstruction::UpdateMovieReview {
title,
rating,
description,
} => update_movie_review(program_id, accounts, title, rating, description),
MovieInstruction::AddComment { comment } => add_comment(program_id, accounts, comment),
MovieInstruction::InitializeMint => initialize_token_mint(program_id, accounts),
}
}
```
Lastly, declare and implement the `initialize_token_mint` function. This
function will derive the token mint and mint authority PDAs, create the token
mint account, and then initialize the token mint. We won't explain all of this
in detail, but it's worth reading through the code, especially given that the
creation and initialization of the token mint both involve CPIs. Again, if you
need a refresher on tokens and mints, have a look at the
[Token Program lesson](/content/courses/tokens/token-program).
```rust theme={null}
pub fn initialize_token_mint(program_id: &Pubkey, accounts: &[AccountInfo]) -> ProgramResult {
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let token_mint = next_account_info(account_info_iter)?;
let mint_auth = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
let token_program = next_account_info(account_info_iter)?;
let sysvar_rent = next_account_info(account_info_iter)?;
let (mint_pda, mint_bump) = Pubkey::find_program_address(&[b"token_mint"], program_id);
let (mint_auth_pda, _mint_auth_bump) =
Pubkey::find_program_address(&[b"token_auth"], program_id);
msg!("Token mint: {:?}", mint_pda);
msg!("Mint authority: {:?}", mint_auth_pda);
if mint_pda != *token_mint.key {
msg!("Incorrect token mint account");
return Err(ReviewError::IncorrectAccountError.into());
}
if *token_program.key != TOKEN_PROGRAM_ID {
msg!("Incorrect token program");
return Err(ReviewError::IncorrectAccountError.into());
}
if *mint_auth.key != mint_auth_pda {
msg!("Incorrect mint auth account");
return Err(ReviewError::IncorrectAccountError.into());
}
let rent = Rent::get()?;
let rent_lamports = rent.minimum_balance(82);
invoke_signed(
&system_instruction::create_account(
initializer.key,
token_mint.key,
rent_lamports,
82,
token_program.key,
),
&[
initializer.clone(),
token_mint.clone(),
system_program.clone(),
],
&[&[b"token_mint", &[mint_bump]]],
)?;
msg!("Created token mint account");
invoke_signed(
&initialize_mint(
token_program.key,
token_mint.key,
mint_auth.key,
Option::None,
9,
)?,
&[token_mint.clone(), sysvar_rent.clone(), mint_auth.clone()],
&[&[b"token_mint", &[mint_bump]]],
)?;
msg!("Initialized token mint");
Ok(())
}
```
#### 7. Build and deploy
Now we’re ready to build and deploy our program! You can build the program by
running `cargo build-bpf` and then running the command that is returned, it
should look something like `solana program deploy `.
Before you can start testing whether or not adding a review or comment sends you
tokens, you need to initialize the program's token mint. You can use
[this script](https://github.com/Unboxed-Software/solana-movie-token-client) to
do that. Once you'd cloned that repository, replace the `PROGRAM_ID` in
`index.ts` with your program's ID. Then run `npm install` and then `npm start`.
The script assumes you're deploying to Testnet. If you're deploying locally, then
make sure to tailor the script accordingly.
Once you've initialized your token mint, you can use the
[Movie Review frontend](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-add-tokens)
to test adding reviews and comments. Again, the code assumes you're on Testnet so
please act accordingly.
After submitting a review, you should see 10 new tokens in your wallet! When you
add a comment, you should receive 5 tokens. They won't have a fancy name or
image since we didn't add any metadata to the token, but you get the idea.
If you need more time with the concepts from this lesson or got stuck along the
way, feel free to
[take a look at the solution code](https://github.com/Unboxed-Software/solana-movie-program/tree/solution-add-tokens).
Note that the solution to this lab is on the `solution-add-tokens` branch.
## Challenge
To apply what you've learned about CPIs in this lesson, think about how you
could incorporate them into the Student Intro program. You could do something
similar to what we did in the lab here and add some functionality to mint tokens
to users when they introduce themselves. Or if you're feeling really ambitious,
think about how you could take all that you have learned so far in the course
and create something completely new from scratch.
A great example would be to build a decentralized Stack Overflow. The program
could use tokens to determine a user's overall rating, mint tokens when
questions are answered correctly, allow users to upvote answers, etc. All of
that is possible and you now have the skills and knowledge to go and build
something like it on your own!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=ade5d386-809f-42c2-80eb-a6c04c471f53)!
# Deserialize Program Data
Source: https://docs.orbition.network/courses/native-onchain-development/deserialize-custom-data-frontend
Deserialize instructions in JS/TS clients to send to your native program.
## Summary
* Programs store data in PDAs, which stands for **Program Derived Address**.
* PDAs do not have a corresponding secret key.
* To store and locate data, derive a PDA using the
`findProgramAddress(seeds, programid)` method.
* You can get the accounts belonging to a program using
`getProgramAccounts(programId)`.
* Account data needs to be deserialized using the same layout used to store it
in the first place. You can use `@coral-xyz/borsh` to create a schema.
## Lesson
In the last lesson, we serialized program data that was subsequently stored
onchain by a Orbition Native Chain program. In this lesson, we’ll cover in greater detail how
programs store data on the chain, how to retrieve data, and how to deserialize
the data they store.
### Programs
As the saying goes, everything in Orbition Native Chain is an account. Even programs. Programs
are accounts that store code and are marked as executable. This code can be
executed by the Orbition Native Chain runtime when instructed to do so. A program address is a
public keys on the Ed25519 Elliptic Curve. Like all public keys, they have
corresponding secret keys.
Programs store data separately from their code. Programs store data in PDAs,
which stands for **Program Derived Address**. PDAs are a unique concept to
Orbition Native Chain, but the pattern is familiar:
* You can think of PDAs as a key value store, where the address is the key, and
the data inside the account is the value.
* You can also consider PDAs as records in a database, with the address being
the primary key used to look up the values inside.
PDAs combine a program addresss and some developer-chosen seeds to create
addresses that store individual pieces of data. Since PDAs are addresses that
lie *off* the Ed25519 Elliptic curve, PDAs don't have secret keys. Instead, PDAs
can be signed for by the program address used to create them.
PDAs and the data inside them can be consistently found based on the program
address, bump, and seeds. To find a PDA, the program ID and seeds of the
developer’s choice (like a string of text) are passed through the
[`findProgramAddress()`](https://orbition-network.github.io/solana-web3.js/classes/PublicKey.html#findProgramAddress)
function.
Let's have a look at some examples...
##### Example: program with global state
A simple program that has global state - like our ping counter - might wish to
only use a single PDA, based on a simple seed phrase like `"GLOBAL_STATE"`. If
the client wanted to read data from this PDA, it could derive the address using
the program ID and this same seed.
```typescript theme={null}
const [pda, bump] = await findProgramAddress(
Buffer.from("GLOBAL_STATE"),
programId,
);
```
##### Example: program with user-specific data
In programs that store user-specific data, it’s common to use a user’s public
key as the seed. This separates each user’s data into its own PDA. The
separation makes it possible for the client to locate each user’s data by
finding the address using the program ID and the user’s public key.
```typescript theme={null}
const [pda, bump] = await web3.PublicKey.findProgramAddress(
[publicKey.toBuffer()],
programId,
);
```
#### Example: program with multiple data items per user
When there are multiple data items per user, a program may use more seeds to
create and identify accounts. For example, in a note-taking app there may be one
account per note where each PDA is derived with the user’s public key and the
note’s title.
```typescript theme={null}
const [pda, bump] = await web3.PublicKey.findProgramAddress(
[publicKey.toBuffer(), Buffer.from("Shopping list")],
programId,
);
```
In this example we can see both Alice and Bob have a note called 'Shopping List'
but since we use their wallet address as one of the seeds, both these notes can
exist at the same time.
#### Getting Multiple Program Accounts
In addition to deriving addresses, you can fetch all accounts created by a
program using `connection.getProgramAccounts(programId)`. This returns an array
of objects where each object has `pubkey` property representing the public key
of the account and an `account` property of type `AccountInfo`. You can use the
`account` property to get the account data.
```typescript theme={null}
const accounts = connection.getProgramAccounts(programId).then(accounts => {
accounts.map(({ pubkey, account }) => {
console.log("Account:", pubkey);
console.log("Data buffer:", account.data);
});
});
```
### Deserializing program data
The `data` property on an `AccountInfo` object is a buffer. To use it
efficiently, you’ll need to write code that deserializes it into something more
usable. This is similar to the serialization process we covered last lesson.
Just as before, we’ll use [Borsh](https://borsh.io/) and `@coral-xyz/borsh`. If
you need a refresher on either of these, have a look at the previous lesson.
Deserializing requires knowledge of the account layout ahead of time. When
creating your own programs, you will define how this is done as part of that
process. Many programs also have documentation on how to deserialize the account
data. Otherwise, if the program code is available you can look at the source and
determine the structure that way.
To properly deserialize data from an onchain program, you will have to create a
client-side schema mirroring how the data is stored in the account. For example,
the following might be the schema for an account storing metadata about a player
in an onchain game.
```typescript theme={null}
import * as borsh from "@coral-xyz/borsh";
borshAccountSchema = borsh.struct([
borsh.bool("initialized"),
borsh.u16("playerId"),
borsh.str("name"),
]);
```
Once you have your layout defined, simply call `.decode(buffer)` on the schema.
```typescript theme={null}
import * as borsh from "@coral-xyz/borsh";
borshAccountSchema = borsh.struct([
borsh.bool("initialized"),
borsh.u16("playerId"),
borsh.str("name"),
]);
const { playerId, name } = borshAccountSchema.decode(buffer);
```
## Lab
Let’s practice this together by continuing to work on the Movie Review app from
the last lesson. No worries if you’re just jumping into this lesson - it should
be possible to follow either way.
As a refresher, this project uses a Orbition Native Chain program deployed on Testnet which lets
users review movies. Last lesson, we added functionality to the frontend
skeleton letting users submit movie reviews but the list of reviews is still
showing mock data. Let’s fix that by fetching the program’s storage accounts and
deserializing the data stored there.
#### 1. Download the starter code
If you didn’t complete the lab from the last lesson or just want to make sure
that you didn’t miss anything, you can download the
[starter code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-serialize-instruction-data).
The project is a fairly simple Next.js application. It includes the
`WalletContextProvider` we created in the Wallets lesson, a `Card` component for
displaying a movie review, a `MovieList` component that displays reviews in a
list, a `Form` component for submitting a new review, and a `Movie.ts` file that
contains a class definition for a `Movie` object.
Note that when you run `npm run dev`, the reviews displayed on the page are
mocks. We’ll be swapping those out for the real deal.
#### 2. Create the buffer layout
Remember that to properly interact with a Orbition Native Chain program, you need to know how
its data is structured. A reminder:
The program's executable data is in a program account, but individual reviews
are kept in PDAs. We use `findProgramAddress()` to create a PDA that's unique
for every wallet, for every film title. We'll store the following data in the
PDA's `data`:
1. `initialized` as a boolean representing whether or not the account has been
initialized.
2. `rating` as an unsigned, 8-bit integer representing the rating out of 5 that
the reviewer gave the movie.
3. `title` as a string representing the title of the reviewed movie.
4. `description` as a string representing the written portion of the review.
Let’s configure a `borsh` layout in the `Movie` class to represent the movie
account data layout. Start by importing `@coral-xyz/borsh`. Next, create a
`borshAccountSchema` static property and set it to the appropriate `borsh`
struct containing the properties listed above.
```typescript theme={null}
import * as borsh from '@coral-xyz/borsh'
export class Movie {
title: string;
rating: number;
description: string;
...
static borshAccountSchema = borsh.struct([
borsh.bool('initialized'),
borsh.u8('rating'),
borsh.str('title'),
borsh.str('description'),
])
}
```
Remember, the order here *matters*. It needs to match how the account data is
structured.
#### 3. Create a method to deserialize data
Now that we have the buffer layout set up, let’s create a static method in
`Movie` called `deserialize` that will take an optional `Buffer` and return a
`Movie` object or `null`.
```typescript theme={null}
import * as borsh from '@coral-xyz/borsh'
export class Movie {
title: string;
rating: number;
description: string;
...
static borshAccountSchema = borsh.struct([
borsh.bool('initialized'),
borsh.u8('rating'),
borsh.str('title'),
borsh.str('description'),
])
static deserialize(buffer?: Buffer): Movie|null {
if (!buffer) {
return null
}
try {
const { title, rating, description } = this.borshAccountSchema.decode(buffer)
return new Movie(title, rating, description)
} catch(error) {
console.log('Deserialization error:', error)
return null
}
}
}
```
The method first checks whether or not the buffer exists and returns `null` if
it doesn’t. Next, it uses the layout we created to decode the buffer, then uses
the data to construct and return an instance of `Movie`. If the decoding fails,
the method logs the error and returns `null`.
#### 4. Fetch movie review accounts
Now that we have a way to deserialize account data, we need to actually fetch
the accounts. Open `MovieList.tsx` and import `@orbition-network/web3.js`. Then, create a
new `Connection` inside the `MovieList` component. Finally, replace the line
`setMovies(Movie.mocks)` inside `useEffect` with a call to
`connection.getProgramAccounts`. Take the resulting array and convert it into an
array of movies and call `setMovies`.
```typescript theme={null}
import { Card } from "./Card";
import { FC, useEffect, useState } from "react";
import { Movie } from "../models/Movie";
import * as web3 from "@orbition-network/web3.js";
import { useConnection } from "@orbition-network/wallet-adapter-react";
const MOVIE_REVIEW_PROGRAM_ID = "CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN";
export const MovieList: FC = () => {
const { connection } = useConnection();
const [movies, setMovies] = useState([]);
useEffect(() => {
connection
.getProgramAccounts(new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID))
.then(async accounts => {
const movies: Movie[] = accounts.map(({ account }) => {
return Movie.deserialize(account.data);
});
setMovies(movies);
});
}, []);
return (
{movies.map((movie, i) => (
))}
);
};
```
At this point, you should be able to run the app and see the list of movie
reviews retrieved from the program!
Depending on how many reviews have been submitted, this may take a long time to
load or may lock up your browser entirely. But don’t worry — next lesson we’ll
learn how to page and filter accounts so you can be more surgical with what you
load.
If you need more time with this project to feel comfortable with these concepts,
have a look at the
[solution code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-deserialize-account-data)
before continuing.
## Challenge
Now it’s your turn to build something independently. Last lesson, you worked on
the Student Intros app to serialize instruction data and send a new intro to the
network. Now, it's time to fetch and deserialize the program's account data.
Remember, the Orbition Native Chain program that supports this is at
`HdE95RSVsdb315jfJtaykXhXY478h53X6okDupVfY9yf`.
1. You can build this from scratch or you can
[download the starter code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-serialize-instruction-data).
2. Create the account buffer layout in `StudentIntro.ts`. The account data
contains:
1. `initialized` as an unsigned, 8-bit integer representing the instruction
to run (should be 1).
2. `name` as a string representing the student's name.
3. `message` as a string representing the message the student shared about
their Orbition Native Chain journey.
3. Create a static method in `StudentIntro.ts` that will use the buffer layout
to deserialize an account data buffer into a `StudentIntro` object.
4. In the `StudentIntroList` component's `useEffect`, get the program's accounts
and deserialize their data into a list of `StudentIntro` objects.
5. Instead of mock data, you should now be seeing student introductions from the
network!
If you get really stumped, feel free to
[check out the solution code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-deserialize-account-data).
As always, get creative with these challenges and take them beyond the
instructions if you want!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=9cb89e09-2c97-4185-93b0-c89f7aca7677)!
# Create a Basic Program, Part 1 - Handle Instruction Data
Source: https://docs.orbition.network/courses/native-onchain-development/deserialize-instruction-data
Learn how native programs distinguish instructions for different functions.
## Summary
* Most programs support **multiple discrete instruction handlers** - you decide
when writing your program what these instruction handlers are and what data
must accompany them when writing your program what these instruction handlers
are and what data
* Rust **enums** are often used to represent discrete program instruction
* You can use the `borsh` crate and the `derive` attribute to provide Borsh
deserialization and serialization functionality to Rust structs
* Rust `match` expressions help create conditional code paths based on the
provided instruction
## Lesson
One of the most basic elements of a Orbition Native Chain program is the logic for handling
instruction data. Most programs support multiple related functions and use
differences in instruction data to determine which code path to execute. For
example, two different data formats in the instruction data passed to the
program may represent instructions for creating a new piece of data vs deleting
the same piece of data.
Since instruction data is provided to your program's entry point as a byte
array, it's common to create a Rust data type to represent instructions in a way
that's more usable throughout your code. This lesson will walk through how to
set up such a type, how to deserialize the instruction data into this format,
and how to execute the proper code path based on the instruction passed into the
program's entry point.
### Rust basics
Before we dive into the specifics of a basic Orbition Native Chain program, let's talk about
the Rust basics we'll be using throughout this lesson.
#### Variables
Variable assignment in Rust happens with the `let` keyword.
```rust theme={null}
let age = 33;
```
Variables in Rust by default are immutable, meaning a variable's value cannot be
changed once it has been set. To create a variable that we'd like to change at
some point in the future, we use the `mut` keyword. Defining a variable with
this keyword means that the value stored in it can change.
```rust theme={null}
// compiler will throw error
let age = 33;
age = 34;
// this is allowed
let mut mutable_age = 33;
mutable_age = 34;
```
The Rust compiler guarantees that immutable variables truly cannot change so
that you don’t have to keep track of it yourself. This makes your code easier to
reason through and simplifies debugging.
#### Structs
A struct, or structure, is a custom data type that lets you package together and
name multiple related values that make up a meaningful group. Each piece of data
in a struct can be of different types and each has a name associated with it.
These pieces of data are called **fields**. They behave similarly to properties
in other languages.
```rust theme={null}
struct User {
active: bool,
email: String,
age: u64
}
```
To use a struct after we’ve defined it, we create an instance of that struct by
specifying concrete values for each of the fields.
```rust theme={null}
let mut user1 = User {
active: true,
email: String::from("test@test.com"),
age: 36
};
```
To get or set a specific value from a struct, we use dot notation.
```rust theme={null}
user1.age = 37;
```
#### Enumerations
Enumerations (or Enums) are a data struct that allow you to define a type by
enumerating its possible variants. An example of an enum may look like:
```rust theme={null}
enum LightStatus {
On,
Off
}
```
The `LightStatus` enum has two possible variants in this situation: it's either
`On` or `Off`.
You can also embed values into enum variants, similar to adding fields to a
struct.
```rust theme={null}
enum LightStatus {
On {
color: String
},
Off
}
let light_status = LightStatus::On { color: String::from("red") };
```
In this example, setting a variable to the `On` variant of `LightStatus`
requires also setting the value of `color`.
#### Match statements
Match statements are very similar to `switch` statements in C/C++. The `match`
statement allows you to compare a value against a series of patterns and then
execute code based on which pattern matches the value. Patterns can be made of
literal values, variable names, wildcards, and more. The match statement must
include all possible scenarios, otherwise the code will not compile.
```rust theme={null}
enum Coin {
Penny,
Nickel,
Dime,
Quarter
}
fn value_in_cents(coin: Coin) -> u8 {
match coin {
Coin::Penny => 1,
Coin::Nickel => 5,
Coin::Dime => 10,
Coin::Quarter => 25
}
}
```
#### Implementations
The `impl` keyword is used in Rust to define a type's implementations. Functions
and constants can both be defined in an implementation.
```rust theme={null}
struct Example {
number: i32
}
impl Example {
fn boo() {
println!("boo! Example::boo() was called!");
}
fn answer(&mut self) {
self.number += 42;
}
fn get_number(&self) -> i32 {
self.number
}
}
```
The function `boo` here can only be called on the type itself rather than an
instance of the type, like so:
```rust theme={null}
Example::boo();
```
Meanwhile, `answer` requires a mutable instance of `Example` and can be called
with dot syntax:
```rust theme={null}
let mut example = Example { number: 3 };
example.answer();
```
#### Traits and attributes
You won't be creating your own traits or attributes at this stage, so we won't
provide an in depth explanation of either. However, you will be using the
`derive` attribute macro and some traits provided by the `borsh` crate, so it's
important you have a high level understanding of each.
Traits describe an abstract interface that types can implement. If a trait
defines a function `bark()` and a type then adopts that trait, the type must
then implement the `bark()` function.
[Attributes](https://doc.rust-lang.org/rust-by-example/attribute.html) add
metadata to a type and can be used for many different purposes.
When you add the
[`derive` attribute](https://doc.rust-lang.org/rust-by-example/trait/derive.html)
to a type and provide one or more supported traits, code is generated under the
hood to automatically implement the traits for that type. We'll provide a
concrete example of this shortly.
### Representing instructions as a Rust data type
Now that we've covered the Rust basics, let's apply them to Orbition Native Chain programs.
More often than not, programs will have more than one function. For example, you
may have a program that acts as the backend for a note-taking app. Assume this
program accepts instructions for creating a new note, updating an existing note,
and deleting an existing note.
Since instructions have discrete types, they're usually a great fit for an enum
data type.
```rust theme={null}
enum NoteInstruction {
CreateNote {
title: String,
body: String,
id: u64
},
UpdateNote {
title: String,
body: String,
id: u64
},
DeleteNote {
id: u64
}
}
```
Notice that each variant of the `NoteInstruction` enum comes with embedded data
that will be used by the program to accomplish the tasks of creating, updating,
and deleting a note, respectively.
### Deserialize instruction data
Instruction data is passed to the program as a byte array, so you need a way to
deterministically convert that array into an instance of the instruction enum
type.
In previous units, we used Borsh for client-side serialization and
deserialization. To use Borsh program-side, we use the `borsh` crate. This crate
provides traits for `BorshDeserialize` and `BorshSerialize` that you can apply
to your types using the `derive` attribute.
To make deserializing instruction data simple, you can create a struct
representing the data and use the `derive` attribute to apply the
`BorshDeserialize` trait to the struct. This implements the methods defined in
`BorshDeserialize`, including the `try_from_slice` method that we'll be using to
deserialize the instruction data.
Remember, the struct itself needs to match the structure of the data in the byte
array.
```rust theme={null}
#[derive(BorshDeserialize)]
struct NoteInstructionPayload {
id: u64,
title: String,
body: String
}
```
Once this struct has been created, you can create an implementation for your
instruction enum to handle the logic associated with deserializing instruction
data. It's common to see this done inside a function called `unpack` that
accepts the instruction data as an argument and returns the appropriate instance
of the enum with the deserialized data.
It's standard practice to structure your program to expect the first byte (or
other fixed number of bytes) to be an identifier for which instruction the
program should run. This could be an integer or a string identifier. For this
example, we'll use the first byte and map integers 0, 1, and 2 to instructions
create, update, and delete, respectively.
```rust theme={null}
impl NoteInstruction {
// Unpack inbound buffer to associated Instruction
// The expected format for input is a Borsh serialized vector
pub fn unpack(input: &[u8]) -> Result {
// Take the first byte as the variant to
// determine which instruction to execute
let (&variant, rest) = input.split_first().ok_or(ProgramError::InvalidInstructionData)?;
// Use the temporary payload struct to deserialize
let payload = NoteInstructionPayload::try_from_slice(rest).unwrap();
// Match the variant to determine which data struct is expected by
// the function and return the TestStruct or an error
Ok(match variant {
0 => Self::CreateNote {
title: payload.title,
body: payload.body,
id: payload.id
},
1 => Self::UpdateNote {
title: payload.title,
body: payload.body,
id: payload.id
},
2 => Self::DeleteNote {
id: payload.id
},
_ => return Err(ProgramError::InvalidInstructionData)
})
}
}
```
There's a lot in this example so let's take it one step at a time:
1. This function starts by using the `split_first` function on the `input`
parameter to return a tuple. The first element, `variant`, is the first byte
from the byte array and the second element, `rest`, is the rest of the byte
array.
2. The function then uses the `try_from_slice` method on
`NoteInstructionPayload` to deserialize the rest of the byte array into an
instance of `NoteInstructionPayload` called `payload`
3. Finally, the function uses a `match` statement on `variant` to create and
return the appropriate enum instance using information from `payload`
Note that there is Rust syntax in this function that we haven't explained yet.
The `ok_or` and `unwrap` functions are used for error handling and will be
discussed in detail in another lesson.
### Program logic
With a way to deserialize instruction data into a custom Rust type, you can then
use appropriate control flow to execute different code paths in your program
based on which instruction is passed into your program's entry point.
```rust theme={null}
entrypoint!(process_instruction);
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult {
// Call unpack to deserialize instruction_data
let instruction = NoteInstruction::unpack(instruction_data)?;
// Match the returned data struct to what you expect
match instruction {
NoteInstruction::CreateNote { title, body, id } => {
// Execute program code to create a note
},
NoteInstruction::UpdateNote { title, body, id } => {
// Execute program code to update a note
},
NoteInstruction::DeleteNote { id } => {
// Execute program code to delete a note
}
}
}
```
For simple programs where there are only one or two instructions to execute, it
may be fine to write the logic inside the match statement. For programs with
many different possible instructions to match against, your code will be much
more readable if the logic for each instruction is written in a separate
function and simply called from inside the `match` statement.
### Program file structure
The [Hello World lesson’s](hello-world-program) program was simple enough that
it could be confined to one file. But as the complexity of a program grows, it's
important to maintain a project structure that remains readable and extensible.
This involves encapsulating code into functions and data structures as we've
done so far. But it also involves grouping related code into separate files.
For example, a good portion of the code we've worked through so far has to do
with defining and deserializing instructions. That code should live in its own
file rather than be written in the same file as the entry point. By doing so, we
would then have 2 files, one with the program entry point and the other with the
instruction code:
* **lib.rs**
* **instruction.rs**
Once you start splitting your program up like this you will need to make sure
you register all of the files in one central location. We’ll be doing this in
`lib.rs`. **You must register every file in your program like this.**
```rust theme={null}
// This would be inside lib.rs
pub mod instruction;
```
Additionally, any declarations that you would like to be available through `use`
statements in other files will need to be prefaced with the `pub` keyword:
```rust theme={null}
pub enum NoteInstruction { ... }
```
## Lab
For this lesson’s lab, we’ll be building out the first half of the Movie Review
program that we worked with in Module 1. This program stores movie reviews
submitted by users.
For now, we'll focus on deserializing the instruction data. The following lesson
will focus on the second half of this program.
#### 1. Entry point
We’ll be using [Orbition Native Chain Playground](https://beta.solpg.io/) again to build out
this program. Orbition Native Chain Playground saves state in your browser, so everything you
did in the previous lesson may still be there. If it is, let's clear everything
out from the current `lib.rs` file.
Inside lib.rs, we’re going to bring in the following crates and define where
we’d like our entry point to the program to be with the `entrypoint` macro.
```rust theme={null}
use solana_program::{
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
msg,
account_info::AccountInfo,
};
// Entry point is a function call process_instruction
entrypoint!(process_instruction);
// Inside lib.rs
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult {
Ok(())
}
```
#### 2. Deserialize instruction data
Before we continue with the processor logic, we should define our supported
instructions and implement our deserialization function.
For readability, let's create a new file called `instruction.rs`. Inside this
new file, add `use` statements for `BorshDeserialize` and `ProgramError`, then
create a `MovieInstruction` enum with an `AddMovieReview` variant. This variant
should have embedded values for `title,` `rating`, and `description`.
```rust theme={null}
use borsh::{BorshDeserialize};
use solana_program::{program_error::ProgramError};
pub enum MovieInstruction {
AddMovieReview {
title: String,
rating: u8,
description: String
}
}
```
Next, define a `MovieReviewPayload` struct. This will act as an intermediary
type for deserializtion so it should use the `derive` attribute macro to provide
a default implementation for the `BorshDeserialize` trait.
```rust theme={null}
#[derive(BorshDeserialize)]
struct MovieReviewPayload {
title: String,
rating: u8,
description: String
}
```
Finally, create an implementation for the `MovieInstruction` enum that defines
and implements a function called `unpack` that takes a byte array as an argument
and returns a `Result` type. This function should:
1. Use the `split_first` function to split the first byte of the array from the
rest of the array
2. Deserialize the rest of the array into an instance of `MovieReviewPayload`
3. Use a `match` statement to return the `AddMovieReview` variant of
`MovieInstruction` if the first byte of the array was a 0 or return a program
error otherwise
```rust theme={null}
impl MovieInstruction {
// Unpack inbound buffer to associated Instruction
// The expected format for input is a Borsh serialized vector
pub fn unpack(input: &[u8]) -> Result {
// Split the first byte of data
let (&variant, rest) = input.split_first().ok_or(ProgramError::InvalidInstructionData)?;
// `try_from_slice` is one of the implementations from the BorshDeserialization trait
// Deserializes instruction byte data into the payload struct
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
// Match the first byte and return the AddMovieReview struct
Ok(match variant {
0 => Self::AddMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description },
_ => return Err(ProgramError::InvalidInstructionData)
})
}
}
```
#### 3. Program logic
With the instruction deserialization handled, we can return to the `lib.rs` file
to handle some of our program logic.
Remember, since we added code to a different file, we need to register it in the
`lib.rs` file using `pub mod instruction;`. Then we can add a `use` statement to
bring the `MovieInstruction` type into scope.
```rust theme={null}
pub mod instruction;
use instruction::{MovieInstruction};
```
Next, let's define a new function `add_movie_review` that takes as arguments
`program_id`, `accounts`, `title`, `rating`, and `description`. It should also
return an instance of `ProgramResult` Inside this function, let's simply log our
values for now and we'll revisit the rest of the implementation of the function
in the next lesson.
```rust theme={null}
pub fn add_movie_review(
program_id: &Pubkey,
accounts: &[AccountInfo],
title: String,
rating: u8,
description: String
) -> ProgramResult {
// Logging instruction data that was passed in
msg!("Adding movie review...");
msg!("Title: {}", title);
msg!("Rating: {}", rating);
msg!("Description: {}", description);
Ok(())
}
```
With that done, we can call `add_movie_review` from `process_instruction` (the
function we set as our entry point). To pass all the required arguments to the
function, we'll first need to call the `unpack` we created on
`MovieInstruction`, then use a `match` statement to ensure that the instruction
we've received is the `AddMovieReview` variant.
```rust theme={null}
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult {
// Unpack called
let instruction = MovieInstruction::unpack(instruction_data)?;
// Match against the data struct returned into `instruction` variable
match instruction {
MovieInstruction::AddMovieReview { title, rating, description } => {
// Make a call to `add_move_review` function
add_movie_review(program_id, accounts, title, rating, description)
}
}
}
```
And just like that, your program should be functional enough to log the
instruction data passed in when a transaction is submitted!
Build and deploy your program from Orbition Native Chain Program just like in the last lesson.
If you haven't changed the program ID since going through the last lesson, it
will automatically deploy to the same ID. If you'd like it to have a separate
address you can generate a new program ID from the playground before deploying.
You can test your program by submitting a transaction with the right instruction
data. For that, feel free to use
[this script](https://github.com/Unboxed-Software/solana-movie-client) or
[the frontend](https://github.com/Unboxed-Software/solana-movie-frontend) we
built in the
[Serialize Custom Instruction Data lesson](serialize-instruction-data). In both
cases, make sure you copy and paste the program ID for your program into the
appropriate area of the source code to make sure you're testing the right
program.
If you need to spend some more time with this lab before moving on, please do!
You can also have a look at the program
[solution code](https://beta.solpg.io/62aa9ba3b5e36a8f6716d45b) if you get
stuck.
## Challenge
For this lesson's challenge, try replicating the Student Intro program from
Module 1. Recall that we created a frontend application that lets students
introduce themselves! The program takes a user's name and a short message as the
`instruction_data` and creates an account to store the data onchain.
Using what you've learned in this lesson, build the Student Intro program to the
point where you can print the `name` and `message` provided by the user to the
program logs when the program is invoked.
You can test your program by building the
[frontend](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-serialize-instruction-data)
we created in the
[Serialize Custom Instruction Data lesson](serialize-instruction-data) and then
checking the program logs on Orbition Native Chain Explorer. Remember to replace the program ID
in the frontend code with the one you've deployed.
Try to do this independently if you can! But if you get stuck, feel free to
reference the [solution code](https://beta.solpg.io/62b0ce53f6273245aca4f5b0).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=74a157dc-01a7-4b08-9a5f-27aa51a4346c)!
# Hello World
Source: https://docs.orbition.network/courses/native-onchain-development/hello-world-program
Create an onchain program for Orbition Native Chain using native Rust, without Anchor.
## Summary
* Native Orbition Native Chain programs have a single **entry point** to process instructions
* A program processes an instruction using the **program\_id**, list of
**accounts**, and **instruction\_data** included with the instruction
## Lesson
The following guide assumes you are familiar with Orbition Native Chain program basics - if not, check out [introduction to onchain programming](/developers/courses/onchain-development/intro-to-onchain).
This lesson will give you a basic introduction to writing and deploying a Orbition Native Chain
program using the Rust programming language, without any framework. This gives
you greater control, but also requires you to perform much of the basic work of
creating an onchain program yourself.
To avoid the distraction of setting up a local development environment, we'll be
using a browser-based IDE called Orbition Native Chain Playground.
### Rust Basics
Before we dive into the building our "Hello, world!" program, let’s first go
over some Rust basics. If you want to dig deeper into Rust, have a look at
the [Rust language book](https://doc.rust-lang.org/book/ch00-00-introduction.html).
#### Module System
Rust organizes code using what is collectively referred to as the “module
system”.
This includes:
* **Modules** - A module separates code into logical units to provide isolated
namespaces for organization, scope, and privacy of paths
* **Crates** - A crate is either a library or an executable program. The source
code for a crate is usually subdivided into multiple modules.
* **Packages** - A package contains a collection of crates as well as a manifest
file for specifying metadata and dependencies between packages
Throughout this lesson, we’ll focus on using crates and modules.
#### Paths and scope
Crates in Rust contain modules that define functionality which can be shared
with multiple projects. If we want to access an item within a module, then we
need to know its "path" (like when we're navigating a filesystem).
Think of the crate structure as a tree where the crate is the base and modules
are branches, each of which can have submodules or items that are additional
branches.
The path to a particular module or item is the name of each step from the crate
to that module where each is separated by `::`. As an example, let's look at the
following structure:
1. The base crate is `solana_program`
2. `solana_program` contains a module named `account_info`
3. `account_info` contains a struct named `AccountInfo`
The path to `AccountInfo` would be `solana_program::account_info::AccountInfo`.
Absent of any other keywords, we would need to reference this entire path to use
`AccountInfo` in our code.
However, with the
[`use`](https://doc.rust-lang.org/stable/book/ch07-04-bringing-paths-into-scope-with-the-use-keyword.html)
keyword we can bring an item into scope so that it can be reused throughout a
file without specifying the full path each time. It's common to see a series of
`use` commands at the top of a Rust file.
```rust theme={null}
use solana_program::account_info::AccountInfo
```
#### Declaring Functions in Rust
We define a function in Rust by using the `fn` keyword followed by a function
name and a set of parentheses.
```rust theme={null}
fn process_instruction()
```
We can then add arguments to our function by including variable names and
specifying its corresponding data type within the parentheses.
Rust is known as a ”statically typed” language and every value in Rust is of a
certain ”data type”. This meaning that Rust must know the types of all variables
at compile time. In cases when multiple types are possible, we must add a type
annotation to our variables.
In the example below, we create a function named `process_instruction` that
requires the following arguments:
* `program_id` - required to be type `&Pubkey`
* `accounts` - required to be type `&[AccountInfo]`
* `instruction_data` - required to be type `&[u8]`
Note the `&` in front of the type for each argument listed in the
`process_instruction` function. In Rust, `&` represents a ”reference” to another
variable. This allows you to refer to some value without taking ownership of it.
The “reference” is guaranteed to point to a valid value of a particular type.
The action of creating a reference in Rust is called “borrowing”.
In this example, when the `process_instruction` function is called, a user must
pass in values for the required arguments. The `process_instruction` function
then references the values passed in by the user, and guarantees that each value
is the correct data type specified in the `process_instruction` function.
Additionally, note the brackets `[]` around `&[AccountInfo]` and `&[u8]`. This
means that the `accounts` and `instruction_data` arguments expect “slices” of
types `AccountInfo` and `u8`, respectively. A “slice” is similar to an array
(collection of objects of the same type), except the length is not known at
compile time. In other words, the `accounts` and `instruction_data` arguments
expect inputs of unknown length.
```rust theme={null}
fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
)
```
We can then have our functions return values by declaring the return type using
an arrow `->` after the function.
In the example below, the `process_instruction` function will now return a value
of type `ProgramResult`. We will go over this in the next section.
```rust theme={null}
fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> ProgramResult
```
#### Result enum
`Result` is a standard library type that represents two discrete outcomes:
success (`Ok`) or failure (`Err`). We'll talk more about enums in a future
lesson, but you'll see `Ok` used later in this lesson so it's important to cover
the basics.
When you use `Ok` or `Err`, you must include a value, the type of which is
determined by the context of the code. For example, a function that requires a
return value of type `Result` is saying that the function can
either return `Ok` with an embedded string value or `Err` with an embedded
integer. In this example, the integer is an error code that can be used to
appropriately handle the error.
To return a success case with a string value, you would do the following:
```rust theme={null}
Ok(String::from("Success!"));
```
To return an error with an integer, you would do the following:
```rust theme={null}
Err(404);
```
### Orbition Native Chain Programs
Recall that all data stored on the Orbition Native Chain network are contained in what are
referred to as accounts. Each account has its own unique address which is used
to identify and access the account data. Orbition Native Chain programs are just a particular
type of Orbition Native Chain account that store and execute instructions.
#### Orbition Native Chain Program Crate
To write Orbition Native Chain programs with Rust, we use the `solana_program` library crate.
The `solana_program` crate acts as a standard library for Orbition Native Chain programs. This
standard library contains the modules and macros that we'll use to develop our
Orbition Native Chain programs. If you want to dig deeper into the `solana_program` crate, have
a look
[at the `solana_program` crate documentation](https://docs.rs/solana-program/latest/solana_program/index.html).
For a basic program we will need to bring into scope the following items from
the `solana_program` crate:
```rust theme={null}
use solana_program::{
account_info::AccountInfo,
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
msg
};
```
* `AccountInfo` - a struct within the `account_info` module that allows us to
access account information
* `entrypoint` - a macro that declares the entry point of the program
* `ProgramResult` - a type within the `entrypoint` module that returns either
a `Result` or `ProgramError`
* `Pubkey` - a struct within the `pubkey` module that allows us to access
addresses as a public key
* `msg` - a macro that allows us to print messages to the program log
#### Orbition Native Chain Program Entry Point
Orbition Native Chain programs require a single entry point to process program instructions.
The entry point is declared using the `entrypoint!` macro.
The entry point to a Orbition Native Chain program requires a `process_instruction` function
with the following arguments:
* `program_id` - the address of the account where the program is stored
* `accounts` - the list of accounts required to process the instruction
* `instruction_data` - the serialized, instruction-specific data
```rust theme={null}
entrypoint!(process_instruction);
fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8],
) -> ProgramResult;
```
Recall that Orbition Native Chain program accounts only store the logic to process
instructions. This means program accounts are "read-only" and “stateless”. The
“state” (the set of data) that a program requires to process an instruction is
stored in data accounts (separate from the program account).
To process an instruction, the data accounts that an instruction requires must
be explicitly passed into the program through the `accounts` argument. Any
additional inputs must be passed in through the `instruction_data` argument.
Following program execution, the program must return a value of type
`ProgramResult`. This type is a `Result` where the embedded value of a success
case is `()` and the embedded value of a failure case is `ProgramError`. `()` is
an empty value and `ProgramError` is an error type defined in the
`solana_program` crate.
...and there you have it - you now know all the things you need for the
foundations of creating a Orbition Native Chain program using Rust. Let’s practice what we’ve
learned so far!
## Lab
We're going to build a "Hello, World!" program using Orbition Native Chain Playground. Orbition Native Chain
Playground is a tool that allows you to write and deploy Orbition Native Chain programs from
the browser.
#### 1. Setup
Open the [Orbition Native Chain Playground](https://beta.solpg.io/). Next, go ahead and delete
everything in the default `lib.rs` file and create a Playground wallet.
#### 2. Orbition Native Chain Program Crate
First, let's bring into scope everything we’ll need from the `solana_program`
crate.
```rust theme={null}
use solana_program::{
account_info::AccountInfo,
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
msg
};
```
Next, let's set up the entry point to our program using the `entrypoint!` macro
and create the `process_instruction` function. The `msg!` macro then allows us
to print “Hello, world!” to the program log when the program is invoked.
#### 3. Entry Point
```rust theme={null}
entrypoint!(process_instruction);
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult{
msg!("Hello, world!");
Ok(())
}
```
All together, the “Hello, world!” program will look like this:
```rust theme={null}
use solana_program::{
account_info::AccountInfo,
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
msg
};
entrypoint!(process_instruction);
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult{
msg!("Hello, world!");
Ok(())
}
```
#### 4. Build and Deploy
Now let's build and deploy our program using Orbition Native Chain Playground.
#### 5. Invoke Program
Finally, let's invoke our program from the client side. The focus of this lesson
is to build our Orbition Native Chain program, so we’ve gone ahead and provided
[the client code to invoke our “Hello, world!” program](https://github.com/Unboxed-Software/solana-hello-world-client)
for you to download.
The code provided includes a `sayHello` helper function that builds and submits
our transaction. We then call `sayHello` in the main function and print a Orbition Native Chain
Explorer URL to view our transaction details in the browser.
Open the `index.ts` file you should see a variable named `programId`. Go ahead
and update this with the program ID of the “Hello, world!" program you just
deployed using Orbition Native Chain Playground.
```typescript theme={null}
let programId = new web3.PublicKey("");
```
You can locate the program ID on Orbition Native Chain Playground referencing the image below.
Next, install the Node modules with `npm i`.
Now, go ahead and run `npm start`. This command will:
1. Generate a new keypair and create a `.env` file if one does not already exist
2. Airdrop testnet OBN
3. Invoke the “Hello, world!” program
4. Output the transaction URL to view on Orbition Native Chain Explorer
Copy the transaction URL printed in the console into your browser. Scroll down
to see “Hello, world!” under Program Instruction Logs.
Congratulations, you’ve just successfully built and deployed a Orbition Native Chain program!
## Challenge
Now it’s your turn to build something independently. Because we're starting with
very simple programs, yours will look almost identical to what we just created.
It's useful to try and get to the point where you can write it from scratch
without referencing prior code, so try not to copy and paste here.
1. Write a new program that uses the `msg!` macro to print your own message to
the program log.
2. Build and deploy your program like we did in the lab.
3. Invoke your newly deployed program and use Orbition Native Chain Explorer to check that your
message was printed in the program log.
As always, get creative with these challenges and take them beyond the basic
instructions if you want - and have fun!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=5b56c69c-1490-46e4-850f-a7e37bbd79c2)!
# Page, Order, and Filter Program Data
Source: https://docs.orbition.network/courses/native-onchain-development/paging-ordering-filtering-data-frontend
Learn how to efficiently query account data from Orbition Native Chain.
## Summary
* This lesson delves into some functionality of the RPC calls that we used in
the deserializing account data lesson
* To save on computing time, you can fetch a large number of accounts without
their data by filtering them to return just an array of public keys
* Once you have a filtered list of public keys, you can order them and fetch the
account data they belong to
## Lesson
You may have noticed in the last lesson that while we could fetch and display a
list of account data, we didn’t have any granular control over how many accounts
to fetch or their order. In this lesson, we’ll learn about some configuration
options for the `getProgramAccounts` function that will enable things like
paging, ordering accounts, and filtering.
### Use `dataSlice` to only fetch the data you need
Imagine the Movie Review app we worked on in past lessons having four million
movie reviews and the average review is 500 bytes. That would make the total
download for all review accounts over 2GB. Not something you want to have your
frontend download every time the page refreshes.
Fortunately, the `getProgramAccounts` function that you use to get all of the
accounts takes a configuration object as an argument. One of the configuration
options is `dataSlice` which lets you provide two things:
* `offset` - the offset from the beginning of the data buffer to start slicing
* `length` - the number of bytes to return, starting from the provided offset
When you include a `dataSlice` in the configuration object, the function will
only return the subset of the data buffer that you specified.
#### Paging Accounts
One area where this becomes helpful is with paging. If you want to have a list
that displays all accounts but there are so many accounts that you don’t want to
pull all the data at once, you can fetch all of the accounts but not fetch their
data by using a `dataSlice` of `{ offset: 0, length: 0 }`. You can then map the
result to a list of account keys whose data you can fetch only when needed.
```tsx theme={null}
const accountsWithoutData = await connection.getProgramAccounts(programId, {
dataSlice: { offset: 0, length: 0 },
});
const accountKeys = accountsWithoutData.map(account => account.pubkey);
```
With this list of keys, you can then fetch account data in “pages” using the
`getMultipleAccountsInfo` method:
```tsx theme={null}
const paginatedKeys = accountKeys.slice(0, 10);
const accountInfos = await connection.getMultipleAccountsInfo(paginatedKeys);
const deserializedObjects = accountInfos.map(accountInfo => {
// put logic to deserialize accountInfo.data here
});
```
#### Ordering Accounts
The `dataSlice` option is also helpful when you need to order a list of accounts
while paging. You still don’t want to fetch all the data at once, but you do
need all of the keys and a way to order them upfront. In this case, you need to
understand the layout of the account data and configure the data slice to only
be the data you need to use for ordering.
For example, you might have an account that stores contact information like so:
* `initialized` as a boolean
* `phoneNumber` as an unsigned, 64-bit integer
* `firstName` as a string
* `secondName` as a string
If you want to order all of the account keys alphabetically based on the user’s
first name, you need to find out the offset where the name starts. The first
field, `initialized`, takes the first byte, then `phoneNumber` takes another 8,
so the `firstName` field starts at offset `1 + 8 = 9`. However, dynamic data
fields in borsh use the first 4 bytes to record the length of the data, so we
can skip an additional 4 bytes, making the offset 13.
You then need to determine the length to make the data slice. Since the length
is variable, we can’t know for sure before fetching the data. But you can choose
a length that is large enough to cover most cases and short enough to not be too
much of a burden to fetch. 15 bytes is plenty for most first names but would
result in a small enough download even with a million users.
Once you’ve fetched accounts with the given data slice, you can use the `sort`
method to sort the array before mapping it to an array of public keys.
```tsx theme={null}
const accounts = await connection.getProgramAccounts(programId, {
dataSlice: { offset: 13, length: 15 },
});
accounts.sort((a, b) => {
const lengthA = a.account.data.readUInt32LE(0);
const lengthB = b.account.data.readUInt32LE(0);
const dataA = a.account.data.slice(4, 4 + lengthA);
const dataB = b.account.data.slice(4, 4 + lengthB);
return dataA.compare(dataB);
});
const accountKeys = accounts.map(account => account.pubkey);
```
Note that in the snippet above we don’t compare the data as given. This is
because for dynamically sized types like strings, Borsh places an unsigned,
32-bit (4 byte) integer at the start to indicate the length of the data
representing that field. So to compare the first names directly, we need to get
the length for each, then create a data slice with a 4 byte offset and the
proper length.
### Use `filters` to only retrieve specific accounts
Limiting the data received per account is great, but what if you only want to
return accounts that match a specific criteria rather than all of them? That’s
where the `filters` configuration option comes in. This option is an array that
can have objects matching the following:
* `memcmp` - compares a provided series of bytes with program account data at a
particular offset. Fields:
* `offset` - the number to offset into program account data before comparing
data
* `bytes` - a base-58 encoded string representing the data to match; limited
to less than 129 bytes
* `dataSize` - compares the program account data length with the provided data
size
These let you filter based on matching data and/or total data size.
For example, you could search through a list of contacts by including a `memcmp`
filter:
```tsx theme={null}
async function fetchMatchingContactAccounts(
connection: web3.Connection,
search: string,
): Promise<(web3.AccountInfo | null)[]> {
const accounts = await connection.getProgramAccounts(programId, {
dataSlice: { offset: 0, length: 0 },
filters: [
{
memcmp: {
offset: 13,
bytes: bs58.encode(Buffer.from(search)),
},
},
],
});
}
```
Two things to note in the example above:
1. We’re setting the offset to 13 because we determined previously that the
offset for `firstName` in the data layout is 9 and we want to additionally
skip the first 4 bytes indicating the length of the string.
2. We’re using a third-party library
`bs58`` to perform base-58 encoding on the search term. You can install it using `npm
install bs58\`.
## Lab
Remember that Movie Review app we worked on in the last two lessons? We’re going
to spice it up a little by paging the review list, ordering the reviews so they
aren’t so random, and adding some basic search functionality. No worries if
you’re just jumping into this lesson without having looked at the previous
ones - as long as you have the prerequisite knowledge, you should be able to
follow the lab without having worked in this specific project yet.
#### **1. Download the starter code**
If you didn’t complete the lab from the last lesson or just want to make sure
that you didn’t miss anything, you can download the
[starter code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-deserialize-account-data).
The project is a fairly simple Next.js application. It includes the
`WalletContextProvider` we created in the Wallets lesson, a `Card` component for
displaying a movie review, a `MovieList` component that displays reviews in a
list, a `Form` component for submitting a new review, and a `Movie.ts` file that
contains a class definition for a `Movie` object.
#### 2. Add paging to the reviews
First things first, let’s create a space to encapsulate the code for fetching
account data. Create a new file `MovieCoordinator.ts` and declare a
`MovieCoordinator` class. Then let’s move the `MOVIE_REVIEW_PROGRAM_ID` constant
from `MovieList` into this new file since we’ll be moving all references to it
```tsx theme={null}
const MOVIE_REVIEW_PROGRAM_ID = "CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN";
export class MovieCoordinator {}
```
Now we can use `MovieCoordinator` to create a paging implementation. A quick
note before we dive in: this will be as simple a paging implementation as
possible so that we can focus on the complex part of interacting with Orbition Native Chain
accounts. You can, and should, do better for a production application.
With that out of the way, let’s create a static property `accounts` of type
`web3.PublicKey[]`, a static function
`prefetchAccounts(connection: web3.Connection)`, and a static function
`fetchPage(connection: web3.Connection, page: number, perPage: number): Promise`.
You’ll also need to import `@orbition-network/web3.js` and `Movie`.
```tsx theme={null}
import * as web3 from "@orbition-network/web3.js";
import { Movie } from "../models/Movie";
const MOVIE_REVIEW_PROGRAM_ID = "CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN";
export class MovieCoordinator {
static accounts: web3.PublicKey[] = [];
static async prefetchAccounts(connection: web3.Connection) {}
static async fetchPage(
connection: web3.Connection,
page: number,
perPage: number,
): Promise {}
}
```
The key to paging is to prefetch all the accounts without data. Let’s fill in
the body of `prefetchAccounts` to do this and set the retrieved public keys to
the static `accounts` property.
```tsx theme={null}
static async prefetchAccounts(connection: web3.Connection) {
const accounts = await connection.getProgramAccounts(
new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
{
dataSlice: { offset: 0, length: 0 },
}
)
this.accounts = accounts.map(account => account.pubkey)
}
```
Now, let’s fill in the `fetchPage` method. First, if the accounts haven’t been
prefetched yet, we’ll need to do that. Then, we can get the account public keys
that correspond to the requested page and call
`connection.getMultipleAccountsInfo`. Finally, we deserialize the account data
and return the corresponding `Movie` objects.
```tsx theme={null}
static async fetchPage(connection: web3.Connection, page: number, perPage: number): Promise {
if (this.accounts.length === 0) {
await this.prefetchAccounts(connection)
}
const paginatedPublicKeys = this.accounts.slice(
(page - 1) * perPage,
page * perPage,
)
if (paginatedPublicKeys.length === 0) {
return []
}
const accounts = await connection.getMultipleAccountsInfo(paginatedPublicKeys)
const movies = accounts.reduce((accum: Movie[], account) => {
const movie = Movie.deserialize(account?.data)
if (!movie) {
return accum
}
return [...accum, movie]
}, [])
return movies
}
```
With that done, we can reconfigure `MovieList` to use these methods. In
`MovieList.tsx`, add `const [page, setPage] = useState(1)` near the existing
`useState` calls. Then, update `useEffect` to call `MovieCoordinator.fetchPage`
instead of fetching the accounts inline.
```tsx theme={null}
const { connection } = useConnection();
const [movies, setMovies] = useState([]);
const [page, setPage] = useState(1);
useEffect(() => {
MovieCoordinator.fetchPage(connection, page, 10).then(setMovies);
}, [page]);
```
Lastly, we need to add buttons to the bottom of the list for navigating to
different pages:
```tsx theme={null}
return (
);
```
At this point, you should be able to run the project and click between pages!
#### 3. Order reviews alphabetically by title
If you look at the reviews, you might notice they aren’t in any specific order.
We can fix this by adding back just enough data into our data slice to help us
do some sorting. The various properties in the movie review data buffer are laid
out as follows
* `initialized` - unsigned 8-bit integer; 1 byte
* `rating` - unsigned 8-bit integer; 1 byte
* `title` - string; unknown number of bytes
* `description` - string; unknown number of bytes
Based on this, the offset we need to provide to the data slice to access `title`
is 2. The length, however, is indeterminate, so we can just provide what seems
to be a reasonable length. I’ll stick with 18 as that will cover the length of
most titles without fetching too much data every time.
Once we’ve modified the data slice in `getProgramAccounts`, we then need to
actually sort the returned array. To do this, we need to compare the part of the
data buffer that actually corresponds to `title`. The first 4 bytes of a dynamic
field in Borsh are used to store the length of the field in bytes. So in any
given buffer `data` that is sliced the way we discussed above, the string
portion is `data.slice(4, 4 + data[0])`.
Now that we’ve thought through this, let’s modify the implementation of
`prefetchAccounts` in `MovieCoordinator`:
```tsx theme={null}
static async prefetchAccounts(connection: web3.Connection, filters: AccountFilter[]) {
const accounts = await connection.getProgramAccounts(
new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
{
dataSlice: { offset: 2, length: 18 },
}
)
accounts.sort( (a, b) => {
const lengthA = a.account.data.readUInt32LE(0)
const lengthB = b.account.data.readUInt32LE(0)
const dataA = a.account.data.slice(4, 4 + lengthA)
const dataB = b.account.data.slice(4, 4 + lengthB)
return dataA.compare(dataB)
})
this.accounts = accounts.map(account => account.pubkey)
}
```
And just like that, you should be able to run the app and see the list of movie
reviews ordered alphabetically.
#### 4. Add search
The last thing we’ll do to improve this app is to add some basic search
capability. Let’s add a `search` parameter to `prefetchAccounts` and reconfigure
the body of the function to use it.
We can use the `filters` property of the `config` parameter of
`getProgramAccounts` to filter accounts by specific data. The offset to the
`title` fields is 2, but the first 4 bytes are the length of the title so the
actual offset to the string itself is 6. Remember that the bytes need to be base
58 encoded, so let’s install and import `bs58`.
```tsx theme={null}
import bs58 from 'bs58'
...
static async prefetchAccounts(connection: web3.Connection, search: string) {
const accounts = await connection.getProgramAccounts(
new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
{
dataSlice: { offset: 2, length: 18 },
filters: search === '' ? [] : [
{
memcmp:
{
offset: 6,
bytes: bs58.encode(Buffer.from(search))
}
}
]
}
)
accounts.sort( (a, b) => {
const lengthA = a.account.data.readUInt32LE(0)
const lengthB = b.account.data.readUInt32LE(0)
const dataA = a.account.data.slice(4, 4 + lengthA)
const dataB = b.account.data.slice(4, 4 + lengthB)
return dataA.compare(dataB)
})
this.accounts = accounts.map(account => account.pubkey)
}
```
Now, add a `search` parameter to `fetchPage` and update its call to
`prefetchAccounts` to pass it along. We’ll also need to add a `reload` boolean
parameter to `fetchPage` so that we can force a refresh of the account
prefetching every time the search value changes.
```tsx theme={null}
static async fetchPage(connection: web3.Connection, page: number, perPage: number, search: string, reload: boolean = false): Promise {
if (this.accounts.length === 0 || reload) {
await this.prefetchAccounts(connection, search)
}
const paginatedPublicKeys = this.accounts.slice(
(page - 1) * perPage,
page * perPage,
)
if (paginatedPublicKeys.length === 0) {
return []
}
const accounts = await connection.getMultipleAccountsInfo(paginatedPublicKeys)
const movies = accounts.reduce((accum: Movie[], account) => {
const movie = Movie.deserialize(account?.data)
if (!movie) {
return accum
}
return [...accum, movie]
}, [])
return movies
}
```
With that in place, let’s update the code in `MovieList` to call this properly.
First, add `const [search, setSearch] = useState('')` near the other `useState`
calls. Then update the call to `MovieCoordinator.fetchPage` in the `useEffect`
to pass the `search` parameter and to reload when `search !== ''`.
```tsx theme={null}
const { connection } = useConnection();
const [movies, setMovies] = useState([]);
const [page, setPage] = useState(1);
const [search, setSearch] = useState("");
useEffect(() => {
MovieCoordinator.fetchPage(connection, page, 2, search, search !== "").then(
setMovies,
);
}, [page, search]);
```
Finally, add a search bar that will set the value of `search`:
```tsx theme={null}
return (
);
```
And that’s it! The app now has ordered reviews, paging, and search.
That was a lot to digest, but you made it through. If you need to spend some
more time with the concepts, feel free to reread the sections that were most
challenging for you and/or have a look at the
[solution code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-paging-account-data).
## Challenge
Now it’s your turn to try and do this on your own. Using the Student Intros app
from last lesson, add paging, ordering alphabetically by name, and searching by
name.
1. You can build this from scratch or you can download the
[starter code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-deserialize-account-data)
2. Add paging to the project by prefetching accounts without data, then only
fetching the account data for each account when it’s needed.
3. Order the accounts displayed in the app alphabetically by name.
4. Add the ability to search through introductions by a student’s name.
This is challenging. If you get stuck, feel free to reference the
[solution code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-paging-account-data).
As always, get creative with these challenges and take them beyond the
instructions if you want!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=9342ad0a-1741-41a5-9f68-662642c8ec93)!
# Program Derived Addresses (PDAs)
Source: https://docs.orbition.network/courses/native-onchain-development/program-derived-addresses
Get a deeper understanding of PDAs.
## Summary
* A **Program Derived Address** (PDA) is derived from a **program ID** and an
optional list of **seeds**
* PDAs are owned and controlled by the program they are derived from
* PDA derivation provides a deterministic way to find data based on the seeds
used for the derivation
* Seeds can be used to map to the data stored in a separate PDA account
* A program can sign instructions on behalf of the PDAs derived from its ID
## Lesson
### What is a Program Derived Address?
Program Derived Addresses (PDAs) are account addresses designed to be signed for
by a program rather than a secret key. As the name suggests, PDAs are derived
using a program ID. Optionally, these derived accounts can also be found using
the ID along with a set of "seeds." More on this later, but these seeds will
play an important role in how we use PDAs for data storage and retrieval.
PDAs serve two main functions:
1. Provide a deterministic way to find a given item of data for a program
2. Authorize the program from which a PDA was derived to sign on its behalf in
the same way a user may sign with their secret key
In this lesson we'll focus on using PDAs to find and store data. We'll discuss
signing with a PDA more thoroughly in a future lesson where we cover Cross
Program Invocations (CPIs).
### Finding PDAs
PDAs are not technically created. Rather, they are *found* or *derived* based on
a program ID and one or more input seeds.
Orbition Native Chain keypairs can be found on what is called the Ed25519 Elliptic Curve
(Ed25519). Ed25519 is a deterministic signature scheme that Orbition Native Chain uses to
generate corresponding public and secret keys. Together, we call these keypairs.
Alternatively, PDAs are addresses that lie *off* the Ed25519 curve. This means
PDAs are not public keys, and don't have private keys. This property of PDAs is
essential for programs to be able to sign on their behalf, but we'll cover that
in a future lesson.
To find a PDA within a Orbition Native Chain program, we'll use the `find_program_address`
function. This function takes an optional list of “seeds” and a program ID as
inputs, and then returns the PDA and a bump seed.
```rust theme={null}
let (pda, bump_seed) = Pubkey::find_program_address(&[user.key.as_ref(), user_input.as_bytes().as_ref(), "SEED".as_bytes()], program_id)
```
#### Seeds
“Seeds” are optional inputs used in the `find_program_address` function to
derive a PDA. For example, seeds can be any combination of public keys, inputs
provided by a user, or hardcoded values. A PDA can also be derived using only
the program ID and no additional seeds. Using seeds to find our PDAs, however,
allows us to create an arbitrary number of accounts that our program can own.
While you, the developer, determine the seeds to pass into the
`find_program_address` function, the function itself provides an additional seed
called a "bump seed." The cryptographic function for deriving a PDA results in a
key that lies *on* the Ed25519 curve about 50% of the time. To ensure that the
result *is not* on the Ed25519 curve and therefore does not have a secret key,
the `find_program_address` function adds a numeric seed called a bump seed.
The function starts by using the value `255` as the bump seed, then checks to
see if the output is a valid PDA. If the result is not a valid PDA, the function
decreases the bump seed by 1 and tries again (`255`, `254`, `253`, et cetera).
Once a valid PDA is found, the function returns both the PDA and the bump that
was used to derive the PDA.
#### Under the hood of `find_program_address`
Let's take a look at the source code for `find_program_address`.
```rust theme={null}
pub fn find_program_address(seeds: &[&[u8]], program_id: &Pubkey) -> (Pubkey, u8) {
Self::try_find_program_address(seeds, program_id)
.unwrap_or_else(|| panic!("Unable to find a viable program address bump seed"))
}
```
Under the hood, the `find_program_address` function passes the input `seeds` and
`program_id` to the `try_find_program_address` function.
The `try_find_program_address` function then introduces the `bump_seed`. The
`bump_seed` is a `u8` variable with a value ranging between 0 to 255. Iterating
over a descending range starting from 255, a `bump_seed` is appended to the
optional input seeds which are then passed to the `create_program_address`
function. If the output of `create_program_address` is not a valid PDA, then the
`bump_seed` is decreased by 1 and the loop continues until a valid PDA is found.
```rust theme={null}
pub fn try_find_program_address(seeds: &[&[u8]], program_id: &Pubkey) -> Option<(Pubkey, u8)> {
let mut bump_seed = [std::u8::MAX];
for _ in 0..std::u8::MAX {
{
let mut seeds_with_bump = seeds.to_vec();
seeds_with_bump.push(&bump_seed);
match Self::create_program_address(&seeds_with_bump, program_id) {
Ok(address) => return Some((address, bump_seed[0])),
Err(PubkeyError::InvalidSeeds) => (),
_ => break,
}
}
bump_seed[0] -= 1;
}
None
}
```
The `create_program_address` function performs a set of hash operations over the
seeds and `program_id`. These operations compute a key, then verify if the
computed key lies on the Ed25519 elliptic curve or not. If a valid PDA is found
(i.e. an address that is *off* the curve), then the PDA is returned. Otherwise,
an error is returned.
```rust theme={null}
pub fn create_program_address(
seeds: &[&[u8]],
program_id: &Pubkey,
) -> Result {
let mut hasher = crate::hash::Hasher::default();
for seed in seeds.iter() {
hasher.hash(seed);
}
hasher.hashv(&[program_id.as_ref(), PDA_MARKER]);
let hash = hasher.result();
if bytes_are_curve_point(hash) {
return Err(PubkeyError::InvalidSeeds);
}
Ok(Pubkey::new(hash.as_ref()))
}
```
In summary, the `find_program_address` function passes our input seeds and
`program_id` to the `try_find_program_address` function. The
`try_find_program_address` function adds a `bump_seed` (starting from 255) to
our input seeds, then calls the `create_program_address` function until a valid
PDA is found. Once found, both the PDA and the `bump_seed` are returned.
Note that for the same input seeds, different valid bumps will generate
different valid PDAs. The `bump_seed` returned by `find_program_address` will
always be the first valid PDA found. Because the function starts with a
`bump_seed` value of 255 and iterates downwards to zero, the `bump_seed` that
ultimately gets returned will always be the largest valid 8-bit value possible.
This `bump_seed` is commonly referred to as the "*canonical bump*". To avoid
confusion, it's recommended to only use the canonical bump, and to *always
validate every PDA passed into your program.*
One point to emphasize is that the `find_program_address` function only returns
a Program Derived Address and the bump seed used to derive it. The
`find_program_address` function does *not* initialize a new account, nor is any
PDA returned by the function necessarily associated with an account that stores
data.
### Use PDA accounts to store data
Since programs themselves are stateless, program state is managed through
external accounts. Given that you can use seeds for mapping and that programs
can sign on their behalf, using PDA accounts to store data related to the
program is an extremely common design choice. While programs can invoke the
System Program to create non-PDA accounts and use those to store data as well,
PDAs tend to be the way to go.
If you need a refresher on how to store data in PDAs, have a look at the
[State Management lesson](/content/courses/native-onchain-development/program-state-management).
### Map to data stored in PDA accounts
Storing data in PDA accounts is only half of the equation. You also need a way
to retrieve that data. We'll talk about two approaches:
1. Creating a PDA "map" account that stores the addresses of various accounts
where data is stored
2. Strategically using seeds to locate the appropriate PDA accounts and retrieve
the necessary data
#### Map to data using PDA "map" accounts
One approach to organizing data storage is to store clusters of relevant data in
their own PDAs and then to have a separate PDA account that stores a mapping of
where all of the data is.
For example, you might have a note-taking app whose backing program uses random
seeds to generate PDA accounts and stores one note in each account. The program
would also have a single global PDA "map" account that stores a mapping of
users' public keys to the list of PDAs where their notes are stored. This map
account would be derived using a static seed, e.g. "GLOBAL\_MAPPING".
When it comes time to retrieve a user's notes, you could then look at the map
account, see the list of addresses associated with a user's public key, then
retrieve the account for each of those addresses.
While such a solution is perhaps more approachable for traditional web
developers, it does come with some drawbacks that are particular to web3
development. Since the size of the mapping stored in the map account will grow
over time, you'll either need to allocate more size than necessary to the
account when you first create it, or you'll need to reallocate space for it
every time a new note is created. On top of that, you'll eventually reach the
account size limit of 10 megabytes.
You could mitigate this issue to some degree by creating a separate map account
for each user. For example, rather than having a single PDA map account for the
entire program, you would construct a PDA map account per user. Each of these
map accounts could be derived with the user's public key. The addresses for each
note could then be stored inside the corresponding user's map account.
This approach reduces the size required for each map account, but ultimately
still adds an unnecessary requirement to the process: having to read the
information on the map account *before* being able to find the accounts with the
relevant note data.
There may be times where using this approach makes sense for your application,
but we don't recommend it as your "go to" strategy.
#### Map to data using PDA derivation
If you're strategic about the seeds you use to derive PDAs, you can embed the
required mappings into the seeds themselves. This is the natural evolution of
the note-taking app example we just discussed. If you start to use the note
creator's public key as a seed to create one map account per user, then why not
use both the creator's public key and some other known piece of information to
derive a PDA for the note itself?
Now, without talking about it explicitly, we’ve been mapping seeds to accounts
this entire course. Think about the Movie Review program we've been built in
previous lessons. This program uses a review creator's public key and the title
of the movie they're reviewing to find the address that *should* be used to
store the review. This approach lets the program create a unique address for
every new review while also making it easy to locate a review when needed. When
you want to find a user's review of "Spiderman," you know that it is stored at
the PDA account whose address can be derived using the user's public key and the
text "Spiderman" as seeds.
```rust theme={null}
let (pda, bump_seed) = Pubkey::find_program_address(&[
initializer.key.as_ref(),
title.as_bytes().as_ref()
],
program_id)
```
#### Associated token account addresses
Another practical example of this type of mapping is how associated token
account (ATA) addresses are determined. Tokens are often held in an ATA whose
address was derived using a wallet address and the mint address of a specific
token. The address for an ATA is found using the `get_associated_token_address`
function which takes a `wallet_address` and `token_mint_address` as inputs.
```rust theme={null}
let associated_token_address = get_associated_token_address(&wallet_address, &token_mint_address);
```
Under the hood, the associated token address is a PDA found using the
`wallet_address`, `token_program_id`, and `token_mint_address` as seeds. This
provides a deterministic way to find a token account associated with any wallet
address for a specific token mint.
```rust theme={null}
fn get_associated_token_address_and_bump_seed_internal(
wallet_address: &Pubkey,
token_mint_address: &Pubkey,
program_id: &Pubkey,
token_program_id: &Pubkey,
) -> (Pubkey, u8) {
Pubkey::find_program_address(
&[
&wallet_address.to_bytes(),
&token_program_id.to_bytes(),
&token_mint_address.to_bytes(),
],
program_id,
)
}
```
The mappings between seeds and PDA accounts that you use will be highly
dependent on your specific program. While this isn't a lesson on system design
or architecture, it's worth calling out a few guidelines:
* Use seeds that will be known at the time of PDA derivation
* Be thoughtful about what data is grouped together into a single account
* Be thoughtful about the data structure used within each account
* Simpler is usually better
## Lab
Let’s practice together with the Movie Review program we've worked on in
previous lessons. No worries if you’re just jumping into this lesson without
having done the previous lesson - it should be possible to follow along either
way.
As a refresher, the Movie Review program lets users create movie reviews. These
reviews are stored in an account using a PDA derived with the initializer’s
public key and the title of the movie they are reviewing.
Previously, we finished implementing the ability to update a movie review in a
secure manner. In this lab, we'll add the ability for users to comment on a
movie review. We'll use building this feature as an opportunity to work through
how to structure the comment storage using PDA accounts.
#### 1. Get the starter code
To begin, you can find
[the movie program starter code](https://github.com/Unboxed-Software/solana-movie-program/tree/starter)
on the `starter` branch.
If you've been following along with the Movie Review labs, you'll notice that
this is the program we’ve built out so far. Previously, we
used [Orbition Native Chain Playground](https://beta.solpg.io/) to write, build, and deploy our
code. In this lesson, we’ll build and deploy the program locally.
Open the folder, then run `cargo-build-bpf` to build the program. The
`cargo-build-bpf` command will output instruction to deploy the program.
```sh theme={null}
cargo-build-bpf
```
Deploy the program by copying the output of `cargo-build-bpf` and running the
`solana program deploy` command.
```sh theme={null}
solana program deploy
```
You can test the program by using the movie review
[frontend](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-update-reviews)
and updating the program ID with the one you’ve just deployed. Make sure you use
the `solution-update-reviews` branch.
#### 2. Plan out the account structure
Adding comments means we need to make a few decisions about how to store the
data associated with each comment. The criteria for a good structure here are:
* Not overly complicated
* Data is easily retrievable
* Each comment has something to link it to the review it's associated with
To do this, we'll create two new account types:
* Comment counter account
* Comment account
There will be one comment counter account per review and one comment account per
comment. The comment counter account will be linked to a given review by using a
review's address as a seed for finding the comment counter PDA. It will also use
the static string "comment" as a seed.
The comment account will be linked to a review in the same way. However, it will
not include the "comment" string as a seed and will instead use the *actual
comment count* as a seed. That way the client can easily retrieve comments for a
given review by doing the following:
1. Read the data on the comment counter account to determine the number of
comments on a review.
2. Where `n` is the total number of comments on the review, loop `n` times. Each
iteration of the loop will derive a PDA using the review address and the
current number as seeds. The result is `n` number of PDAs, each of which is
the address of an account that stores a comment.
3. Fetch the accounts for each of the `n` PDAs and read the data stored in each.
This ensures that every one of our accounts can be deterministically retrieved
using data that is already known ahead of time.
To implement these changes, we'll need to do the following:
* Define structs to represent the comment counter and comment accounts
* Update the existing `MovieAccountState` to contain a discriminator (more on
this later)
* Add an instruction variant to represent the `add_comment` instruction
* Update the existing `add_movie_review` instruction processing function to
include creating the comment counter account
* Create a new `add_comment` instruction processing function
#### 3. Define `MovieCommentCounter` and `MovieComment` structs
Recall that the `state.rs` file defines the structs our program uses to populate
the data field of a new account.
We’ll need to define two new structs to enable commenting.
1. `MovieCommentCounter` - to store a counter for the number of comments
associated with a review
2. `MovieComment` - to store data associated with each comment
To start, let’s define the structs we’ll be using for our program. Note that we
are adding a `discriminator` field to each struct, including the existing
`MovieAccountState`. Since we now have multiple account types, we need a way to
only fetch the account type we need from the client. This discriminator is a
string that can be used to filter through accounts when we fetch our program
accounts.
```rust theme={null}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct MovieAccountState {
pub discriminator: String,
pub is_initialized: bool,
pub reviewer: Pubkey,
pub rating: u8,
pub title: String,
pub description: String,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct MovieCommentCounter {
pub discriminator: String,
pub is_initialized: bool,
pub counter: u64
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct MovieComment {
pub discriminator: String,
pub is_initialized: bool,
pub review: Pubkey,
pub commenter: Pubkey,
pub comment: String,
pub count: u64
}
impl Sealed for MovieAccountState {}
impl IsInitialized for MovieAccountState {
fn is_initialized(&self) -> bool {
self.is_initialized
}
}
impl IsInitialized for MovieCommentCounter {
fn is_initialized(&self) -> bool {
self.is_initialized
}
}
impl IsInitialized for MovieComment {
fn is_initialized(&self) -> bool {
self.is_initialized
}
}
```
Since we've added a new `discriminator` field to our existing struct, the
account size calculation needs to change. Let's use this as an opportunity to
clean up some of our code a bit. We'll add an implementation for each of the
three structs above that adds a constant `DISCRIMINATOR` and either a constant
`SIZE` or function `get_account_size` so we can quickly get the size needed when
initializing an account.
```rust theme={null}
impl MovieAccountState {
pub const DISCRIMINATOR: &'static str = "review";
pub fn get_account_size(title: String, description: String) -> usize {
return (4 + MovieAccountState::DISCRIMINATOR.len())
+ 1
+ 1
+ (4 + title.len())
+ (4 + description.len());
}
}
impl MovieCommentCounter {
pub const DISCRIMINATOR: &'static str = "counter";
pub const SIZE: usize = (4 + MovieCommentCounter::DISCRIMINATOR.len()) + 1 + 8;
}
impl MovieComment {
pub const DISCRIMINATOR: &'static str = "comment";
pub fn get_account_size(comment: String) -> usize {
return (4 + MovieComment::DISCRIMINATOR.len()) + 1 + 32 + 32 + (4 + comment.len()) + 8;
}
}
```
Now everywhere we need the discriminator or account size we can use this
implementation and not risk unintentional typos.
#### 4. Create `AddComment` instruction
Recall that the `instruction.rs` file defines the instructions our program will
accept and how to deserialize the data for each. We need to add a new
instruction variant for adding comments. Let’s start by adding a new variant
`AddComment` to the `MovieInstruction` enum.
```rust theme={null}
pub enum MovieInstruction {
AddMovieReview {
title: String,
rating: u8,
description: String
},
UpdateMovieReview {
title: String,
rating: u8,
description: String
},
AddComment {
comment: String
}
}
```
Next, let's create a `CommentPayload` struct to represent the instruction data
associated with this new instruction. Most of the data we'll include in the
account are public keys associated with accounts passed into the program, so the
only thing we actually need here is a single field to represent the comment
text.
```rust theme={null}
#[derive(BorshDeserialize)]
struct CommentPayload {
comment: String
}
```
Now let’s update how we unpack the instruction data. Notice that we’ve moved the
deserialization of instruction data into each matching case using the associated
payload struct for each instruction.
```rust theme={null}
impl MovieInstruction {
pub fn unpack(input: &[u8]) -> Result {
let (&variant, rest) = input.split_first().ok_or(ProgramError::InvalidInstructionData)?;
Ok(match variant {
0 => {
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
Self::AddMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description }
},
1 => {
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
Self::UpdateMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description
}
},
2 => {
let payload = CommentPayload::try_from_slice(rest).unwrap();
Self::AddComment {
comment: payload.comment
}
}
_ => return Err(ProgramError::InvalidInstructionData)
})
}
}
```
Lastly, let's update the `process_instruction` function in `processor.rs` to use
the new instruction variant we've created.
In `processor.rs`, bring into scope the new structs from `state.rs`.
```rust theme={null}
use crate::state::{MovieAccountState, MovieCommentCounter, MovieComment};
```
Then in `process_instruction` let’s match our deserialized `AddComment`
instruction data to the `add_comment` function we’ll be implementing shortly.
```rust theme={null}
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult {
let instruction = MovieInstruction::unpack(instruction_data)?;
match instruction {
MovieInstruction::AddMovieReview { title, rating, description } => {
add_movie_review(program_id, accounts, title, rating, description)
},
MovieInstruction::UpdateMovieReview { title, rating, description } => {
update_movie_review(program_id, accounts, title, rating, description)
},
MovieInstruction::AddComment { comment } => {
add_comment(program_id, accounts, comment)
}
}
}
```
#### 5. Update `add_movie_review` to create comment counter account
Before we implement the `add_comment` function, we need to update the
`add_movie_review` function to create the review's comment counter account.
Remember that this account will keep track of the total number of comments that
exist for an associated review. It's address will be a PDA derived using the
movie review address and the word “comment” as seeds. Note that how we store the
counter is simply a design choice. We could also add a “counter” field to the
original movie review account.
Within the `add_movie_review` function, let’s add a `pda_counter` to represent
the new counter account we’ll be initializing along with the movie review
account. This means we now expect four accounts to be passed into
the `add_movie_review` function through the `accounts` argument.
```rust theme={null}
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
let pda_counter = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
```
Next, there's a check to make sure `total_len` is less than 1000 bytes, but
`total_len` is no longer accurate since we added the discriminator. Let's
replace `total_len` with a call to `MovieAccountState::get_account_size`:
```rust theme={null}
let account_len: usize = 1000;
if MovieAccountState::get_account_size(title.clone(), description.clone()) > account_len {
msg!("Data length is larger than 1000 bytes");
return Err(ReviewError::InvalidDataLength.into());
}
```
Note that this also needs to be updated in the `update_movie_review` function
for that instruction to work properly.
Once we’ve initialized the review account, we’ll also need to update the
`account_data` with the new fields we specified in the `MovieAccountState`
struct.
```rust theme={null}
account_data.discriminator = MovieAccountState::DISCRIMINATOR.to_string();
account_data.reviewer = *initializer.key;
account_data.title = title;
account_data.rating = rating;
account_data.description = description;
account_data.is_initialized = true;
```
Finally, let’s add the logic to initialize the counter account within the
`add_movie_review` function. This means:
1. Calculating the rent exemption amount for the counter account
2. Deriving the counter PDA using the review address and the string "comment" as
seeds
3. Invoking the system program to create the account
4. Set the starting counter value
5. Serialize the account data and return from the function
All of this should be added to the end of the `add_movie_review` function before
the `Ok(())`.
```rust theme={null}
msg!("create comment counter");
let rent = Rent::get()?;
let counter_rent_lamports = rent.minimum_balance(MovieCommentCounter::SIZE);
let (counter, counter_bump) =
Pubkey::find_program_address(&[pda.as_ref(), "comment".as_ref()], program_id);
if counter != *pda_counter.key {
msg!("Invalid seeds for PDA");
return Err(ProgramError::InvalidArgument);
}
invoke_signed(
&system_instruction::create_account(
initializer.key,
pda_counter.key,
counter_rent_lamports,
MovieCommentCounter::SIZE.try_into().unwrap(),
program_id,
),
&[
initializer.clone(),
pda_counter.clone(),
system_program.clone(),
],
&[&[pda.as_ref(), "comment".as_ref(), &[counter_bump]]],
)?;
msg!("comment counter created");
let mut counter_data =
try_from_slice_unchecked::(&pda_counter.data.borrow()).unwrap();
msg!("checking if counter account is already initialized");
if counter_data.is_initialized() {
msg!("Account already initialized");
return Err(ProgramError::AccountAlreadyInitialized);
}
counter_data.discriminator = MovieCommentCounter::DISCRIMINATOR.to_string();
counter_data.counter = 0;
counter_data.is_initialized = true;
msg!("comment count: {}", counter_data.counter);
counter_data.serialize(&mut &mut pda_counter.data.borrow_mut()[..])?;
```
Now when a new review is created, two accounts are initialized:
1. The first is the review account that stores the contents of the review. This
is unchanged from the version of the program we started with.
2. The second account stores the counter for comments
#### 6. Implement `add_comment`
Finally, let’s implement our `add_comment` function to create new comment
accounts.
When a new comment is created for a review, we will increment the count on the
comment counter PDA account and derive the PDA for the comment account using the
review address and current count.
Like in other instruction processing functions, we'll start by iterating through
accounts passed into the program. Then before we do anything else we need to
deserialize the counter account so we have access to the current comment count:
```rust theme={null}
pub fn add_comment(
program_id: &Pubkey,
accounts: &[AccountInfo],
comment: String
) -> ProgramResult {
msg!("Adding Comment...");
msg!("Comment: {}", comment);
let account_info_iter = &mut accounts.iter();
let commenter = next_account_info(account_info_iter)?;
let pda_review = next_account_info(account_info_iter)?;
let pda_counter = next_account_info(account_info_iter)?;
let pda_comment = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
let mut counter_data = try_from_slice_unchecked::(&pda_counter.data.borrow()).unwrap();
Ok(())
}
```
Now that we have access to the counter data, we can continue with the remaining
steps:
1. Calculate the rent exempt amount for the new comment account
2. Derive the PDA for the comment account using the review address and the
current comment count as seeds
3. Invoke the System Program to create the new comment account
4. Set the appropriate values to the newly created account
5. Serialize the account data and return from the function
```rust theme={null}
pub fn add_comment(
program_id: &Pubkey,
accounts: &[AccountInfo],
comment: String
) -> ProgramResult {
msg!("Adding Comment...");
msg!("Comment: {}", comment);
let account_info_iter = &mut accounts.iter();
let commenter = next_account_info(account_info_iter)?;
let pda_review = next_account_info(account_info_iter)?;
let pda_counter = next_account_info(account_info_iter)?;
let pda_comment = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
let mut counter_data = try_from_slice_unchecked::(&pda_counter.data.borrow()).unwrap();
let account_len = MovieComment::get_account_size(comment.clone());
let rent = Rent::get()?;
let rent_lamports = rent.minimum_balance(account_len);
let (pda, bump_seed) = Pubkey::find_program_address(&[pda_review.key.as_ref(), counter_data.counter.to_be_bytes().as_ref(),], program_id);
if pda != *pda_comment.key {
msg!("Invalid seeds for PDA");
return Err(ReviewError::InvalidPDA.into())
}
invoke_signed(
&system_instruction::create_account(
commenter.key,
pda_comment.key,
rent_lamports,
account_len.try_into().unwrap(),
program_id,
),
&[commenter.clone(), pda_comment.clone(), system_program.clone()],
&[&[pda_review.key.as_ref(), counter_data.counter.to_be_bytes().as_ref(), &[bump_seed]]],
)?;
msg!("Created Comment Account");
let mut comment_data = try_from_slice_unchecked::(&pda_comment.data.borrow()).unwrap();
msg!("checking if comment account is already initialized");
if comment_data.is_initialized() {
msg!("Account already initialized");
return Err(ProgramError::AccountAlreadyInitialized);
}
comment_data.discriminator = MovieComment::DISCRIMINATOR.to_string();
comment_data.review = *pda_review.key;
comment_data.commenter = *commenter.key;
comment_data.comment = comment;
comment_data.is_initialized = true;
comment_data.serialize(&mut &mut pda_comment.data.borrow_mut()[..])?;
msg!("Comment Count: {}", counter_data.counter);
counter_data.counter += 1;
counter_data.serialize(&mut &mut pda_counter.data.borrow_mut()[..])?;
Ok(())
}
```
#### 7. Build and deploy
We're ready to build and deploy our program!
Build the updated program by running `cargo-build-bpf`. Then deploy the program
by running the `solana program deploy` command printed to the console.
You can test your program by submitting a transaction with the right instruction
data. You can create your own script or feel free to use
[this frontend](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-add-comments).
Be sure to use the `solution-add-comments` branch and replace the
`MOVIE_REVIEW_PROGRAM_ID` in `utils/constants.ts` with your program's ID or the
frontend won't work with your program.
Keep in mind that we made breaking changes to the review accounts (i.e. adding a
discriminator). If you were to use the same program ID that you've used
previously when deploying this program, none of the reviews you created
previously will show on this frontend due to a data mismatch.
If you need more time with this project to feel comfortable with these concepts,
have a look at
the [solution code](https://github.com/Unboxed-Software/solana-movie-program/tree/solution-add-comments)
before continuing. Note that the solution code is on the `solution-add-comments`
branch of the linked repository.
## Challenge
Now it’s your turn to build something independently! Go ahead and work with the
Student Intro program that we've used in past lessons. The Student Intro program
is a Orbition Native Chain program that lets students introduce themselves. This program takes
a user's name and a short message as the `instruction_data` and creates an
account to store the data onchain. For this challenge you should:
1. Add an instruction allowing other users to reply to an intro
2. Build and deploy the program locally
If you haven't been following along with past lessons or haven't saved your work
from before, feel free to use the starter code on the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-student-intro-program/tree/starter).
Try to do this independently if you can! If you get stuck though, feel free to
reference the
[solution code](https://github.com/Unboxed-Software/solana-student-intro-program/tree/solution-add-replies).
Note that the solution code is on the `solution-add-replies` branch and that
your code may look slightly different.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=89d367b4-5102-4237-a7f4-4f96050fe57e)!
# Create a Basic Program, Part 3 - Basic Security and Validation
Source: https://docs.orbition.network/courses/native-onchain-development/program-security
How to implement account checks and validate instruction data.
## Summary
* **Thinking like an attacker** means asking "How do I break this?"
* Perform **owner checks** to ensure that the provided account is owned by the
public key you expect, e.g. ensuring that an account you expect to be a PDA is
owned by `program_id`
* Perform **signer checks** to ensure that any account modification has been
signed by the right party or parties
* **Account validation** entails ensuring that provided accounts are the
accounts you expect them to be, e.g. deriving PDAs with the expected seeds to
make sure the address matches the provided account
* **Data validation** entails ensuring that any provided data meets the criteria
required by the program
## Lesson
In the last two lessons we worked through building a Movie Review program
together. The end result is pretty cool! It's exciting to get something working
in a new development environment.
Proper program development, however, doesn't end at "get it working." It's
important to think through the possible failure points in your code to mitigate
them. Failure points are where undesirable behavior in your code could
potentially occur. Whether the undesirable behavior happens due to users
interacting with your program in unexpected ways or bad actors intentionally
trying to exploit your program, anticipating failure points is essential to
secure program development.
Remember, **you have no control over the transactions that will be sent to your
program once it’s deployed**. You can only control how your program handles
them. While this lesson is far from a comprehensive overview of program
security, we'll cover some of the basic pitfalls to look out for.
### Think like an attacker
[Neodyme](https://workshop.neodyme.io/) gave a presentation at Breakpoint 2021
entitled "Think Like An Attacker: Bringing Smart Contracts to Their Break(ing)
Point." If there's one thing you take away from this lesson, it's that you
should think like an attacker.
In this lesson, of course, we cannot cover everything that could possibly go
wrong with your programs. Ultimately, every program will have different security
risks associated with it. While understanding common pitfalls is *essential* to
engineering good programs, it is *insufficient* for deploying secure ones. To
have the broadest security coverage possible, you have to approach your code
with the right mindset.
As Neodyme mentioned in their presentation, the right mindset requires moving
from the question "Is this broken?" to "How do I break this?" This is the first
and most essential step in understanding what your code *actually does* as
opposed to what you wrote it to do.
#### All programs can be broken
It's not a question of "if."
Rather, it's a question of "how much effort and dedication would it take."
Our job as developers is to close as many holes as possible and increase the
effort and dedication required to break our code. For example, in the Movie
Review program we built together over the last two lessons, we wrote code to
create new accounts to store movie reviews. If we take a closer look at the
code, however, we'll notice how the program also facilitates a lot of
unintentional behavior we could easily catch by asking "How do I break this?"
We'll dig into some of these problems and how to fix them in this lesson, but
remember that memorizing a few pitfalls isn't sufficient. It's up to you to
change your mindset toward security.
### Error handling
Before we dive into some of the common security pitfalls and how to avoid them,
it's important to know how to use errors in your program. While your code can
handle some issues gracefully, other issues will require that your program stop
execution and return a program error.
#### How to create errors
While the `solana_program` crate provides a `ProgramError` enum with a list of
generic errors we can use, it will often be useful to create your own. Your
custom errors will be able to provide more context and detail while you're
debugging your code.
We can define our own errors by creating an enum type listing the errors we want
to use. For example, the `NoteError` contains variants `Forbidden` and
`InvalidLength`. The enum is made into a Rust `Error` type by using the `derive`
attribute macro to implement the `Error` trait from the `thiserror` library.
Each error type also has its own `#[error("...")]` notation. This lets you
provide an error message for each particular error type.
```rust theme={null}
use solana_program::{program_error::ProgramError};
use thiserror::Error;
#[derive(Error)]
pub enum NoteError {
#[error("Wrong note owner")]
Forbidden,
#[error("Text is too long")]
InvalidLength,
}
```
#### How to return errors
The compiler expects errors returned by the program to be of type `ProgramError`
from the `solana_program` crate. That means we won't be able to return our
custom error unless we have a way to convert it into this type. The following
implementation handles conversion between our custom error and the
`ProgramError` type.
```rust theme={null}
impl From for ProgramError {
fn from(e: NoteError) -> Self {
ProgramError::Custom(e as u32)
}
}
```
To return the custom error from the program, simply use the `into()` method to
convert the error into an instance of `ProgramError`.
```rust theme={null}
if pda != *note_pda.key {
return Err(NoteError::Forbidden.into());
}
```
### Basic security checks
While these won't comprehensively secure your program, there are a few security
checks you can keep in mind to fill in some of the larger gaps in your code:
* Ownership checks - used to verify that an account is owned by the program
* Signer checks - used to verify that an account has signed a transaction
* General Account Validation - used to verify that an account is the expected
account
* Data Validation - used to verify the inputs provided by a user
#### Ownership checks
An ownership check verifies that an account is owned by the expected public key.
Let's use the note-taking app example that we've referenced in previous lessons.
In this app, users can create, update, and delete notes that are stored by the
program in PDA accounts.
When a user invokes the `update` instruction, they also provide a `pda_account`.
We presume the provided `pda_account` is for the particular note they want to
update, but the user can input any instruction data they want. They could even
potentially send data which matches the data format of a note account but was
not also created by the note-taking program. This security vulnerability is one
potential way to introduce malicious code.
The simplest way to avoid this problem is to always check that the owner of an
account is the public key you expect it to be. In this case, we expect the note
account to be a PDA account owned by the program itself. When this is not the
case, we can report it as an error accordingly.
```rust theme={null}
if note_pda.owner != program_id {
return Err(ProgramError::InvalidNoteAccount);
}
```
As a side note, using PDAs whenever possible is more secure than trusting
externally-owned accounts, even if they are owned by the transaction signer. The
only accounts that the program has complete control over are PDA accounts,
making them the most secure.
#### Signer checks
A signer check simply verifies that the right parties have signed a transaction.
In the note-taking app, for example, we would want to verify that the note
creator signed the transaction before we process the `update` instruction.
Otherwise, anyone can update another user's notes by simply passing in the
user's public key as the initializer.
```rust theme={null}
if !initializer.is_signer {
msg!("Missing required signature");
return Err(ProgramError::MissingRequiredSignature)
}
```
#### General account validation
In addition to checking the signers and owners of accounts, it's important to
ensure that the provided accounts are what your code expects them to be. For
example, you would want to validate that a provided PDA account's address can be
derived with the expected seeds. This ensures that it is the account you expect
it to be.
In the note-taking app example, that would mean ensuring that you can derive a
matching PDA using the note creator's public key and the ID as seeds (that's
what we're assuming was used when creating the note). That way a user couldn't
accidentally pass in a PDA account for the wrong note or, more importantly, that
the user isn't passing in a PDA account that represents somebody else's note
entirely.
```rust theme={null}
let (pda, bump_seed) = Pubkey::find_program_address(&[note_creator.key.as_ref(), id.as_bytes().as_ref(),], program_id);
if pda != *note_pda.key {
msg!("Invalid seeds for PDA");
return Err(ProgramError::InvalidArgument)
}
```
### Data validation
Similar to validating accounts, you should also validate any data provided by
the client.
For example, you may have a game program where a user can allocate character
attribute points to various categories. You may have a maximum limit in each
category of 100, in which case you would want to verify that the existing
allocation of points plus the new allocation doesn't exceed the maximum.
```rust theme={null}
if character.agility + new_agility > 100 {
msg!("Attribute points cannot exceed 100");
return Err(AttributeError::TooHigh.into())
}
```
Or, the character may have an allowance of attribute points they can allocate
and you want to make sure they don't exceed that allowance.
```rust theme={null}
if attribute_allowance < new_agility {
msg!("Trying to allocate more points than allowed");
return Err(AttributeError::ExceedsAllowance.into())
}
```
Without these checks, program behavior would differ from what you expect. In
some cases, however, it's more than just an issue of undefined behavior.
Sometimes failure to validate data can result in security loopholes that are
financially devastating.
For example, imagine that the character referenced in these examples is an NFT.
Further, imagine that the program allows the NFT to be staked to earn token
rewards proportional to the NFTs number of attribute points. Failure to
implement these data validation checks would allow a bad actor to assign an
obscenely high number of attribute points and quickly drain your treasury of all
the rewards that were meant to be spread more evenly amongst a larger pool of
stakers.
#### Integer overflow and underflow
Rust integers have fixed sizes. This means they can only support a specific
range of numbers. An arithmetic operation that results in a higher or lower
value than what is supported by the range will cause the resulting value to wrap
around. For example, a `u8` only supports numbers 0-255, so the result of
addition that would be 256 would actually be 0, 257 would be 1, etc.
This is always important to keep in mind, but especially so when dealing with
any code that represents true value, such as depositing and withdrawing tokens.
To avoid integer overflow and underflow, either:
1. Have logic in place that ensures overflow or underflow *cannot* happen or
2. Use checked math like `checked_add` instead of `+`
```rust theme={null}
let first_int: u8 = 5;
let second_int: u8 = 255;
let sum = first_int.checked_add(second_int);
```
## Lab
Let’s practice together with the Movie Review program we've worked on in
previous lessons. No worries if you’re just jumping into this lesson without
having done the previous lesson - it should be possible to follow along either
way.
As a refresher, the Movie Review program lets users store movie reviews in PDA
accounts. Last lesson, we finished implementing the basic functionality of
adding a movie review. Now, we'll add some security checks to the functionality
we've already created and add the ability to update a movie review in a secure
manner.
Just as before, we'll be using [Orbition Native Chain Playground](https://beta.solpg.io/) to
write, build, and deploy our code.
### 1. Get the starter code
To begin, you can find
[the movie review starter code](https://beta.solpg.io/62b552f3f6273245aca4f5c9).
If you've been following along with the Movie Review labs, you'll notice that
we've refactored our program.
The refactored starter code is almost the same as what it was before. Since
`lib.rs` was getting rather large and unwieldy, we've separated its code into 3
files: `lib.rs`, `entrypoint.rs`, and `processor.rs`. `lib.rs` now *only*
registers the code's modules, `entrypoint.rs` *only* defines and sets the
program's entrypoint, and `processor.rs` handles the program logic for
processing instructions. We've also added an `error.rs` file where we'll be
defining custom errors. The complete file structure is as follows:
* **lib.rs** - register modules
* **entrypoint.rs -** entry point to the program
* **instruction.rs -** serialize and deserialize instruction data
* **processor.rs -** program logic to process instructions
* **state.rs -** serialize and deserialize state
* **error.rs -** custom program errors
In addition to some changes to file structure, we've updated a small amount of
code that will let this lab be more focused on security without having you write
unnecessary boiler plate.
Since we'll be allowing updates to movie reviews, we also changed `account_len`
in the `add_movie_review` function (now in `processor.rs`). Instead of
calculating the size of the review and setting the account length to only as
large as it needs to be, we're simply going to allocate 1000 bytes to each
review account. This way, we don’t have to worry about reallocating size or
re-calculating rent when a user updates their movie review.
We went from this:
```rust theme={null}
let account_len: usize = 1 + 1 + (4 + title.len()) + (4 + description.len());
```
To this:
```rust theme={null}
let account_len: usize = 1000;
```
The [realloc](https://docs.rs/solana-sdk/latest/solana_sdk/account_info/struct.AccountInfo.html#method.realloc) method
was just recently enabled by Orbition Native Chain Labs which allows you to dynamically change
the size of your accounts. We will not be using this method for this lab, but
it’s something to be aware of.
Finally, we've also implemented some additional functionality for our
`MovieAccountState` struct in `state.rs` using the `impl` keyword.
For our movie reviews, we want the ability to check whether an account has
already been initialized. To do this, we create an `is_initialized` function
that checks the `is_initialized` field on the `MovieAccountState` struct.
`Sealed` is Orbition Native Chain's version of Rust's `Sized` trait. This simply specifies that
`MovieAccountState` has a known size and provides for some compiler
optimizations.
```rust theme={null}
// inside state.rs
impl Sealed for MovieAccountState {}
impl IsInitialized for MovieAccountState {
fn is_initialized(&self) -> bool {
self.is_initialized
}
}
```
Before moving on, make sure you have a solid grasp on the current state of the
program. Look through the code and spend some time thinking through any spots
that are confusing to you. It may be helpful to compare the starter code to the
[solution code from the previous lesson](https://beta.solpg.io/62b23597f6273245aca4f5b4).
### 2. Custom Errors
Let's begin by writing our custom program errors. We'll need errors that we can
use in the following situations:
* The update instruction has been invoked on an account that hasn't been
initialized yet
* The provided PDA doesn't match the expected or derived PDA
* The input data is larger than the program allows
* The rating provided does not fall in the 1-5 range
The starter code includes an empty `error.rs` file. Open that file and add
errors for each of the above cases.
```rust theme={null}
// inside error.rs
use solana_program::{program_error::ProgramError};
use thiserror::Error;
#[derive(Debug, Error)]
pub enum ReviewError{
// Error 0
#[error("Account not initialized yet")]
UninitializedAccount,
// Error 1
#[error("PDA derived does not equal PDA passed in")]
InvalidPDA,
// Error 2
#[error("Input data exceeds max length")]
InvalidDataLength,
// Error 3
#[error("Rating greater than 5 or less than 1")]
InvalidRating,
}
impl From for ProgramError {
fn from(e: ReviewError) -> Self {
ProgramError::Custom(e as u32)
}
}
```
Note that in addition to adding the error cases, we also added the
implementation that lets us convert our error into a `ProgramError` type as
needed.
Before moving on, let’s bring `ReviewError` into scope in the `processor.rs`. We
will be using these errors shortly when we add our security checks.
```rust theme={null}
// inside processor.rs
use crate::error::ReviewError;
```
### 3. Add security checks to `add_movie_review`
Now that we have errors to use, let's implement some security checks to our
`add_movie_review` function.
#### Signer check
The first thing we should do is ensure that the `initializer` of a review is
also a signer on the transaction. This ensures that you can't submit movie
reviews impersonating somebody else. We'll put this check right after iterating
through the accounts.
```rust theme={null}
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
if !initializer.is_signer {
msg!("Missing required signature");
return Err(ProgramError::MissingRequiredSignature)
}
```
#### Account validation
Next, let's make sure the `pda_account` passed in by the user is the `pda` we
expect. Recall we derived the `pda` for a movie review using the `initializer`
and `title` as seeds. Within our instruction we’ll derive the `pda` again and
then check if it matches the `pda_account`. If the addresses do not match, we’ll
return our custom `InvalidPDA` error.
```rust theme={null}
// Derive PDA and check that it matches client
let (pda, _bump_seed) = Pubkey::find_program_address(&[initializer.key.as_ref(), account_data.title.as_bytes().as_ref(),], program_id);
if pda != *pda_account.key {
msg!("Invalid seeds for PDA");
return Err(ReviewError::InvalidPDA.into())
}
```
#### Data validation
Now let's perform some data validation.
We'll start by making sure `rating` falls within the 1 to 5 scale. If the rating
provided by the user outside of this range, we’ll return our custom
`InvalidRating` error.
```rust theme={null}
if rating > 5 || rating < 1 {
msg!("Rating cannot be higher than 5");
return Err(ReviewError::InvalidRating.into())
}
```
Next, let’s check that the content of the review does not exceed the 1000 bytes
we’ve allocated for the account. If the size exceeds 1000 bytes, we’ll return
our custom `InvalidDataLength` error.
```rust theme={null}
let total_len: usize = 1 + 1 + (4 + title.len()) + (4 + description.len());
if total_len > 1000 {
msg!("Data length is larger than 1000 bytes");
return Err(ReviewError::InvalidDataLength.into())
}
```
Lastly, let's checking if the account has already been initialized by calling
the `is_initialized` function we implemented for our `MovieAccountState`. If the
account already exists, then we will return an error.
```rust theme={null}
if account_data.is_initialized() {
msg!("Account already initialized");
return Err(ProgramError::AccountAlreadyInitialized);
}
```
All together, the `add_movie_review` function should look something like this:
```rust theme={null}
pub fn add_movie_review(
program_id: &Pubkey,
accounts: &[AccountInfo],
title: String,
rating: u8,
description: String
) -> ProgramResult {
msg!("Adding movie review...");
msg!("Title: {}", title);
msg!("Rating: {}", rating);
msg!("Description: {}", description);
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
if !initializer.is_signer {
msg!("Missing required signature");
return Err(ProgramError::MissingRequiredSignature)
}
let (pda, bump_seed) = Pubkey::find_program_address(&[initializer.key.as_ref(), title.as_bytes().as_ref(),], program_id);
if pda != *pda_account.key {
msg!("Invalid seeds for PDA");
return Err(ProgramError::InvalidArgument)
}
if rating > 5 || rating < 1 {
msg!("Rating cannot be higher than 5");
return Err(ReviewError::InvalidRating.into())
}
let total_len: usize = 1 + 1 + (4 + title.len()) + (4 + description.len());
if total_len > 1000 {
msg!("Data length is larger than 1000 bytes");
return Err(ReviewError::InvalidDataLength.into())
}
let account_len: usize = 1000;
let rent = Rent::get()?;
let rent_lamports = rent.minimum_balance(account_len);
invoke_signed(
&system_instruction::create_account(
initializer.key,
pda_account.key,
rent_lamports,
account_len.try_into().unwrap(),
program_id,
),
&[initializer.clone(), pda_account.clone(), system_program.clone()],
&[&[initializer.key.as_ref(), title.as_bytes().as_ref(), &[bump_seed]]],
)?;
msg!("PDA created: {}", pda);
msg!("unpacking state account");
let mut account_data = try_from_slice_unchecked::(&pda_account.data.borrow()).unwrap();
msg!("borrowed account data");
msg!("checking if movie account is already initialized");
if account_data.is_initialized() {
msg!("Account already initialized");
return Err(ProgramError::AccountAlreadyInitialized);
}
account_data.title = title;
account_data.rating = rating;
account_data.description = description;
account_data.is_initialized = true;
msg!("serializing account");
account_data.serialize(&mut &mut pda_account.data.borrow_mut()[..])?;
msg!("state account serialized");
Ok(())
}
```
### 4. Support movie review updates in `MovieInstruction`
Now that `add_movie_review` is more secure, let's turn our attention to
supporting the ability to update a movie review.
Let’s begin by updating `instruction.rs`. We’ll start by adding an
`UpdateMovieReview` variant to `MovieInstruction` that includes embedded data
for the new title, rating, and description.
```rust theme={null}
// inside instruction.rs
pub enum MovieInstruction {
AddMovieReview {
title: String,
rating: u8,
description: String
},
UpdateMovieReview {
title: String,
rating: u8,
description: String
}
}
```
The payload struct can stay the same since aside from the variant type, the
instruction data is the same as what we used for `AddMovieReview`.
Lastly, in the `unpack` function we need to add `UpdateMovieReview` to the match
statement.
```rust theme={null}
// inside instruction.rs
impl MovieInstruction {
pub fn unpack(input: &[u8]) -> Result {
let (&variant, rest) = input.split_first().ok_or(ProgramError::InvalidInstructionData)?;
let payload = MovieReviewPayload::try_from_slice(rest).unwrap();
Ok(match variant {
0 => Self::AddMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description },
1 => Self::UpdateMovieReview {
title: payload.title,
rating: payload.rating,
description: payload.description },
_ => return Err(ProgramError::InvalidInstructionData)
})
}
}
```
### 5. Define `update_movie_review` function
Now that we can unpack our `instruction_data` and determine which instruction of
the program to run, we can add `UpdateMovieReview` to the match statement in
the `process_instruction` function in the `processor.rs` file.
```rust theme={null}
// inside processor.rs
pub fn process_instruction(
program_id: &Pubkey,
accounts: &[AccountInfo],
instruction_data: &[u8]
) -> ProgramResult {
// unpack instruction data
let instruction = MovieInstruction::unpack(instruction_data)?;
match instruction {
MovieInstruction::AddMovieReview { title, rating, description } => {
add_movie_review(program_id, accounts, title, rating, description)
},
// add UpdateMovieReview to match against our new data structure
MovieInstruction::UpdateMovieReview { title, rating, description } => {
// make call to update function that we'll define next
update_movie_review(program_id, accounts, title, rating, description)
}
}
}
```
Next, we can define the new `update_movie_review` function. The definition
should have the same parameters as the definition of `add_movie_review`.
```rust theme={null}
pub fn update_movie_review(
program_id: &Pubkey,
accounts: &[AccountInfo],
title: String,
rating: u8,
description: String
) -> ProgramResult {
}
```
### 6. Implement `update_movie_review` function
All that's left now is to fill in the logic for updating a movie review. Only
let's make it secure from the start.
Just like the `add_movie_review` function, let's start by iterating through the
accounts. The only accounts we'll need are the first two: `initializer` and
`pda_account`.
```rust theme={null}
pub fn update_movie_review(
program_id: &Pubkey,
accounts: &[AccountInfo],
title: String,
rating: u8,
description: String
) -> ProgramResult {
msg!("Updating movie review...");
// Get Account iterator
let account_info_iter = &mut accounts.iter();
// Get accounts
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
}
```
#### Ownership Check
Before we continue, let's implement some basic security checks. We'll start with
an ownership check on for `pda_account` to verify that it is owned by our
program. If it isn't, we'll return an `InvalidOwner` error.
```rust theme={null}
if pda_account.owner != program_id {
return Err(ProgramError::InvalidOwner)
}
```
#### Signer Check
Next, let’s perform a signer check to verify that the `initializer` of the
update instruction has also signed the transaction. Since we are updating the
data for a movie review, we want to ensure that the original `initializer` of
the review has approved the changes by signing the transaction. If the
`initializer` did not sign the transaction, we’ll return an error.
```rust theme={null}
if !initializer.is_signer {
msg!("Missing required signature");
return Err(ProgramError::MissingRequiredSignature)
}
```
#### Account Validation
Next, let’s check that the `pda_account` passed in by the user is the PDA we
expect by deriving the PDA using `initializer` and `title` as seeds. If the
addresses do not match, we’ll return our custom `InvalidPDA` error. We'll
implement this the same way we did in the `add_movie_review` function.
```rust theme={null}
// Derive PDA and check that it matches client
let (pda, _bump_seed) = Pubkey::find_program_address(&[initializer.key.as_ref(), account_data.title.as_bytes().as_ref(),], program_id);
if pda != *pda_account.key {
msg!("Invalid seeds for PDA");
return Err(ReviewError::InvalidPDA.into())
}
```
#### Unpack `pda_account` and perform data validation
Now that our code ensures we can trust the passed in accounts, let's unpack the
`pda_account` and perform some data validation. We'll start by unpacking
`pda_account` and assigning it to a mutable variable `account_data`.
```rust theme={null}
msg!("unpacking state account");
let mut account_data = try_from_slice_unchecked::(&pda_account.data.borrow()).unwrap();
msg!("borrowed account data");
```
Now that we have access to the account and its fields, the first thing we need
to do is verify that the account has already been initialized. An uninitialized
account can't be updated so the program should return our custom
`UninitializedAccount` error.
```rust theme={null}
if !account_data.is_initialized() {
msg!("Account is not initialized");
return Err(ReviewError::UninitializedAccount.into());
}
```
Next, we need to validate the `rating`, `title`, and `description` data just
like in the `add_movie_review` function. We want to limit the `rating` to a
scale of 1 to 5 and limit the overall size of the review to be fewer than 1000
bytes. If the rating provided by the user outside of this range, then we’ll
return our custom `InvalidRating` error. If the review is too long, then we'll
return our custom `InvalidDataLength` error.
```rust theme={null}
if rating > 5 || rating < 1 {
msg!("Rating cannot be higher than 5");
return Err(ReviewError::InvalidRating.into())
}
let total_len: usize = 1 + 1 + (4 + account_data.title.len()) + (4 + description.len());
if total_len > 1000 {
msg!("Data length is larger than 1000 bytes");
return Err(ReviewError::InvalidDataLength.into())
}
```
#### Update the movie review account
Now that we've implemented all of the security checks, we can finally update the
movie review account by updating `account_data` and re-serializing it. At that
point, we can return `Ok` from our program.
```rust theme={null}
account_data.rating = rating;
account_data.description = description;
account_data.serialize(&mut &mut pda_account.data.borrow_mut()[..])?;
Ok(())
```
All together, the `update_movie_review` function should look something like the
code snippet below. We've included some additional logging for clarity in
debugging.
```rust theme={null}
pub fn update_movie_review(
program_id: &Pubkey,
accounts: &[AccountInfo],
title: String,
rating: u8,
description: String
) -> ProgramResult {
msg!("Updating movie review...");
let account_info_iter = &mut accounts.iter();
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
if pda_account.owner != program_id {
return Err(ProgramError::IllegalOwner)
}
if !initializer.is_signer {
msg!("Missing required signature");
return Err(ProgramError::MissingRequiredSignature)
}
msg!("unpacking state account");
let mut account_data = try_from_slice_unchecked::(&pda_account.data.borrow()).unwrap();
msg!("review title: {}", account_data.title);
let (pda, _bump_seed) = Pubkey::find_program_address(&[initializer.key.as_ref(), account_data.title.as_bytes().as_ref(),], program_id);
if pda != *pda_account.key {
msg!("Invalid seeds for PDA");
return Err(ReviewError::InvalidPDA.into())
}
msg!("checking if movie account is initialized");
if !account_data.is_initialized() {
msg!("Account is not initialized");
return Err(ReviewError::UninitializedAccount.into());
}
if rating > 5 || rating < 1 {
msg!("Invalid Rating");
return Err(ReviewError::InvalidRating.into())
}
let update_len: usize = 1 + 1 + (4 + description.len()) + account_data.title.len();
if update_len > 1000 {
msg!("Data length is larger than 1000 bytes");
return Err(ReviewError::InvalidDataLength.into())
}
msg!("Review before update:");
msg!("Title: {}", account_data.title);
msg!("Rating: {}", account_data.rating);
msg!("Description: {}", account_data.description);
account_data.rating = rating;
account_data.description = description;
msg!("Review after update:");
msg!("Title: {}", account_data.title);
msg!("Rating: {}", account_data.rating);
msg!("Description: {}", account_data.description);
msg!("serializing account");
account_data.serialize(&mut &mut pda_account.data.borrow_mut()[..])?;
msg!("state account serialized");
Ok(())
}
```
### 7. Build and upgrade
We're ready to build and upgrade our program! You can test your program by
submitting a transaction with the right instruction data. For that, feel free to
use this
[frontend](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-update-reviews).
Remember, to make sure you're testing the right program you'll need to replace
`MOVIE_REVIEW_PROGRAM_ID` with your program ID in `Form.tsx` and
`MovieCoordinator.ts`.
If you need more time with this project to feel comfortable with these concepts,
have a look at the
[solution code](https://beta.solpg.io/62c8c6dbf6273245aca4f5e7) before
continuing.
## Challenge
Now it’s your turn to build something independently by building on top of the
Student Intro program that you've used in previous lessons. If you haven't been
following along or haven't saved your code from before, feel free to use
[this starter code](https://beta.solpg.io/62b11ce4f6273245aca4f5b2).
The Student Intro program is a Orbition Native Chain Program that lets students introduce
themselves. The program takes a user's name and a short message as the
instruction\_data and creates an account to store the data onchain.
Using what you've learned in this lesson, try applying what you've learned to
the Student Intro Program. The program should:
1. Add an instruction allowing students to update their message
2. Implement the basic security checks we've learned in this lesson
Try to do this independently if you can! But if you get stuck, feel free to
reference the [solution code](https://beta.solpg.io/62c9120df6273245aca4f5e8).
Note that your code may look slightly different than the solution code depending
on the checks you implement and the errors you write.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=3dfb98cc-7ba9-463d-8065-7bdb1c841d43)!
# Create a Basic Program, Part 2 - State Management
Source: https://docs.orbition.network/courses/native-onchain-development/program-state-management
Learn how programs store data, using Orbition Native Chain's inbuilt -key-value store.
## Summary
* Program state is stored in other accounts rather than in the program itself
* A Program Derived Address (PDA) is derived from a program ID and an optional
list of seeds. Once derived, PDAs are subsequently used as the address for a
storage account.
* Creating an account requires that we calculate the space required and the
corresponding rent to allocate for the new account
* Creating a new account requires a Cross Program Invocation (CPI) to the
`create_account` instruction on the System Program
* Updating the data field on an account requires that we serialize (convert to
byte array) the data into the account
## Lesson
Orbition Native Chain maintains speed, efficiency, and extensibility in part by making programs
stateless. Rather than having state stored on the program itself, programs use
Orbition Native Chain's account model to read state from and write state to separate PDA
accounts.
While this is an extremely flexible model, it's also a paradigm that can be
difficult to work in if its unfamiliar. But don't worry! We'll start simple in
this lesson and work up to more complex programs in the next unit.
In this lesson we'll learn the basics of state management for a Orbition Native Chain program,
including representing state as a Rust type, creating accounts using Program
Derived Addresses, and serializing account data.
### Program state
All Orbition Native Chain accounts have a `data` field that holds a byte array. This makes
accounts as flexible as files on a computer. You can store literally anything in
an account (so long as the account has the storage space for it).
Just as files in a traditional filesystem conform to specific data formats like
PDF or MP3, the data stored in a Orbition Native Chain account needs to follow some kind of
pattern so that the data can be retrieved and deserialized into something
usable.
#### Represent state as a Rust type
When writing a program in Rust, we typically create this "format" by defining a
Rust data type. If you went through the
[first part of this lesson](basic-program-pt-1), this is very similar to what we
did when we created an enum to represent discrete instructions.
While this type should reflect the structure of your data, for most use cases a
simple struct is sufficient. For example, a note-taking program that stores
notes in separate accounts would likely have data for a title, body, and maybe
an ID of some kind. We could create a struct to represent that as follows:
```rust theme={null}
struct NoteState {
title: String,
body: String,
id: u64
}
```
#### Using Borsh for serialization and deserialization
Just as with instruction data, we need a mechanism for converting from our Rust
data type to a byte array, and vice versa. **Serialization** is the process of
converting an object into a byte array. **Deserialization** is the process of
reconstructing an object from a byte array.
We'll continue to use Borsh for serialization and deserialization. In Rust, we
can use the `borsh` crate to get access to the `BorshSerialize` and
`BorshDeserialize` traits. We can then apply those traits using the `derive`
attribute macro.
```rust theme={null}
use borsh::{BorshSerialize, BorshDeserialize};
#[derive(BorshSerialize, BorshDeserialize)]
struct NoteState {
title: String,
body: String,
id: u64
}
```
These traits will provide methods on `NoteState` that we can use to serialize
and deserialize the data as needed.
### Creating accounts
Before we can update the data field of an account, we have to first create that
account.
To create a new account within our program we must:
1. Calculate the space and rent required for the account
2. Have an address to assign the new account
3. Invoke the system program to create the new account
#### Space and rent
Recall that storing data on the Orbition Native Chain network requires users to allocate rent
in the form of lamports. The amount of rent required by a new account depends on
the amount of space you would like allocated to that account. That means we need
to know before creating the account how much space to allocate.
Note that rent is more like a deposit. All the lamports allocated for rent can
be fully refunded when an account is closed. Additionally, all new accounts are
now required to be
[rent-exempt](https://twitter.com/jacobvcreech/status/1524790032938287105),
meaning lamports are not deducted from the account over time. An account is
considered rent-exempt if it holds at least 2 years worth of rent. In other
words, accounts are stored onchain permanently until the owner closes the
account and withdraws the rent.
In our note-taking app example, the `NoteState` struct specifies three fields
that need to be stored in an account: `title`, `body`, and `id`. To calculate
the size the account needs to be, you would simply add up the size required to
store the data in each field.
For dynamic data, like strings, Borsh adds an additional 4 bytes at the
beginning to store the length of that particular field. That means `title` and
`body` are each 4 bytes plus their respective sizes. The `id` field is a 64-bit
integer, or 8 bytes.
You can add up those lengths and then calculate the rent required for that
amount of space using the `minimum_balance` function from the `rent` module of
the `solana_program` crate.
```rust theme={null}
// Calculate account size required for struct NoteState
let account_len: usize = (4 + title.len()) + (4 + body.len()) + 8;
// Calculate rent required
let rent = Rent::get()?;
let rent_lamports = rent.minimum_balance(account_len);
```
#### Program Derived Addresses (PDA)
Before creating an account, we also need to have an address to assign the
account. For program owned accounts, this will be a program derived address
(PDA) found using the `find_program_address` function.
As the name implies, PDAs are derived using the program ID (address of the
program creating the account) and an optional list of “seeds”. Optional seeds
are additional inputs used in the `find_program_address` function to derive the
PDA. The function used to derive PDAs will return the same address every time
when given the same inputs. This gives us the ability to create any number of
PDA accounts and a deterministic way to find each account.
In addition to the seeds you provide for deriving a PDA, the
`find_program_address` function will provide one additional "bump seed." What
makes PDAs unique from other Orbition Native Chain account addresses is that they do not have a
corresponding secret key. This ensures that only the program that owns the
address can sign on behalf of the PDA. When the `find_program_address` function
attempts to derive a PDA using the provided seeds, it passes in the number 255
as the "bump seed." If the resulting address is invalid (i.e. has a
corresponding secret key), then the function decreases the bump seed by 1 and
derives a new PDA with that bump seed. Once a valid PDA is found, the function
returns both the PDA and the bump that was used to derive the PDA.
For our note-taking program, we will use the note creator's public key and the
ID as the optional seeds to derive the PDA. Deriving the PDA this way allows us
to deterministically find the account for each note.
```rust theme={null}
let (note_pda_account, bump_seed) = Pubkey::find_program_address(&[note_creator.key.as_ref(), id.as_bytes().as_ref(),], program_id);
```
#### Cross Program Invocation (CPI)
Once we’ve calculated the rent required for our account and found a valid PDA to
assign as the address of the new account, we are finally ready to create the
account. Creating a new account within our program requires a Cross Program
Invocation (CPI). A CPI is when one program invokes an instruction on another
program. To create a new account within our program, we will invoke the
`create_account` instruction on the system program.
CPIs can be done using either `invoke` or `invoke_signed`.
```rust theme={null}
pub fn invoke(
instruction: &Instruction,
account_infos: &[AccountInfo<'_>]
) -> ProgramResult
```
```rust theme={null}
pub fn invoke_signed(
instruction: &Instruction,
account_infos: &[AccountInfo<'_>],
signers_seeds: &[&[&[u8]]]
) -> ProgramResult
```
For this lesson we will use `invoke_signed`. Unlike a regular signature where a
secret key is used to sign, `invoke_signed` uses the optional seeds, bump seed,
and program ID to derive a PDA and sign an instruction. This is done by
comparing the derived PDA against all accounts passed into the instruction. If
any of the accounts match the PDA, then the signer field for that account is set
to true.
A program can securely sign transactions this way because `invoke_signed`
generates the PDA used for signing with the program ID of the program invoking
the instruction. Therefore, it is not possible for one program to generate a
matching PDA to sign for an account with a PDA derived using another program ID.
```rust theme={null}
invoke_signed(
// instruction
&system_instruction::create_account(
note_creator.key,
note_pda_account.key,
rent_lamports,
account_len.try_into().unwrap(),
program_id,
),
// account_infos
&[note_creator.clone(), note_pda_account.clone(), system_program.clone()],
// signers_seeds
&[&[note_creator.key.as_ref(), note_id.as_bytes().as_ref(), &[bump_seed]]],
)?;
```
### Serializing and deserializing account data
Once we've created a new account, we need to access and update the account's
data field. This means deserializing its byte array into an instance of the type
we created, updating the fields on that instance, then serializing that instance
back into a byte array.
#### Deserialize account data
The first step to updating an account's data is to deserialize its `data` byte
array into its Rust type. You can do this by first borrowing the data field on
the account. This allows you to access the data without taking ownership.
You can then use the `try_from_slice_unchecked` function to deserialize the data
field of the borrowed account using the format of the type you created to
represent the data. This gives you an instance of your Rust type so you can
easily update fields using dot notation. If we were to do this with the
note-taking app example we've been using, it would look like this:
```rust theme={null}
let mut account_data = try_from_slice_unchecked::(note_pda_account.data.borrow()).unwrap();
account_data.title = title;
account_data.body = rating;
account_data.id = id;
```
#### Serialize account data
Once the Rust instance representing the account's data has been updated with the
appropriate values, you can "save" the data on the account.
This is done with the `serialize` function on the instance of the Rust type you
created. You'll need to pass in a mutable reference to the account data. The
syntax here is tricky, so don't worry if you don't understand it completely.
Borrowing and references are two of the toughest concepts in Rust.
```rust theme={null}
account_data.serialize(&mut &mut note_pda_account.data.borrow_mut()[..])?;
```
The above example converts the `account_data` object to a byte array and sets it
to the `data` property on `note_pda_account`. This saves the updated
`account_data` variable to the data field of the new account. Now when a user
fetches the `note_pda_account` and deserializes the data, it will display the
updated data we’ve serialized into the account.
### Iterators
You may have noticed in the previous examples that we referenced `note_creator`
and didn't show where that came from.
To get access to this and other accounts, we use an
[Iterator](https://doc.rust-lang.org/std/iter/trait.Iterator.html). An iterator
is a Rust trait used to give sequential access to each element in a collection
of values. Iterators are used in Orbition Native Chain programs to safely iterate over the list
of accounts passed into the program entry point through the `accounts` argument.
#### Rust iterator
The iterator pattern allows you to perform some task on a sequence of items. The
`iter()` method creates an iterator object that references a collection. An
iterator is responsible for the logic of iterating over each item and
determining when the sequence has finished. In Rust, iterators are lazy, meaning
they have no effect until you call methods that consume the iterator to use it
up. Once you've created an iterator, you must call the `next()` function on it
to get the next item.
```rust theme={null}
let v1 = vec![1, 2, 3];
// create the iterator over the vec
let v1_iter = v1.iter();
// use the iterator to get the first item
let first_item = v1_iter.next();
// use the iterator to get the second item
let second_item = v1_iter.next();
```
#### Orbition Native Chain accounts iterator
Recall that the `AccountInfo` for all accounts required by an instruction are
passing through a single `accounts` argument. To parse through the accounts and
use them within our instruction, we will need to create an iterator with a
mutable reference to the `accounts`.
At that point, instead of using the iterator directly, we pass it to the
`next_account_info` function from the `account_info` module provided by the
`solana_program` crate.
For example, the instruction to create a new note in a note-taking program would
at minimum require the accounts for the user creating the note, a PDA to store
the note, and the `system_program` to initialize a new account. All three
accounts would be passed into the program entry point through the `accounts`
argument. An iterator of `accounts` is then used to separate out the
`AccountInfo` associated with each account to process the instruction.
Note that `&mut` means a mutable reference to the `accounts` argument. You can
read more about
[references in Rust](https://doc.rust-lang.org/book/ch04-02-references-and-borrowing.html)
and [the `mut` keyword](https://doc.rust-lang.org/std/keyword.mut.html).
```rust theme={null}
// Get Account iterator
let account_info_iter = &mut accounts.iter();
// Get accounts
let note_creator = next_account_info(account_info_iter)?;
let note_pda_account = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
```
## Lab
This overview covered a lot of new concepts. Let’s practice them together by
continuing to work on the Movie Review program from the last lesson. No worries
if you’re just jumping into this lesson without having done the previous
lesson - it should be possible to follow along either way. We'll be using the
[Orbition Native Chain Playground](https://beta.solpg.io) to write, build, and deploy our code.
As a refresher, we are building a Orbition Native Chain program which lets users review movies.
Last lesson, we deserialized the instruction data passed in by the user but we
have not yet stored this data in an account. Let’s now update our program to
create new accounts to store the user’s movie review.
#### 1. Get the starter code
If you didn’t complete the lab from the last lesson or just want to make sure
that you didn’t miss anything, you can reference
[the starter code](https://beta.solpg.io/6295b25b0e6ab1eb92d947f7).
Our program currently includes the `instruction.rs` file we use to deserialize
the `instruction_data` passed into the program entry point. We have also
completed `lib.rs` file to the point where we can print our deserialized
instruction data to the program log using the `msg!` macro.
#### 2. Create struct to represent account data
Let’s begin by creating a new file named `state.rs`.
This file will:
1. Define the struct our program uses to populate the data field of a new
account
2. Add `BorshSerialize` and `BorshDeserialize` traits to this struct
First, let’s bring into scope everything we’ll need from the `borsh` crate.
```rust theme={null}
use borsh::{BorshSerialize, BorshDeserialize};
```
Next, let’s create our `MovieAccountState` struct. This struct will define the
parameters that each new movie review account will store in its data field. Our
`MovieAccountState` struct will require the following parameters:
* `is_initialized` - shows whether or not the account has been initialized
* `rating` - user’s rating of the movie
* `description` - user’s description of the movie
* `title` - title of the movie the user is reviewing
```rust theme={null}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct MovieAccountState {
pub is_initialized: bool,
pub rating: u8,
pub title: String,
pub description: String
}
```
#### 3. Update `lib.rs`
Next, let’s update our `lib.rs` file. First, we’ll bring into scope everything
we will need to complete our Movie Review program. You can read more about the
details each item we are using from
[the `solana_program` crate](https://docs.rs/solana-program/latest/solana_program/).
```rust theme={null}
use solana_program::{
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
msg,
account_info::{next_account_info, AccountInfo},
system_instruction,
program_error::ProgramError,
sysvar::{rent::Rent, Sysvar},
program::{invoke_signed},
borsh::try_from_slice_unchecked,
};
use std::convert::TryInto;
pub mod instruction;
pub mod state;
use instruction::MovieInstruction;
use state::MovieAccountState;
use borsh::BorshSerialize;
```
#### 4. Iterate through `accounts`
Next, let’s continue building out our `add_movie_review` function. Recall that
an array of accounts is passed into the `add_movie_review` function through a
single `accounts` argument. To process our instruction, we will need to iterate
through `accounts` and assign the `AccountInfo` for each account to its own
variable.
```rust theme={null}
// Get Account iterator
let account_info_iter = &mut accounts.iter();
// Get accounts
let initializer = next_account_info(account_info_iter)?;
let pda_account = next_account_info(account_info_iter)?;
let system_program = next_account_info(account_info_iter)?;
```
#### 5. Derive PDA
Next, within our `add_movie_review` function, let’s independently derive the PDA
we expect the user to have passed in. We'll need to provide the bump seed for
the derivation later, so even though `pda_account` should reference the same
account, we still need to call `find_program_address`.
Note that we derive the PDA for each new account using the initializer’s public
key and the movie title as optional seeds. Setting up the PDA this way restricts
each user to only one review for any one movie title. However, it still allows
the same user to review movies with different titles and different users to
review movies with the same title.
```rust theme={null}
// Derive PDA
let (pda, bump_seed) = Pubkey::find_program_address(&[initializer.key.as_ref(), title.as_bytes().as_ref(),], program_id);
```
#### 6. Calculate space and rent
Next, let’s calculate the rent that our new account will need. Recall that rent
is the amount of lamports a user must allocate to an account for storing data on
the Orbition Native Chain network. To calculate rent, we must first calculate the amount of
space our new account requires.
The `MovieAccountState` struct has four fields. We will allocate 1 byte each for
`rating` and `is_initialized`. For both `title` and `description` we will
allocate space equal to 4 bytes plus the length of the string.
```rust theme={null}
// Calculate account size required
let account_len: usize = 1 + 1 + (4 + title.len()) + (4 + description.len());
// Calculate rent required
let rent = Rent::get()?;
let rent_lamports = rent.minimum_balance(account_len);
```
#### 7. Create new account
Once we’ve calculated the rent and verified the PDA, we are ready to create our
new account. To create a new account, we must call the `create_account`
instruction from the system program. We do this with a Cross Program Invocation
(CPI) using the `invoke_signed` function. We use `invoke_signed` because we are
creating the account using a PDA and need the Movie Review program to “sign” the
instruction.
```rust theme={null}
// Create the account
invoke_signed(
&system_instruction::create_account(
initializer.key,
pda_account.key,
rent_lamports,
account_len.try_into().unwrap(),
program_id,
),
&[initializer.clone(), pda_account.clone(), system_program.clone()],
&[&[initializer.key.as_ref(), title.as_bytes().as_ref(), &[bump_seed]]],
)?;
msg!("PDA created: {}", pda);
```
#### 8. Update account data
Now that we’ve created a new account, we are ready to update the data field of
the new account using the format of the `MovieAccountState` struct from our
`state.rs` file. We first deserialize the account data from `pda_account` using
`try_from_slice_unchecked`, then set the values of each field.
```rust theme={null}
msg!("unpacking state account");
let mut account_data = try_from_slice_unchecked::(&pda_account.data.borrow()).unwrap();
msg!("borrowed account data");
account_data.title = title;
account_data.rating = rating;
account_data.description = description;
account_data.is_initialized = true;
```
Lastly, we serialize the updated `account_data` into the data field of our
`pda_account`.
```rust theme={null}
msg!("serializing account");
account_data.serialize(&mut &mut pda_account.data.borrow_mut()[..])?;
msg!("state account serialized");
```
#### 9. Build and deploy
We're ready to build and deploy our program!
You can test your program by submitting a transaction with the right instruction
data. For that, feel free to use
[this script](https://github.com/Unboxed-Software/solana-movie-client) or
[the frontend](https://github.com/Unboxed-Software/solana-movie-frontend) we
built in the
[Deserialize Custom Instruction Data lesson](deserialize-custom-data). In both
cases, make sure you copy and paste the program ID for your program into the
appropriate area of the source code to make sure you're testing the right
program.
If you use the frontend, simply replace the `MOVIE_REVIEW_PROGRAM_ID` in both
the `MovieList.tsx` and `Form.tsx` components with the address of the program
you’ve deployed. Then run the frontend, submit a view, and refresh the browser
to see the review.
If you need more time with this project to feel comfortable with these concepts,
have a look at the
[solution code](https://beta.solpg.io/62b23597f6273245aca4f5b4) before
continuing.
## Challenge
Now it’s your turn to build something independently. Equipped with the concepts
intoduced in this lesson, you now know everything you'll need to recreate the
entirety of the Student Intro program from Module 1.
The Student Intro program is a Orbition Native Chain Program that lets students introduce
themselves. The program takes a user's name and a short message as the
`instruction_data` and creates an account to store the data onchain.
Using what you've learned in this lesson, build out this program. In addition to
taking a name a short message as instruction data, the program should:
1. Create a separate account for each student
2. Store `is_initialized` as a boolean, `name` as a string, and `msg` as a
string in each account
You can test your program by building the
[frontend](https://github.com/Unboxed-Software/solana-student-intros-frontend)
we created in the
[Page, Order, and Filter Program Data lesson](/content/courses/native-onchain-development/paging-ordering-filtering-data-frontend).
Remember to replace the program ID in the frontend code with the one you've
deployed.
Try to do this independently if you can! But if you get stuck, feel free to
reference the [solution code](https://beta.solpg.io/62b11ce4f6273245aca4f5b2).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=8320fc87-2b6d-4b3a-8b1a-54b55afed781)!
# Serialize Custom Instruction Data for Native Program Development
Source: https://docs.orbition.network/courses/native-onchain-development/serialize-instruction-data-frontend
How to deserialize data fetched from Orbition Native Chain accounts.
## Summary
* Native (non-Anchor) Orbition Native Chain development requires manual serialization and
deserialization of data.
* Transactions are made up of an array of instructions, a single transaction can
have any number of instructions in it, each targeting different programs. When
a transaction is submitted, the Orbition Native Chain runtime will process its instructions
in order and atomically, meaning that if any of the instructions fail for any
reason, the entire transaction will fail to be processed.
* Every *instruction* is made up of 3 components: the intended program's ID, an
array of all accounts involved, and a byte buffer of instruction data.
* Every *transaction* contains an array of all accounts it intends to read or
write, one or more instructions, a recent blockhash, and one or more
signatures.
* To pass instruction data from a client, it must be serialized into a byte
buffer. To facilitate this process of serialization, we will be using
[Borsh](https://borsh.io/).
* Transactions can fail to be processed by the blockchain for any number of
reasons, we’ll discuss some of the most common ones here.
## Lesson
### Transactions
This course requires completing
[Introduction to Orbition Native Chain](/developers/courses/intro-to-solana) or equivalent
knowledge. It's also aimed at advanced developers that prefer more control over
the ease of use and safe defaults Anchor provides. If you're new to developing
onchain programs you may prefer
[Anchor](/developers/courses/onchain-development)
In [Introduction to Orbition Native Chain](/developers/courses/intro-to-solana) we learned how
to create transactions with instructions for common Orbition Native Chain programs.
This lessons shows how to create instructions for our own native Orbition Native Chain
programs, which we will develop in a few lessons. Specifically, we're going to
learn about serialization and deserialization, which is requiredfor native
(non-Anchor) program development.
#### Transaction Contents
Every transaction contains:
* An array that includes every account it intends to read or write
* One or more instructions
* A recent blockhash
* One or more signatures
`@orbition-network/web3.js` simplifies this process for you so that all you need to focus
on is adding instructions and signatures. The library builds the array of
accounts based on that information and handles the logic for including a recent
blockhash.
### Instructions
Every instruction contains:
* The program ID (public key) of the intended program
* An array listing every account that will be read from or written to during
execution
* A byte buffer of instruction data
Identifying the program by its public key ensures that the instruction is
carried out by the correct program.
Including an array of every account that will be read from or written to allows
the network to perform several optimizations which allow for high transaction
load and quicker execution.
The byte buffer lets you pass external data to a program.
You can include multiple instructions in a single transaction. The Orbition Native Chain
runtime will process these instructions in order and atomically. In other words,
if every instruction succeeds then the transaction as a whole will be
successful, but if a single instruction fails then the entire transaction will
fail immediately with no side-effects.
The account array is not just an array of the accounts’ public keys. Each object
in the array includes the account’s public key, whether or not it is a signer on
the transaction, and whether or not it is writable. Including whether or not an
account is writable during the execution of an instruction allows the runtime to
facilitate parallel processing of smart contracts. Because you must define which
accounts are read-only and which you will write to, the runtime can determine
which transactions are non-overlapping or read-only and allow them to execute
concurrently. To learn more about Orbition Native Chain’s runtime, check out this
[blog post on Sealevel](https://orbition.network/news/sealevel---parallel-processing-thousands-of-smart-contracts).
#### Instruction Data
The ability to add arbitrary data to an instruction ensures that programs can be
dynamic and flexible enough for broad use cases in the same way that the body of
an HTTP request lets you build dynamic and flexible REST APIs.
Just as the structure of the body of an HTTP request is dependent on the
endpoint you intend to call, the structure of the byte buffer used as
instruction data is entirely dependent on the recipient program. If you’re
building a full-stack dApp on your own, then you’ll need to copy the same
structure that you used when building the program over to the client-side code.
If you’re working with another developer who is handling the program
development, you can coordinate to ensure matching buffer layouts.
Let’s think about a concrete example. Imagine working on a Web3 game and being
responsible for writing client-side code that interacts with a player inventory
program. The program was designed to allow the client to:
* Add inventory based on a player’s game-play results
* Transfer inventory from one player to another
* Equip a player with selected inventory items
This program would have been structured such that each of these is encapsulated
in its own function.
Each program, however, only has one entry point. You would instruct the program
on which of these functions to run through the instruction data.
You would also include in the instruction data any information the function
needs to execute properly, e.g. an inventory item’s ID, a player to transfer
inventory to, etc.
Exactly *how* this data would be structured would depend on how the program was
written, but it’s common to have the first field in instruction data be a number
that the program can map to a function, after which additional fields act as
function arguments.
### Serialization
In addition to knowing what information to include in an instruction data
buffer, you also need to serialize it properly. The most common serializer used
in Orbition Native Chain is [Borsh](https://borsh.io). Per the website:
> Borsh stands for Binary Object Representation Serializer for Hashing. It is
> meant to be used in security-critical projects as it prioritizes consistency,
> safety, speed; and comes with a strict specification.
Borsh maintains a [JS library](https://github.com/near/borsh-js) that handles
serializing common types into a buffer. There are also other packages built on
top of Borsh that try to make this process even easier. We’ll be using the
`@coral-xyz/borsh` library which can be installed using `npm`.
Building off of the previous game inventory example, let’s look at a
hypothetical scenario where we are instructing the program to equip a player
with a given item. Assume the program is designed to accept a buffer that
represents a struct with the following properties:
1. `variant` is an unsigned, 8-bit integer that instructs the program which
instruction, or function, to execute.
2. `playerId` is an unsigned, 16-bit integer that represents the player ID of
the player who is to be equipped with the given item.
3. `itemId` is an unsigned, 256-bit integer that represents the item ID of the
item that will be equipped for the given player.
All of this will be passed as a byte buffer that will be read in order, so
ensuring proper buffer layout order is crucial. You would create the buffer
layout schema or template for the above as follows:
```typescript theme={null}
import * as borsh from "@coral-xyz/borsh";
const equipPlayerSchema = borsh.struct([
borsh.u8("variant"),
borsh.u16("playerId"),
borsh.u256("itemId"),
]);
```
You can then encode data using this schema with the `encode` method. This method
accepts as arguments an object representing the data to be serialized and a
buffer. In the below example, we allocate a new buffer that’s much larger than
needed, then encode the data into that buffer and slice the original buffer down
into a new buffer that’s only as large as needed.
```typescript theme={null}
import * as borsh from "@coral-xyz/borsh";
const equipPlayerSchema = borsh.struct([
borsh.u8("variant"),
borsh.u16("playerId"),
borsh.u256("itemId"),
]);
const buffer = Buffer.alloc(1000);
equipPlayerSchema.encode(
{ variant: 2, playerId: 1435, itemId: 737498 },
buffer,
);
const instructionBuffer = buffer.slice(0, equipPlayerSchema.getSpan(buffer));
```
Once a buffer is properly created and the data serialized, all that’s left is
building the transaction. This is similar to what you’ve done in previous
lessons. The example below assumes that:
* `player`, `playerInfoAccount`, and `PROGRAM_ID` are already defined somewhere
outside the code snippet
* `player` is a user’s public key
* `playerInfoAccount` is the public key of the account where inventory changes
will be written
* `SystemProgram` will be used in the process of executing the instruction.
```typescript theme={null}
import * as borsh from "@coral-xyz/borsh";
import * as web3 from "@orbition-network/web3.js";
const equipPlayerSchema = borsh.struct([
borsh.u8("variant"),
borsh.u16("playerId"),
borsh.u256("itemId"),
]);
const buffer = Buffer.alloc(1000);
equipPlayerSchema.encode(
{ variant: 2, playerId: 1435, itemId: 737498 },
buffer,
);
const instructionBuffer = buffer.slice(0, equipPlayerSchema.getSpan(buffer));
const endpoint = web3.clusterApiUrl("testnet");
const connection = new web3.Connection(endpoint);
const transaction = new web3.Transaction();
const instruction = new web3.TransactionInstruction({
keys: [
{
pubkey: player.publicKey,
isSigner: true,
isWritable: false,
},
{
pubkey: playerInfoAccount,
isSigner: false,
isWritable: true,
},
{
pubkey: web3.SystemProgram.programId,
isSigner: false,
isWritable: false,
},
],
data: instructionBuffer,
programId: PROGRAM_ID,
});
transaction.add(instruction);
web3.sendAndConfirmTransaction(connection, transaction, [player]).then(txid => {
console.log(
`Transaction submitted: https://explorer.orbition.network/tx/${txid}?cluster=testnet`,
);
});
```
## Lab
Let’s practice this together by building a Movie Review app that lets users
submit a movie review and have it stored on Orbition Native Chain’s network. We’ll build this
app a little bit at a time over the next few lessons, adding new functionality
each lesson.
Here's a quick diagram of the program we'll build:
The public key of the Orbition Native Chain program we’ll use for this application is
`CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN`.
#### 1. Download the starter code
Before we get started, go ahead and download the
[starter code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/starter).
The project is a fairly simple Next.js application. It includes the
`WalletContextProvider` we created in the Wallets lesson, a `Card` component for
displaying a movie review, a `MovieList` component that displays reviews in a
list, a `Form` component for submitting a new review, and a `Movie.ts` file that
contains a class definition for a `Movie` object.
Note that for now, the movies displayed on the page when you run `npm run dev`
are mocks. In this lesson, we’ll focus on adding a new review but we won’t be
able to see that review displayed. Next lesson, we’ll focus on deserializing
custom data from onchain accounts.
#### 2. Create the buffer layout
Remember that to properly interact with a Orbition Native Chain program, you need to know how
it expects data to be structured. Our Movie Review program expects instruction
data to contain:
1. `variant` as an unsigned, 8-bit integer representing which instruction should
be executed (in other words which function on the program should be called).
2. `title` as a string representing the title of the movie that you are
reviewing.
3. `rating` as an unsigned, 8-bit integer representing the rating out of 5 that
you are giving to the movie you are reviewing.
4. `description` as a string representing the written portion of the review you
are leaving for the movie.
Let’s configure a `borsh` layout in the `Movie` class. Start by importing
`@coral-xyz/borsh`. Next, create a `borshInstructionSchema` property and set it
to the appropriate `borsh` struct containing the properties listed above.
```typescript theme={null}
import * as borsh from '@coral-xyz/borsh'
export class Movie {
title: string;
rating: number;
description: string;
...
borshInstructionSchema = borsh.struct([
borsh.u8('variant'),
borsh.str('title'),
borsh.u8('rating'),
borsh.str('description'),
])
}
```
Keep in mind that *order matters*. If the order of properties here differs from
how the program is structured, the transaction will fail.
#### 3. Create a method to serialize data
Now that we have the buffer layout set up, let’s create a method in `Movie`
called `serialize()` that will return a `Buffer` with a `Movie` object’s
properties encoded into the appropriate layout.
```typescript theme={null}
import * as borsh from '@coral-xyz/borsh'
export class Movie {
title: string;
rating: number;
description: string;
...
borshInstructionSchema = borsh.struct([
borsh.u8('variant'),
borsh.str('title'),
borsh.u8('rating'),
borsh.str('description'),
])
serialize(): Buffer {
const buffer = Buffer.alloc(1000)
this.borshInstructionSchema.encode({ ...this, variant: 0 }, buffer)
return buffer.slice(0, this.borshInstructionSchema.getSpan(buffer))
}
}
```
The method shown above first creates a large enough buffer for our object, then
encodes `{ ...this, variant: 0 }` into the buffer. Because the `Movie` class
definition contains 3 of the 4 properties required by the buffer layout and uses
the same naming, we can use it directly with the spread operator and just add
the `variant` property. Finally, the method returns a new buffer that leaves off
the unused portion of the original.
#### 4. Send a transaction when the user submits the form
Now that we have the building blocks for the instruction data, we can create and
send the transaction when a user submits the form. Open `Form.tsx` and locate
the `handleTransactionSubmit` function. This gets called by `handleSubmit` each
time a user submits the Movie Review form.
Inside this function, we’ll be creating and sending the transaction that
contains the data submitted through the form.
Start by importing `@orbition-network/web3.js` and importing `useConnection` and
`useWallet` from `@orbition-network/wallet-adapter-react`.
```tsx theme={null}
import { FC } from "react";
import { Movie } from "../models/Movie";
import { useState } from "react";
import {
Box,
Button,
FormControl,
FormLabel,
Input,
NumberDecrementStepper,
NumberIncrementStepper,
NumberInput,
NumberInputField,
NumberInputStepper,
Textarea,
} from "@chakra-ui/react";
import * as web3 from "@orbition-network/web3.js";
import { useConnection, useWallet } from "@orbition-network/wallet-adapter-react";
```
Next, before the `handleSubmit` function, call `useConnection()` to get a
`connection` object and call `useWallet()` to get `publicKey` and
`sendTransaction`.
```tsx theme={null}
import { FC } from 'react'
import { Movie } from '../models/Movie'
import { useState } from 'react'
import { Box, Button, FormControl, FormLabel, Input, NumberDecrementStepper, NumberIncrementStepper, NumberInput, NumberInputField, NumberInputStepper, Textarea } from '@chakra-ui/react'
import * as web3 from '@orbition-network/web3.js'
import { useConnection, useWallet } from '@orbition-network/wallet-adapter-react'
const MOVIE_REVIEW_PROGRAM_ID = 'CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN'
export const Form: FC = () => {
const [title, setTitle] = useState('')
const [rating, setRating] = useState(0)
const [message, setMessage] = useState('')
const { connection } = useConnection();
const { publicKey, sendTransaction } = useWallet();
const handleSubmit = (event: any) => {
event.preventDefault()
const movie = new Movie(title, rating, description)
handleTransactionSubmit(movie)
}
...
}
```
Before we implement `handleTransactionSubmit`, let’s talk about what needs to be
done. We need to:
1. Check that `publicKey` exists to ensure that the user has connected their
wallet.
2. Call `serialize()` on `movie` to get a buffer representing the instruction
data.
3. Create a new `Transaction` object.
4. Get all of the accounts that the transaction will read or write.
5. Create a new `Instruction` object that includes all of these accounts in the
`keys` argument, includes the buffer in the `data` argument, and includes the
program’s public key in the `programId` argument.
6. Add the instruction from the last step to the transaction.
7. Call `sendTransaction`, passing in the assembled transaction.
That’s quite a lot to process! But don’t worry, it gets easier the more you do
it. Let’s start with the first 3 steps from above:
```typescript theme={null}
const handleTransactionSubmit = async (movie: Movie) => {
if (!publicKey) {
alert("Please connect your wallet!");
return;
}
const buffer = movie.serialize();
const transaction = new web3.Transaction();
};
```
The next step is to get all of the accounts that the transaction will read or
write. In past lessons, the account where data will be stored has been given to
you. This time, the account’s address is more dynamic, so it needs to be
computed. We’ll cover this in-depth in the next lesson, but for now, you can use
the following, where `pda` is the address to the account where data will be
stored:
```typescript theme={null}
const [pda] = await web3.PublicKey.findProgramAddress(
[publicKey.toBuffer(), Buffer.from(movie.title)],
new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
);
```
In addition to this account, the program will also need to read from
`SystemProgram`, so our array needs to include `web3.SystemProgram.programId` as
well.
With that, we can finish the remaining steps:
```typescript theme={null}
const handleTransactionSubmit = async (movie: Movie) => {
if (!publicKey) {
alert("Please connect your wallet!");
return;
}
const buffer = movie.serialize();
const transaction = new web3.Transaction();
const [pda] = await web3.PublicKey.findProgramAddress(
[publicKey.toBuffer(), new TextEncoder().encode(movie.title)],
new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
);
const instruction = new web3.TransactionInstruction({
keys: [
{
pubkey: publicKey,
isSigner: true,
isWritable: false,
},
{
pubkey: pda,
isSigner: false,
isWritable: true,
},
{
pubkey: web3.SystemProgram.programId,
isSigner: false,
isWritable: false,
},
],
data: buffer,
programId: new web3.PublicKey(MOVIE_REVIEW_PROGRAM_ID),
});
transaction.add(instruction);
try {
let txid = await sendTransaction(transaction, connection);
console.log(
`Transaction submitted: https://explorer.orbition.network/tx/${txid}?cluster=testnet`,
);
} catch (e) {
alert(JSON.stringify(e));
}
};
```
And that’s it! You should now be able to use the form on the site to submit a
movie review. While you won’t see the UI update to reflect the new review, you
can look at the transaction’s program logs on Orbition Native Chain Explorer to see that it was
successful.
If you need a bit more time with this project to feel comfortable, have a look
at the complete
[solution code](https://github.com/Unboxed-Software/solana-movie-frontend/tree/solution-serialize-instruction-data).
## Challenge
Now it’s your turn to build something independently. Create an application that
lets students of this course introduce themselves! The Orbition Native Chain program that
supports this is at `HdE95RSVsdb315jfJtaykXhXY478h53X6okDupVfY9yf`.
1. You can build this from scratch or you can
[download the starter code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/starter).
2. Create the instruction buffer layout in `StudentIntro.ts`. The program
expects instruction data to contain:
1. `variant` as an unsigned, 8-bit integer representing the instruction to
run (should be 0).
2. `name` as a string representing the student's name.
3. `message` as a string representing the message the student is sharing
about their Orbition Native Chain journey.
3. Create a method in `StudentIntro.ts` that will use the buffer layout to
serialize a `StudentIntro` object.
4. In the `Form` component, implement the `handleTransactionSubmit` function so
that it serializes a `StudentIntro`, builds the appropriate transaction and
transaction instructions, and submits the transaction to the user's wallet.
5. You should now be able to submit introductions and have the information
stored onchain! Be sure to log the transaction ID and look at it in Orbition Native Chain
Explorer to verify that it worked.
If you get stumped, you can
[check out the solution code](https://github.com/Unboxed-Software/solana-student-intros-frontend/tree/solution-serialize-instruction-data).
Feel free to get creative with these challenges and take them even further. The
instructions aren't here to hold you back!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=6cb40094-3def-4b66-8a72-dd5f00298f61)!
# Anchor CPIs and Errors
Source: https://docs.orbition.network/courses/onchain-development/anchor-cpi
Invoke other Orbition Native Chain programs from your Anchor app.
## Summary
* Anchor provides a simplified way to create CPIs using a **`CpiContext`**
* Anchor's **`cpi`** feature generates CPI helper functions for invoking
instructions on existing Anchor programs
* If you do not have access to CPI helper functions, you can still use `invoke`
and `invoke_signed` directly
* The **`error_code`** attribute macro is used to create custom Anchor Errors
## Lesson
Anchor makes invoking other Orbition Native Chain programs easier, especially if the program
you're invoking is also an Anchor program whose crate you can access.
In this lesson, you'll learn how to construct an Anchor CPI. You'll also learn
how to throw custom errors from an Anchor program so that you can start to write
more sophisticated Anchor programs.
### Cross Program Invocations (CPIs) with Anchor
CPIs allow programs to invoke instructions on other programs using the `invoke`
or `invoke_signed` functions. This allows new programs to build on top of
existing programs (we call that composability).
While making CPIs directly using `invoke` or `invoke_signed` is still an option,
Anchor also provides a simplified way to make CPIs by using a `CpiContext`.
In this lesson, you'll use the `anchor_spl` crate to make CPIs to the SPL Token
Program. You can
[explore what's available in the `anchor_spl` crate](https://docs.rs/anchor-spl/latest/anchor_spl/#).
#### `CpiContext`
The first step in making a CPI is to create an instance of `CpiContext`.
`CpiContext` is very similar to `Context`, the first argument type required by
Anchor instruction functions. They are both declared in the same module and
share similar functionality.
The `CpiContext` type specifies non-argument inputs for cross program
invocations:
* `accounts` - the list of accounts required for the instruction being invoked
* `remaining_accounts` - any remaining accounts
* `program` - the program ID of the program being invoked
* `signer_seeds` - if a PDA is signing, include the seeds required to derive the
PDA
```rust theme={null}
pub struct CpiContext<'a, 'b, 'c, 'info, T>
where
T: ToAccountMetas + ToAccountInfos<'info>,
{
pub accounts: T,
pub remaining_accounts: Vec>,
pub program: AccountInfo<'info>,
pub signer_seeds: &'a [&'b [&'c [u8]]],
}
```
You use `CpiContext::new` to construct a new instance when passing along the
original transaction signature.
```rust theme={null}
CpiContext::new(cpi_program, cpi_accounts)
```
```rust theme={null}
pub fn new(
program: AccountInfo<'info>,
accounts: T
) -> Self {
Self {
accounts,
program,
remaining_accounts: Vec::new(),
signer_seeds: &[],
}
}
```
You use `CpiContext::new_with_signer` to construct a new instance when signing
on behalf of a PDA for the CPI.
```rust theme={null}
CpiContext::new_with_signer(cpi_program, cpi_accounts, seeds)
```
```rust theme={null}
pub fn new_with_signer(
program: AccountInfo<'info>,
accounts: T,
signer_seeds: &'a [&'b [&'c [u8]]],
) -> Self {
Self {
accounts,
program,
signer_seeds,
remaining_accounts: Vec::new(),
}
}
```
#### CPI accounts
One of the main things about `CpiContext` that simplifies cross-program
invocations is that the `accounts` argument is a generic type that lets you pass
in any object that adopts the `ToAccountMetas` and `ToAccountInfos<'info>`
traits.
These traits are added by the `#[derive(Accounts)]` attribute macro that you've
used before when creating structs to represent instruction accounts. That means
you can use similar structs with `CpiContext`.
This helps with code organization and type safety.
#### Invoke an instruction on another Anchor program
When the program you're calling is an Anchor program with a published crate,
Anchor can generate instruction builders and CPI helper functions for you.
Simply declare your program's dependency on the program you're calling in your
program's `Cargo.toml` file as follows:
```
[dependencies]
callee = { path = "../callee", features = ["cpi"]}
```
By adding `features = ["cpi"]`, you enable the `cpi` feature and your program
gains access to the `callee::cpi` module.
The `cpi` module exposes `callee`'s instructions as a Rust function that takes
as arguments a `CpiContext` and any additional instruction data. These functions
use the same format as the instruction functions in your Anchor programs, only
with `CpiContext` instead of `Context`. The `cpi` module also exposes the
accounts structs required for calling the instructions.
For example, if `callee` has the instruction `do_something` that requires the
accounts defined in the `DoSomething` struct, you could invoke `do_something` as
follows:
```rust theme={null}
use anchor_lang::prelude::*;
use callee;
...
#[program]
pub mod lootbox_program {
use super::*;
pub fn call_another_program(ctx: Context, params: InitUserParams) -> Result<()> {
callee::cpi::do_something(
CpiContext::new(
ctx.accounts.callee.to_account_info(),
callee::DoSomething {
user: ctx.accounts.user.to_account_info()
}
)
)
Ok(())
}
}
...
```
#### Invoke an instruction on a non-Anchor program
When the program you're calling is *not* an Anchor program, there are two
possible options:
1. It's possible that the program maintainers have published a crate with their
own helper functions for calling into their program. For example, the
`anchor_spl` crate provides helper functions that are virtually identical
from a call-site perspective to what you would get with the `cpi` module of
an Anchor program. E.g. you can mint using the
[`mint_to` helper function](https://docs.rs/anchor-spl/latest/src/anchor_spl/token.rs.html#36-58)
and use the
[`MintTo` accounts struct](https://docs.rs/anchor-spl/latest/anchor_spl/token/struct.MintTo.html).
```rust theme={null}
token::mint_to(
CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::MintTo {
mint: ctx.accounts.mint_account.to_account_info(),
to: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.mint_authority.to_account_info(),
},
&[&[
"mint".as_bytes(),
&[*ctx.bumps.get("mint_authority").unwrap()],
]]
),
amount,
)?;
```
2. If there is no helper module for the program whose instruction(s) you need to
invoke, you can fall back to using `invoke` and `invoke_signed`. In fact, the
source code of the `mint_to` helper function referenced above shows an
example using `invoke_signed` when given a `CpiContext`. You can follow a
similar pattern if you decide to use an accounts struct and `CpiContext` to
organize and prepare your CPI.
```rust theme={null}
pub fn mint_to<'a, 'b, 'c, 'info>(
ctx: CpiContext<'a, 'b, 'c, 'info, MintTo<'info>>,
amount: u64,
) -> Result<()> {
let ix = spl_token::instruction::mint_to(
&spl_token::ID,
ctx.accounts.mint.key,
ctx.accounts.to.key,
ctx.accounts.authority.key,
&[],
amount,
)?;
solana_program::program::invoke_signed(
&ix,
&[
ctx.accounts.to.clone(),
ctx.accounts.mint.clone(),
ctx.accounts.authority.clone(),
],
ctx.signer_seeds,
)
.map_err(Into::into)
}
```
### Throw errors in Anchor
We're deep enough into Anchor at this point that it's important to know how to
create custom errors.
Ultimately, all programs return the same error
type: [`ProgramError`](https://docs.rs/solana-program/latest/solana_program/program_error/enum.ProgramError.html).
However, when writing a program using Anchor you can use `AnchorError` as an
abstraction on top of `ProgramError`. This abstraction provides additional
information when a program fails, including:
* The error name and number
* Location in the code where the error was thrown
* The account that violated a constraint
```rust theme={null}
pub struct AnchorError {
pub error_name: String,
pub error_code_number: u32,
pub error_msg: String,
pub error_origin: Option,
pub compared_values: Option,
}
```
Anchor Errors can be divided into:
* Anchor Internal Errors that the framework returns from inside its own code
* Custom errors that you the developer can create
You can add errors unique to your program by using the `error_code` attribute.
Simply add this attribute to a custom `enum` type. You can then use the variants
of the `enum` as errors in your program. Additionally, you can add an error
message to each variant using the `msg` attribute. Clients can then display this
error message if the error occurs.
```rust theme={null}
#[error_code]
pub enum MyError {
#[msg("MyAccount may only hold data below 100")]
DataTooLarge
}
```
To return a custom error you can use
the [err](https://docs.rs/anchor-lang/latest/anchor_lang/macro.err.html) or
the [error](https://docs.rs/anchor-lang/latest/anchor_lang/prelude/macro.error.html)
macro from an instruction function. These add file and line information to the
error that is then logged by Anchor to help you with debugging.
```rust theme={null}
#[program]
mod hello_anchor {
use super::*;
pub fn set_data(ctx: Context, data: MyAccount) -> Result<()> {
if data.data >= 100 {
return err!(MyError::DataTooLarge);
}
ctx.accounts.my_account.set_inner(data);
Ok(())
}
}
#[error_code]
pub enum MyError {
#[msg("MyAccount may only hold data below 100")]
DataTooLarge
}
```
Alternatively, you can use
the [require](https://docs.rs/anchor-lang/latest/anchor_lang/macro.require.html) macro
to simplify returning errors. The code above can be refactored to the following:
```rust theme={null}
#[program]
mod hello_anchor {
use super::*;
pub fn set_data(ctx: Context, data: MyAccount) -> Result<()> {
require!(data.data < 100, MyError::DataTooLarge);
ctx.accounts.my_account.set_inner(data);
Ok(())
}
}
#[error_code]
pub enum MyError {
#[msg("MyAccount may only hold data below 100")]
DataTooLarge
}
```
## Lab
Let’s practice the concepts we’ve gone over in this lesson by building on top of
the Movie Review program from previous lessons.
In this lab we’ll update the program to mint tokens to users when they submit a
new movie review.
#### 1. Starter
To get started, we will be using the final state of the Anchor Movie Review
program from the previous lesson. So, if you just completed that lesson then
you’re all set and ready to go. If you are just jumping in here, no worries, you
can [download the starter code](https://github.com/Unboxed-Software/anchor-movie-review-program/tree/solution-pdas).
We'll be using the `solution-pdas` branch as our starting point.
#### 2. Add dependencies to `Cargo.toml`
Before we get started we need enable the `init-if-needed` feature and add the
`anchor-spl` crate to the dependencies in `Cargo.toml`. If you need to brush up
on the `init-if-needed` feature take a look at the
[Anchor PDAs and Accounts lesson](anchor-pdas).
```rust theme={null}
[dependencies]
anchor-lang = { version = "0.25.0", features = ["init-if-needed"] }
anchor-spl = "0.25.0"
```
#### 3. Initialize reward token
Next, navigate to `lib.rs` and create an instruction to initialize a new token
mint. This will be the token that is minted each time a user leaves a review.
Note that we don't need to include any custom instruction logic since the
initialization can be handled entirely through Anchor constraints.
```rust theme={null}
pub fn initialize_token_mint(_ctx: Context) -> Result<()> {
msg!("Token mint initialized");
Ok(())
}
```
Now, implement the `InitializeMint` context type and list the accounts and
constraints the instruction requires. Here we initialize a new `Mint` account
using a PDA with the string "mint" as a seed. Note that we can use the same PDA
for both the address of the `Mint` account and the mint authority. Using a PDA
as the mint authority enables our program to sign for the minting of the tokens.
To initialize the `Mint` account, we'll need to include the `token_program`,
`rent`, and `system_program` in the list of accounts.
```rust theme={null}
#[derive(Accounts)]
pub struct InitializeMint<'info> {
#[account(
init,
seeds = ["mint".as_bytes()],
bump,
payer = user,
mint::decimals = 6,
mint::authority = mint,
)]
pub mint: Account<'info, Mint>,
#[account(mut)]
pub user: Signer<'info>,
pub token_program: Program<'info, Token>,
pub rent: Sysvar<'info, Rent>,
pub system_program: Program<'info, System>
}
```
There may be some constraints above that you haven't seen yet. Adding
`mint::decimals` and `mint::authority` along with `init` ensures that the
account is initialized as a new token mint with the appropriate decimals and
mint authority set.
#### 4. Anchor Error
Next, let’s create an Anchor Error that we’ll use when validating the `rating`
passed to either the `add_movie_review` or `update_movie_review` instruction.
```rust theme={null}
#[error_code]
enum MovieReviewError {
#[msg("Rating must be between 1 and 5")]
InvalidRating
}
```
#### 5. Update `add_movie_review` instruction
Now that we've done some setup, let’s update the `add_movie_review` instruction
and `AddMovieReview` context type to mint tokens to the reviewer.
Next, update the `AddMovieReview` context type to add the following accounts:
* `token_program` - we'll be using the Token Program to mint tokens
* `mint` - the mint account for the tokens that we'll mint to users when they
add a movie review
* `token_account` - the associated token account for the afforementioned `mint`
and reviewer
* `associated_token_program` - required because we'll be using the
`associated_token` constraint on the `token_account`
* `rent` - required because we are using the `init-if-needed` constraint on the
`token_account`
```rust theme={null}
#[derive(Accounts)]
#[instruction(title: String, description: String)]
pub struct AddMovieReview<'info> {
#[account(
init,
seeds=[title.as_bytes(), initializer.key().as_ref()],
bump,
payer = initializer,
space = 8 + 32 + 1 + 4 + title.len() + 4 + description.len()
)]
pub movie_review: Account<'info, MovieAccountState>,
#[account(mut)]
pub initializer: Signer<'info>,
pub system_program: Program<'info, System>,
// ADDED ACCOUNTS BELOW
pub token_program: Program<'info, Token>,
#[account(
seeds = ["mint".as_bytes()],
bump,
mut
)]
pub mint: Account<'info, Mint>,
#[account(
init_if_needed,
payer = initializer,
associated_token::mint = mint,
associated_token::authority = initializer
)]
pub token_account: Account<'info, TokenAccount>,
pub associated_token_program: Program<'info, AssociatedToken>,
pub rent: Sysvar<'info, Rent>
}
```
Again, some of the above constraints may be unfamiliar to you. The
`associated_token::mint` and `associated_token::authority` constraints along
with the `init_if_needed` constraint ensures that if the account has not already
been initialized, it will be initialized as an associated token account for the
specified mint and authority.
Next, let’s update the `add_movie_review` instruction to do the following:
* Check that `rating` is valid. If it is not a valid rating, return the
`InvalidRating` error.
* Make a CPI to the token program’s `mint_to` instruction using the mint
authority PDA as a signer. Note that we'll mint 10 tokens to the user but need
to adjust for the mint decimals by making it `10*10^6`.
Fortunately, we can use the `anchor_spl` crate to access helper functions and
types like `mint_to` and `MintTo` for constructing our CPI to the Token Program.
`mint_to` takes a `CpiContext` and integer as arguments, where the integer
represents the number of tokens to mint. `MintTo` can be used for the list of
accounts that the mint instruction needs.
Update your `use` statements to include:
```rust theme={null}
use anchor_spl::token::{mint_to, MintTo, Mint, TokenAccount, Token};
use anchor_spl::associated_token::AssociatedToken;
```
Next, update the `add_movie_review` function to:
```rust theme={null}
pub fn add_movie_review(ctx: Context, title: String, description: String, rating: u8) -> Result<()> {
msg!("Movie review account created");
msg!("Title: {}", title);
msg!("Description: {}", description);
msg!("Rating: {}", rating);
require!(rating >= 1 && rating <= 5, MovieReviewError::InvalidRating);
let movie_review = &mut ctx.accounts.movie_review;
movie_review.reviewer = ctx.accounts.initializer.key();
movie_review.title = title;
movie_review.description = description;
movie_review.rating = rating;
mint_to(
CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
MintTo {
authority: ctx.accounts.mint.to_account_info(),
to: ctx.accounts.token_account.to_account_info(),
mint: ctx.accounts.mint.to_account_info()
},
&[&[
"mint".as_bytes(),
&[ctx.bumps.mint]
]]
),
10*10^6
)?;
msg!("Minted tokens");
Ok(())
}
```
#### 6. Update `update_movie_review` instruction
Here we are only adding the check that `rating` is valid.
```rust theme={null}
pub fn update_movie_review(ctx: Context, title: String, description: String, rating: u8) -> Result<()> {
msg!("Movie review account space reallocated");
msg!("Title: {}", title);
msg!("Description: {}", description);
msg!("Rating: {}", rating);
require!(rating >= 1 && rating <= 5, MovieReviewError::InvalidRating);
let movie_review = &mut ctx.accounts.movie_review;
movie_review.description = description;
movie_review.rating = rating;
Ok(())
}
```
#### 7. Test
Those are all of the changes we need to make to the program! Now, let’s update
our tests.
Start by making sure your imports and `describe` function look like this:
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor"
import { Program } from "@coral-xyz/anchor"
import { expect } from "chai"
import { getAssociatedTokenAddress, getAccount } from "@orbition-network/spl-token"
import { AnchorMovieReviewProgram } from "../target/types/anchor_movie_review_program"
describe("anchor-movie-review-program", () => {
// Configure the client to use the local cluster.
const provider = anchor.AnchorProvider.env()
anchor.setProvider(provider)
const program = anchor.workspace
.AnchorMovieReviewProgram as Program
const movie = {
title: "Just a test movie",
description: "Wow what a good movie it was real great",
rating: 5,
}
const [movie_pda] = anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from(movie.title), provider.wallet.publicKey.toBuffer()],
program.programId
)
const [mint] = anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from("mint")],
program.programId
)
...
}
```
You can run `npm install @orbition-network/spl-token --save-dev` if you don't have it
installed.
With that done, add a test for the `initializeTokenMint` instruction:
```typescript theme={null}
it("Initializes the reward token", async () => {
const tx = await program.methods.initializeTokenMint().rpc();
});
```
Notice that we didn't have to add `.accounts` because they call be inferred,
including the `mint` account (assuming you have seed inference enabled).
Next, update the test for the `addMovieReview` instruction. The primary
additions are:
1. To get the associated token address that needs to be passed into the
instruction as an account that cannot be inferred
2. Check at the end of the test that the associated token account has 10 tokens
```typescript theme={null}
it("Movie review is added`", async () => {
const tokenAccount = await getAssociatedTokenAddress(
mint,
provider.wallet.publicKey,
);
const tx = await program.methods
.addMovieReview(movie.title, movie.description, movie.rating)
.accounts({
tokenAccount: tokenAccount,
})
.rpc();
const account = await program.account.movieAccountState.fetch(movie_pda);
expect(account.title).to.equal(movie.title);
expect(account.rating).to.equal(movie.rating);
expect(account.description).to.equal(movie.description);
expect(account.reviewer.toBase58()).to.equal(
provider.wallet.publicKey.toBase58(),
);
const userAta = await getAccount(provider.connection, tokenAccount);
expect(Number(userAta.amount)).to.equal((10 * 10) ^ 6);
});
```
After that, neither the test for `updateMovieReview` nor the test for
`deleteMovieReview` need any changes.
At this point, run `anchor test` and you should see the following output
```console theme={null}
anchor-movie-review-program
✔ Initializes the reward token (458ms)
✔ Movie review is added (410ms)
✔ Movie review is updated (402ms)
✔ Deletes a movie review (405ms)
5 passing (2s)
```
If you need more time with the concepts from this lesson or got stuck along the
way, feel free to take a look at the
[solution code](https://github.com/Unboxed-Software/anchor-movie-review-program/tree/solution-add-tokens).
Note that the solution to this lab is on the `solution-add-tokens` branch.
## Challenge
To apply what you've learned about CPIs in this lesson, think about how you
could incorporate them into the Student Intro program. You could do something
similar to what we did in the lab here and add some functionality to mint tokens
to users when they introduce themselves.
Try to do this independently if you can! But if you get stuck, feel free to
reference
this [solution code](https://github.com/Unboxed-Software/anchor-student-intro-program/tree/cpi-challenge).
Note that your code may look slightly different than the solution code depending
on your implementation.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=21375c76-b6f1-4fb6-8cc1-9ef151bc5b0a)!
# Anchor PDAs and Accounts
Source: https://docs.orbition.network/courses/onchain-development/anchor-pdas
Store arbitrary data on Orbition Native Chain, using PDAs, an inbuilt key-value store.
## Summary
* The `seeds` and `bump` constraints are used to initialize and validate PDA
accounts in Anchor
* The `init_if_needed` constraint is used to conditionally initialize a new
account
* The `realloc` constraint is used to reallocate space on an existing account
* The `close` constraint is used to close an account and refund its rent
## Lesson
In this lesson you'll learn how to work with PDAs, reallocate accounts, and
close accounts in Anchor.
Recall that Anchor programs separate instruction logic from account validation.
Account validation primarily happens within structs that represent the list of
accounts needed for a given instruction. Each field of the struct represents a
different account, and you can customize the validation performed on the account
using the `#[account(...)]` attribute macro.
In addition to using constraints for account validation, some constraints can
handle repeatable tasks that would otherwise require a lot of boilerplate inside
our instruction logic. This lesson will introduce the `seeds`, `bump`,
`realloc`, and `close` constraints to help you initialize and validate PDAs,
reallocate accounts, and close accounts.
### PDAs with Anchor
Recall that
[PDAs](https://github.com/Unboxed-Software/solana-course/blob/main/content/pda)
are derived using a list of optional seeds, a bump seed, and a program ID.
Anchor provides a convenient way to validate a PDA with the `seeds` and `bump`
constraints.
```rust theme={null}
#[derive(Accounts)]
struct ExampleAccounts {
#[account(
seeds = [b"example_seed"],
bump
)]
pub pda_account: Account<'info, AccountType>,
}
```
During account validation, Anchor will derive a PDA using the seeds specified in
the `seeds` constraint and verify that the account passed into the instruction
matches the PDA found using the specified `seeds`.
When the `bump` constraint is included without specifying a specific bump,
Anchor will default to using the canonical bump (the first bump that results in
a valid PDA). In most cases you should use the canonical bump.
You can access other fields from within the struct from constraints, so you can
specify seeds that are dependent on other accounts like the signer's public key.
You can also reference the deserialized instruction data if you add the
`#[instruction(...)]` attribute macro to the struct.
For example, the following example shows a list of accounts that include
`pda_account` and `user`. The `pda_account` is constrained such that the seeds
must be the string "example\_seed," the public key of `user`, and the string
passed into the instruction as `instruction_data`.
```rust theme={null}
#[derive(Accounts)]
#[instruction(instruction_data: String)]
pub struct Example<'info> {
#[account(
seeds = [b"example_seed", user.key().as_ref(), instruction_data.as_ref()],
bump
)]
pub pda_account: Account<'info, AccountType>,
#[account(mut)]
pub user: Signer<'info>
}
```
If the `pda_account` address provided by the client doesn't match the PDA
derived using the specified seeds and the canonical bump, then the account
validation will fail.
#### Use PDAs with the `init` constraint
You can combine the `seeds` and `bump` constraints with the `init` constraint to
initialize an account using a PDA.
Recall that the `init` constraint must be used in combination with the `payer`
and `space` constraints to specify the account that will pay for account
initialization and the space to allocate on the new account. Additionally, you
must include `system_program` as one of the fields of the account validation
struct.
```rust theme={null}
#[derive(Accounts)]
pub struct InitializePda<'info> {
#[account(
init,
seeds = [b"example_seed", user.key().as_ref()],
bump,
payer = user,
space = 8 + 8
)]
pub pda_account: Account<'info, AccountType>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[account]
pub struct AccountType {
pub data: u64,
}
```
When using `init` for non-PDA accounts, Anchor defaults to setting the owner of
the initialized account to be the program currently executing the instruction.
However, when using `init` in combination with `seeds` and `bump`, the owner
*must* be the executing program. This is because initializing an account for the
PDA requires a signature that only the executing program can provide. In other
words, the signature verification for the initialization of the PDA account
would fail if the program ID used to derive the PDA did not match the program ID
of the executing program.
When determining the value of `space` for an account initialized and owned by
the executing Anchor program, remember that the first 8 bytes are reserved for
the account discriminator. This is an 8-byte value that Anchor calculates and
uses to identify the program account types. You can use this
[reference](https://www.anchor-lang.com/docs/space) to calculate how much space
you should allocate for an account.
#### Seed inference
The account list for an instruction can get really long for some programs. To
simplify the client-side experience when invoking an Anchor program instruction,
we can turn on seed inference.
Seed inference adds information about PDA seeds to the IDL so that Anchor can
infer PDA seeds from existing call-site information. In the previous example,
the seeds are `b"example_seed"` and `user.key()`. The first is static and
therefore known, and the second is known because `user` is the transaction
signer.
If you use seed inference when building your program, then as long as you're
calling the program using Anchor, you don't need to explicitly derive and pass
in the PDA. Instead, the Anchor library will do it for you.
You can turn on seed inference in the `Anchor.toml` file with `seeds = true`
under `[features]`.
```
[features]
seeds = true
```
#### Use the `#[instruction(...)]` attribute macro
Let's briefly look at the `#[instruction(...)]` attribute macro before moving
on. When using `#[instruction(...)]`, the instruction data you provide in the
list of arguments must match and be in the same order as the instruction
arguments. You can omit unused arguments at the end of the list, but you must
include all arguments up until the last one you will be using.
For example, imagine an instruction has arguments `input_one`, `input_two`, and
`input_three`. If your account constraints need to reference `input_one` and
`input_three`, you need to list all three arguments in the `#[instruction(...)]`
attribute macro.
However, if your constraints only reference `input_one` and `input_two`, you can
omit `input_three`.
```rust theme={null}
pub fn example_instruction(
ctx: Context,
input_one: String,
input_two: String,
input_three: String,
) -> Result<()> {
...
Ok(())
}
#[derive(Accounts)]
#[instruction(input_one:String, input_two:String)]
pub struct Example<'info> {
...
}
```
Additionally, you will get an error if you list the inputs in the incorrect
order:
```rust theme={null}
#[derive(Accounts)]
#[instruction(input_two:String, input_one:String)]
pub struct Example<'info> {
...
}
```
### Init-if-needed
Anchor provides an `init_if_needed` constraint that can be used to initialize an
account if the account has not already been initialized.
This feature is gated behind a feature flag to make sure you are intentional
about using it. For security reasons, it's smart to avoid having one instruction
branch into multiple logic paths. And as the name suggests, `init_if_needed`
executes one of two possible code paths depending on the state of the account in
question.
When using `init_if_needed`, you need to make sure to properly protect your
program against re-initialization attacks. You need to include checks in your
code that check that the initialized account cannot be reset to its initial
settings after the first time it was initialized.
To use `init_if_needed`, you must first enable the feature in `Cargo.toml`.
```rust theme={null}
[dependencies]
anchor-lang = { version = "0.25.0", features = ["init-if-needed"] }
```
Once you’ve enabled the feature, you can include the constraint in the
`#[account(…)]` attribute macro. The example below demonstrates using the
`init_if_needed` constraint to initialize a new associated token account if one
does not already exist.
```rust theme={null}
#[program]
mod example {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(
init_if_needed,
payer = payer,
associated_token::mint = mint,
associated_token::authority = payer
)]
pub token_account: Account<'info, TokenAccount>,
pub mint: Account<'info, Mint>,
#[account(mut)]
pub payer: Signer<'info>,
pub system_program: Program<'info, System>,
pub token_program: Program<'info, Token>,
pub associated_token_program: Program<'info, AssociatedToken>,
pub rent: Sysvar<'info, Rent>,
}
```
When the `initialize` instruction is invoked in the previous example, Anchor
will check if the `token_account` exists and initialize it if it does not. If it
already exists, then the instruction will continue without initializing the
account. Just as with the `init` constraint, you can use `init_if_needed` in
conjunction with `seeds` and `bump` if the account is a PDA.
### Realloc
The `realloc` constraint provides a simple way to reallocate space for existing
accounts.
The `realloc` constraint must be used in combination with the following
constraints:
* `mut` - the account must be set as mutable
* `realloc::payer` - the account to subtract or add lamports to depending on
whether the reallocation is decreasing or increasing account space
* `realloc::zero` - boolean to specify if new memory should be zero initialized
As with `init`, you must include `system_program` as one of the accounts in the
account validation struct when using `realloc`.
Below is an example of reallocating space for an account that stores a `data`
field of type `String`.
```rust theme={null}
#[derive(Accounts)]
#[instruction(instruction_data: String)]
pub struct ReallocExample<'info> {
#[account(
mut,
seeds = [b"example_seed", user.key().as_ref()],
bump,
realloc = 8 + 4 + instruction_data.len(),
realloc::payer = user,
realloc::zero = false,
)]
pub pda_account: Account<'info, AccountType>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[account]
pub struct AccountType {
pub data: String,
}
```
Notice that `realloc` is set to `8 + 4 + instruction_data.len()`. This breaks
down as follows:
* `8` is for the account discriminator
* `4` is for the 4 bytes of space that BORSH uses to store the length of the
string
* `instruction_data.len()` is the length of the string itself
If the change in account data length is additive, lamports will be transferred
from the `realloc::payer` to the account to maintain rent exemption. Likewise,
if the change is subtractive, lamports will be transferred from the account back
to the `realloc::payer`.
The `realloc::zero` constraint is required to determine whether the new memory
should be zero initialized after reallocation. This constraint should be set to
true in cases where you expect the memory of an account to shrink and expand
multiple times. That way you zero out space that would otherwise show as stale
data.
### Close
The `close` constraint provides a simple and secure way to close an existing
account.
The `close` constraint marks the account as closed at the end of the
instruction’s execution by setting its discriminator to
the `CLOSED_ACCOUNT_DISCRIMINATOR` and sends its lamports to a specified
account. Setting the discriminator to a special variant makes account revival
attacks (where a subsequent instruction adds the rent exemption lamports again)
impossible. If someone tries to reinitialize the account, the reinitialization
will fail the discriminator check and be considered invalid by the program.
The example below uses the `close` constraint to close the `data_account` and
sends the lamports allocated for rent to the `receiver` account.
```rust theme={null}
pub fn close(ctx: Context) -> Result<()> {
Ok(())
}
#[derive(Accounts)]
pub struct Close<'info> {
#[account(mut, close = receiver)]
pub data_account: Account<'info, AccountType>,
#[account(mut)]
pub receiver: Signer<'info>
}
```
## Lab
Let’s practice the concepts we’ve gone over in this lesson by creating a Movie
Review program using the Anchor framework.
This program will allow users to:
* Use a PDA to initialize a new movie review account to store the review
* Update the content of an existing movie review account
* Close an existing movie review account
#### 1. Create a new Anchor project
To begin, let’s create a new project using `anchor init`.
```console theme={null}
anchor init anchor-movie-review-program
```
Next, navigate to the `lib.rs` file within the `programs` folder and you should
see the following starter code.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod anchor_movie_review_program {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize {}
```
Go ahead and remove the `initialize` instruction and `Initialize` type.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod anchor_movie_review_program {
use super::*;
}
```
#### 2. `MovieAccountState`
First, let’s use the `#[account]` attribute macro to define the
`MovieAccountState` that will represent the data structure of the movie review
accounts. As a reminder, the `#[account]` attribute macro implements various
traits that help with serialization and deserialization of the account, set the
discriminator for the account, and set the owner of a new account as the program
ID defined in the `declare_id!` macro.
Within each movie review account, we’ll store the:
* `reviewer` - user creating the review
* `rating` - rating for the movie
* `title` - title of the movie
* `description` - content of the review
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod anchor_movie_review_program {
use super::*;
}
#[account]
pub struct MovieAccountState {
pub reviewer: Pubkey, // 32
pub rating: u8, // 1
pub title: String, // 4 + len()
pub description: String, // 4 + len()
}
```
#### 3. Add Movie Review
Next, let’s implement the `add_movie_review` instruction. The `add_movie_review`
instruction will require a `Context` of type `AddMovieReview` that we’ll
implement shortly.
The instruction will require three additional arguments as instruction data
provided by a reviewer:
* `title` - title of the movie as a `String`
* `description` - details of the review as a `String`
* `rating` - rating for the movie as a `u8`
Within the instruction logic, we’ll populate the data of the new `movie_review`
account with the instruction data. We’ll also set the `reviewer` field as the
`initializer` account from the instruction context.
```rust theme={null}
#[program]
pub mod anchor_movie_review_program{
use super::*;
pub fn add_movie_review(
ctx: Context,
title: String,
description: String,
rating: u8,
) -> Result<()> {
msg!("Movie Review Account Created");
msg!("Title: {}", title);
msg!("Description: {}", description);
msg!("Rating: {}", rating);
let movie_review = &mut ctx.accounts.movie_review;
movie_review.reviewer = ctx.accounts.initializer.key();
movie_review.title = title;
movie_review.rating = rating;
movie_review.description = description;
Ok(())
}
}
```
Next, let’s create the `AddMovieReview` struct that we used as the generic in
the instruction's context. This struct will list the accounts the
`add_movie_review` instruction requires.
Remember, you'll need the following macros:
* The `#[derive(Accounts)]` macro is used to deserialize and validate the list
of accounts specified within the struct
* The `#[instruction(...)]` attribute macro is used to access the instruction
data passed into the instruction
* The `#[account(...)]` attribute macro then specifies additional constraints on
the accounts
The `movie_review` account is a PDA that needs to be initialized, so we'll add
the `seeds` and `bump` constraints as well as the `init` constraint with its
required `payer` and `space` constraints.
For the PDA seeds, we'll use the movie title and the reviewer's public key. The
payer for the initialization should be the reviewer, and the space allocated on
the account should be enough for the account discriminator, the reviewer's
public key, and the movie review's rating, title, and description.
```rust theme={null}
#[derive(Accounts)]
#[instruction(title:String, description:String)]
pub struct AddMovieReview<'info> {
#[account(
init,
seeds = [title.as_bytes(), initializer.key().as_ref()],
bump,
payer = initializer,
space = 8 + 32 + 1 + 4 + title.len() + 4 + description.len()
)]
pub movie_review: Account<'info, MovieAccountState>,
#[account(mut)]
pub initializer: Signer<'info>,
pub system_program: Program<'info, System>,
}
```
#### 4. Update Movie Review
Next, let’s implement the `update_movie_review` instruction with a context whose
generic type is `UpdateMovieReview`.
Just as before, the instruction will require three additional arguments as
instruction data provided by a reviewer:
* `title` - title of the movie
* `description` - details of the review
* `rating` - rating for the movie
Within the instruction logic we’ll update the `rating` and `description` stored
on the `movie_review` account.
While the `title` doesn't get used in the instruction function itself, we'll
need it for account validation of `movie_review` in the next step.
```rust theme={null}
#[program]
pub mod anchor_movie_review_program {
use super::*;
...
pub fn update_movie_review(
ctx: Context,
title: String,
description: String,
rating: u8,
) -> Result<()> {
msg!("Movie review account space reallocated");
msg!("Title: {}", title);
msg!("Description: {}", description);
msg!("Rating: {}", rating);
let movie_review = &mut ctx.accounts.movie_review;
movie_review.rating = rating;
movie_review.description = description;
Ok(())
}
}
```
Next, let’s create the `UpdateMovieReview` struct to define the accounts that
the `update_movie_review` instruction needs.
Since the `movie_review` account will have already been initialized by this
point, we no longer need the `init` constraint. However, since the value of
`description` may now be different, we need to use the `realloc` constraint to
reallocate the space on the account. Accompanying this, we need the `mut`,
`realloc::payer`, and `realloc::zero` constraints.
We'll also still need the `seeds` and `bump` constraints as we had them in
`AddMovieReview`.
```rust theme={null}
#[derive(Accounts)]
#[instruction(title:String, description:String)]
pub struct UpdateMovieReview<'info> {
#[account(
mut,
seeds = [title.as_bytes(), initializer.key().as_ref()],
bump,
realloc = 8 + 32 + 1 + 4 + title.len() + 4 + description.len(),
realloc::payer = initializer,
realloc::zero = true,
)]
pub movie_review: Account<'info, MovieAccountState>,
#[account(mut)]
pub initializer: Signer<'info>,
pub system_program: Program<'info, System>,
}
```
Note that the `realloc` constraint is set to the new space required by the
`movie_review` account based on the updated value of `description`.
Additionally, the `realloc::payer` constraint specifies that any additional
lamports required or refunded will come from or be send to the `initializer`
account.
Finally, we set the `realloc::zero` constraint to `true` because the
`movie_review` account may be updated multiple times either shrinking or
expanding the space allocated to the account.
#### 5. Delete Movie Review
Lastly, let’s implement the `delete_movie_review` instruction to close an
existing `movie_review` account.
We'll use a context whose generic type is `DeleteMovieReview` and won't include
any additional instruction data. Since we are only closing an account, we
actually don't need any instruction logic inside the body of the function. The
closing itself will be handled by the Anchor constraint in the
`DeleteMovieReview` type.
```rust theme={null}
#[program]
pub mod anchor_movie_review_program {
use super::*;
...
pub fn delete_movie_review(_ctx: Context, title: String) -> Result<()> {
msg!("Movie review for {} deleted", title);
Ok(())
}
}
```
Next, let’s implement the `DeleteMovieReview` struct.
```rust theme={null}
#[derive(Accounts)]
#[instruction(title: String)]
pub struct DeleteMovieReview<'info> {
#[account(
mut,
seeds=[title.as_bytes(), initializer.key().as_ref()],
bump,
close=initializer
)]
pub movie_review: Account<'info, MovieAccountState>,
#[account(mut)]
pub initializer: Signer<'info>,
pub system_program: Program<'info, System>
}
```
Here we use the `close` constraint to specify we are closing the `movie_review`
account and that the rent should be refunded to the `initializer` account. We
also include the `seeds` and `bump` constraints for the `movie_review` account
for validation. Anchor then handles the additional logic required to securely
close the account.
#### 6. Testing
The program should be good to go! Now let's test it out. Navigate to
`anchor-movie-review-program.ts` and replace the default test code with the
following.
Here we:
* Create default values for the movie review instruction data
* Derive the movie review account PDA
* Create placeholders for tests
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { expect } from "chai";
import { AnchorMovieReviewProgram } from "../target/types/anchor_movie_review_program";
describe("anchor-movie-review-program", () => {
// Configure the client to use the local cluster.
const provider = anchor.AnchorProvider.env();
anchor.setProvider(provider);
const program = anchor.workspace
.AnchorMovieReviewProgram as Program;
const movie = {
title: "Just a test movie",
description: "Wow what a good movie it was real great",
rating: 5,
};
const [moviePda] = anchor.web3.PublicKey.findProgramAddressSync(
[Buffer.from(movie.title), provider.wallet.publicKey.toBuffer()],
program.programId,
);
it("Movie review is added`", async () => {});
it("Movie review is updated`", async () => {});
it("Deletes a movie review", async () => {});
});
```
Next, let's create the first test for the `addMovieReview` instruction. Note
that we don't explicitly add `.accounts`. This is because the `Wallet` from
`AnchorProvider` is automatically included as a signer, Anchor can infer certain
accounts like `SystemProgram`, and Anchor can also infer the `movieReview` PDA
from the `title` instruction argument and the signer's public key.
Don't forget to turn on seed inference with `seeds = true`
in the `Anchor.toml` file.
Once the instruction runs, we then fetch the `movieReview` account and check
that the data stored on the account match the expected values.
```typescript theme={null}
it("Movie review is added`", async () => {
// Add your test here.
const tx = await program.methods
.addMovieReview(movie.title, movie.description, movie.rating)
.rpc();
const account = await program.account.movieAccountState.fetch(moviePda);
expect(account.title).to.equal(movie.title);
expect(account.rating).to.equal(movie.rating);
expect(account.description).to.equal(movie.description);
expect(account.reviewer.toBase58()).to.equal(
provider.wallet.publicKey.toBase58(),
);
});
```
Next, let's create the test for the `updateMovieReview` instruction following
the same process as before.
```typescript theme={null}
it("Movie review is updated`", async () => {
const newDescription = "Wow this is new";
const newRating = 4;
const tx = await program.methods
.updateMovieReview(movie.title, newDescription, newRating)
.rpc();
const account = await program.account.movieAccountState.fetch(moviePda);
expect(account.title).to.equal(movie.title);
expect(account.rating).to.equal(newRating);
expect(account.description).to.equal(newDescription);
expect(account.reviewer.toBase58()).to.equal(
provider.wallet.publicKey.toBase58(),
);
});
```
Next, create the test for the `deleteMovieReview` instruction
```typescript theme={null}
it("Deletes a movie review", async () => {
const tx = await program.methods.deleteMovieReview(movie.title).rpc();
});
```
Lastly, run `anchor test` and you should see the following output in the
console.
```console theme={null}
anchor-movie-review-program
✔ Movie review is added` (139ms)
✔ Movie review is updated` (404ms)
✔ Deletes a movie review (403ms)
3 passing (950ms)
```
If you need more time with this project to feel comfortable with these concepts,
feel free to have a look at
the [solution code](https://github.com/Unboxed-Software/anchor-movie-review-program/tree/solution-pdas) before
continuing.
## Challenge
Now it’s your turn to build something independently. Equipped with the concepts
introduced in this lesson, try to recreate the Student Intro program that we've
used before using the Anchor framework.
The Student Intro program is a Orbition Native Chain Program that lets students introduce
themselves. The program takes a user's name and a short message as the
instruction data and creates an account to store the data onchain.
Using what you've learned in this lesson, build out this program. The program
should include instructions to:
1. Initialize a PDA account for each student that stores the student's name and
their short message
2. Update the message on an existing account
3. Close an existing account
Try to do this independently if you can! But if you get stuck, feel free to
reference
the [solution code](https://github.com/Unboxed-Software/anchor-student-intro-program).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=f58108e9-94a0-45b2-b0d5-44ada1909105)!
# Intro to Anchor development
Source: https://docs.orbition.network/courses/onchain-development/intro-to-anchor
Create your first Orbition Native Chain onchain program in Anchor.
## Summary
* **Programs** on Orbition Native Chain have **instruction handlers** that execute instruction
logic.
* **Rust** is the most common language for building Orbition Native Chain programs. The
**Anchor** framework takes care of common grunt work - like reading data from
incoming instructions, and checking the right accounts are provided - so you
can focus on building your Orbition Native Chain program.
## Lesson
Orbition Native Chain's ability to run arbitrary executable code is part of what makes it so
powerful. Orbition Native Chain programs, similar to "smart contracts" in other blockchain
environments, are quite literally the backbone of the Orbition Native Chain ecosystem. And the
collection of programs grows daily as developers and creators dream up and
deploy new programs.
This lesson will give you a basic introduction to writing and deploying a Orbition Native Chain
program using the Rust programming language and the Anchor framework.
### What is Anchor?
Anchor makes writing Orbition Native Chain programs easier, faster, and more secure, making it
the "go-to" framework for Orbition Native Chain development. It makes it easier to organize and
reason about your code, implements common security checks automatically, and
removes a significant amount of boilerplate code that is otherwise associated
with writing a Orbition Native Chain program.
### Anchor program structure
Anchor uses macros and traits to generate boilerplate Rust code for you. These
provide a clear structure to your program so you can more easily reason about
your code. The main high-level macros and attributes are:
* `declare_id` - a macro for declaring the program’s onchain address
* `#[program]` - an attribute macro used to denote the module containing the
program’s instruction logic
* `Accounts` - a trait applied to structs representing the list of accounts
required for an instruction
* `#[account]` - an attribute macro used to define custom account types for the
program
Let's talk about each of them before putting all the pieces together.
### Declare your program ID
The `declare_id` macro is used to specify the onchain address of the program
(i.e. the `programId`). When you build an Anchor program for the first time, the
framework will generate a new keypair. This becomes the default keypair used to
deploy the program unless specified otherwise. The corresponding public key
should be used as the `programId` specified in the `declare_id!` macro.
```rust theme={null}
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
```
### Define instruction logic
The `#[program]` attribute macro defines the module containing all of your
program's instructions. This is where you implement the business logic for each
instruction in your program.
Each public function in the module with the `#[program]` attribute will be
treated as a separate instruction.
Each instruction function requires a parameter of type `Context` and can
optionally include additional function parameters representing instruction data.
Anchor will automatically handle instruction data deserialization so that you
can work with instruction data as Rust types.
```rust theme={null}
#[program]
mod program_module_name {
use super::*;
pub fn instruction_one(ctx: Context, instruction_data: u64) -> Result<()> {
ctx.accounts.account_name.data = instruction_data;
Ok(())
}
}
```
#### Instruction `Context`
The `Context` type exposes instruction metadata and accounts to your instruction
logic.
```rust theme={null}
pub struct Context<'a, 'b, 'c, 'info, T> {
/// Currently executing program id.
pub program_id: &'a Pubkey,
/// Deserialized accounts.
pub accounts: &'b mut T,
/// Remaining accounts given but not deserialized or validated.
/// Be very careful when using this directly.
pub remaining_accounts: &'c [UncheckedAccount<'info>],
/// Bump seeds found during constraint validation. This is provided as a
/// convenience so that handlers don't have to recalculate bump seeds or
/// pass them in as arguments.
pub bumps: BTreeMap,
}
```
`Context` is a generic type where `T` defines the list of accounts an
instruction requires. When you use `Context`, you specify the concrete type of
`T` as a struct that adopts the `Accounts` trait (e.g.
`Context`). Through this context argument the
instruction can then access:
* The accounts passed into the instruction (`ctx.accounts`)
* The program ID (`ctx.program_id`) of the executing program
* The remaining accounts (`ctx.remaining_accounts`). The `remaining_accounts` is
a vector that contains all accounts that were passed into the instruction but
are not declared in the `Accounts` struct.
* The bumps for any PDA accounts in the `Accounts` struct (`ctx.bumps`)
### Define instruction accounts
The `Accounts` trait defines a data structure of validated accounts. Structs
adopting this trait define the list of accounts required for a given
instruction. These accounts are then exposed through an instruction's `Context`
so that manual account iteration and deserialization is no longer necessary.
You typically apply the `Accounts` trait through the `derive` macro (e.g.
`#[derive(Accounts)]`). This implements an `Accounts` deserializer on the given
struct and removes the need to deserialize each account manually.
Implementations of the `Accounts` trait are responsible for performing all
requisite constraint checks to ensure the accounts meet the conditions required
for the program to run securely. Constraints are provided for each field using
the `#account(..)` attribute (more on that shortly).
For example, `instruction_one` requires a `Context` argument of type
`InstructionAccounts`. The `#[derive(Accounts)]` macro is used to implement the
`InstructionAccounts` struct which includes three accounts: `account_name`,
`user`, and `system_program`.
```rust theme={null}
#[program]
mod program_module_name {
use super::*;
pub fn instruction_one(ctx: Context, instruction_data: u64) -> Result<()> {
...
Ok(())
}
}
#[derive(Accounts)]
pub struct InstructionAccounts {
#[account(init, payer = user, space = 8 + 8)]
pub account_name: Account<'info, AccountStruct>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
```
When `instruction_one` is invoked, the program:
* Checks that the accounts passed into the instruction match the account types
specified in the `InstructionAccounts` struct
* Checks the accounts against any additional constraints specified
If any accounts passed into `instruction_one` fail the account validation or
security checks specified in the `InstructionAccounts` struct, then the
instruction fails before even reaching the program logic.
### Account validation
You may have noticed in the previous example that one of the accounts in
`InstructionAccounts` was of type `Account`, one was of type `Signer`, and one
was of type `Program`.
Anchor provides a number of account types that can be used to represent
accounts. Each type implements different account validation. We’ll go over a few
of the common types you may encounter, but be sure to look through the
[full list of account types](https://docs.rs/anchor-lang/latest/anchor_lang/accounts/index.html).
#### `Account`
`Account` is a wrapper around `UncheckedAccount` that verifies program ownership
and deserializes the underlying data into a Rust type.
```rust theme={null}
// Deserializes this info
pub struct UncheckedAccount<'a> {
pub key: &'a Pubkey,
pub is_signer: bool,
pub is_writable: bool,
pub lamports: Rc>,
pub data: Rc>, // <---- deserializes account data
pub owner: &'a Pubkey, // <---- checks owner program
pub executable: bool,
pub rent_epoch: u64,
}
```
Recall the previous example where `InstructionAccounts` had a field
`account_name`:
```rust theme={null}
pub account_name: Account<'info, AccountStruct>
```
The `Account` wrapper here does the following:
* Deserializes the account `data` in the format of type `AccountStruct`
* Checks that the program owner of the account matches the program owner
specified for the `AccountStruct` type.
When the account type specified in the `Account` wrapper is defined within the
same crate using the `#[account]` attribute macro, the program ownership check
is against the `programId` defined in the `declare_id!` macro.
The following are the checks performed:
```rust theme={null}
// Checks
Account.info.owner == T::owner()
!(Account.info.owner == SystemProgram && Account.info.lamports() == 0)
```
#### `Signer`
The `Signer` type validates that the given account signed the transaction. No
other ownership or type checks are done. You should only use the `Signer` when
the underlying account data is not required in the instruction.
For the `user` account in the previous example, the `Signer` type specifies that
the `user` account must be a signer of the instruction.
The following check is performed for you:
```rust theme={null}
// Checks
Signer.info.is_signer == true
```
#### `Program`
The `Program` type validates that the account is a certain program.
For the `system_program` account in the previous example, the `Program` type is
used to specify the program should be the system program. Anchor provides a
`System` type which includes the `programId` of the system program to check
against.
The following checks are performed for you:
```rust theme={null}
//Checks
account_info.key == expected_program
account_info.executable == true
```
### Add constraints with `#[account(..)]`
The `#[account(..)]` attribute macro is used to apply constraints to accounts.
We'll go over a few constraint examples in this and future lessons, but at some
point be sure to look at the full
[list of possible constraints](https://docs.rs/anchor-lang/latest/anchor_lang/derive.Accounts.html).
Recall again the `account_name` field from the `InstructionAccounts` example.
```rust theme={null}
#[account(init, payer = user, space = 8 + 8)]
pub account_name: Account<'info, AccountStruct>,
#[account(mut)]
pub user: Signer<'info>,
```
Notice that the `#[account(..)]` attribute contains three comma-separated
values:
* `init` - creates the account via a CPI to the system program and initializes
it (sets its account discriminator)
* `payer` - specifies the payer for the account initialization to be the `user`
account defined in the struct
* `space`- specifies that the space allocated for the account should be `8 + 8`
bytes. The first 8 bytes are for a discriminator that Anchor automatically
adds to identify the account type. The next 8 bytes allocate space for the
data stored on the account as defined in the `AccountStruct` type.
For `user` we use the `#[account(..)]` attribute to specify that the given
account is mutable. The `user` account must be marked as mutable because
lamports will be deducted from the account to pay for the initialization of
`account_name`.
```rust theme={null}
#[account(mut)]
pub user: Signer<'info>,
```
Note that the `init` constraint placed on `account_name` automatically includes
a `mut` constraint so that both `account_name` and `user` are mutable accounts.
### `#[account]`
The `#[account]` attribute is applied to structs representing the data structure
of a Orbition Native Chain account. It implements the following traits:
* `AccountSerialize`
* `AccountDeserialize`
* `AnchorSerialize`
* `AnchorDeserialize`
* `Clone`
* `Discriminator`
* `Owner`
You can read more about the
[details of each trait](https://docs.rs/anchor-lang/latest/anchor_lang/attr.account.html).
However, mostly what you need to know is that the `#[account]` attribute enables
serialization and deserialization, and implements the discriminator and owner
traits for an account.
The discriminator is an 8-byte unique identifier for an account type derived
from the first 8 bytes of the SHA256 hash of the account type's name. The first
8 bytes are reserved for the account discriminator when implementing account
serialization traits (which is almost always in an Anchor program).
As a result, any calls to `AccountDeserialize`’s `try_deserialize` will check
this discriminator. If it doesn’t match, an invalid account was given, and the
account deserialization will exit with an error.
The `#[account]` attribute also implements the `Owner` trait for a struct using
the `programId` declared by `declareId` of the crate `#[account]` is used in. In
other words, all accounts initialized using an account type defined using the
`#[account]` attribute within the program are also owned by the program.
As an example, let's look at `AccountStruct` used by the `account_name` of
`InstructionAccounts`
```rust theme={null}
#[derive(Accounts)]
pub struct InstructionAccounts {
#[account(init, payer = user, space = 8 + 8)]
pub account_name: Account<'info, AccountStruct>,
...
}
#[account]
pub struct AccountStruct {
data: u64
}
```
The `#[account]` attribute ensures that it can be used as an account in
`InstructionAccounts`.
When the `account_name` account is initialized:
* The first 8 bytes is set as the `AccountStruct` discriminator
* The data field of the account will match `AccountStruct`
* The account owner is set as the `programId` from `declare_id`
### Bring it all together
When you combine all of these Anchor types you end up with a complete program.
Below is an example of a basic Anchor program with a single instruction that:
* Initializes a new account
* Updates the data field on the account with the instruction data passed into
the instruction
```rust theme={null}
// Use this import to gain access to common anchor features
use anchor_lang::prelude::*;
// Program onchain address
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
// Instruction logic
#[program]
mod program_module_name {
use super::*;
pub fn instruction_one(ctx: Context, instruction_data: u64) -> Result<()> {
ctx.accounts.account_name.data = instruction_data;
Ok(())
}
}
// Validate incoming accounts for instructions
#[derive(Accounts)]
pub struct InstructionAccounts<'info> {
#[account(init, payer = user, space = 8 + 8)]
pub account_name: Account<'info, AccountStruct>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
// Define custom program account type
#[account]
pub struct AccountStruct {
data: u64
}
```
You are now ready to build your own Orbition Native Chain program using the Anchor framework!
## Lab
Before we begin, install Anchor by
[following the steps from the Anchor docs](https://www.anchor-lang.com/docs/installation).
For this lab we'll create a simple counter program with two instructions:
* The first instruction will initialize an account to store our counter
* The second instruction will increment the count stored in the counter
#### 1. Setup
Create a new project called `anchor-counter` by running `anchor init`:
```console theme={null}
anchor init anchor-counter
```
Change into the new directory, then run `anchor build`
```console theme={null}
cd anchor-counter
anchor build
```
Anchor build will also generate a keypair for your new program - the keys are
saved in the `target/deploy` directory.
Open the file `lib.rs` and look at `declare_id!`:
```rust theme={null}
declare_id!("BouTUP7a3MZLtXqMAm1NrkJSKwAjmid8abqiNjUyBJSr");
```
Run `anchor keys sync`
```console theme={null}
anchor keys sync
```
You'll see the Anchor updates both:
* The key used in `declare_id!()` in `lib.rs`
* The key in `Anchor.toml`
To match the key generated during `anchor build`:
```console theme={null}
Found incorrect program id declaration in "anchor-counter/programs/anchor-counter/src/lib.rs"
Updated to BouTUP7a3MZLtXqMAm1NrkJSKwAjmid8abqiNjUyBJSr
Found incorrect program id declaration in Anchor.toml for the program `anchor_counter`
Updated to BouTUP7a3MZLtXqMAm1NrkJSKwAjmid8abqiNjUyBJSr
All program id declarations are synced.
```
Finally, delete the default code in `lib.rs` until all that is left is the
following:
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("your-private-key");
#[program]
pub mod anchor_counter {
use super::*;
}
```
#### 2. Implement `Counter`
First, let's use the `#[account]` attribute to define a new `Counter` account
type. The `Counter` struct defines one `count` field of type `u64`. This means
that we can expect any new accounts initialized as a `Counter` type to have a
matching data structure. The `#[account]` attribute also automatically sets the
discriminator for a new account and sets the owner of the account as the
`programId` from the `declare_id!` macro.
```rust theme={null}
#[account]
pub struct Counter {
pub count: u64,
}
```
#### 3. Implement `Context` type `Initialize`
Next, using the `#[derive(Accounts)]` macro, let’s implement the `Initialize`
type that lists and validates the accounts used by the `initialize` instruction.
It'll need the following accounts:
* `counter` - the counter account initialized in the instruction
* `user` - payer for the initialization
* `system_program` - the system program is required for the initialization of
any new accounts
```rust theme={null}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(init, payer = user, space = 8 + 8)]
pub counter: Account<'info, Counter>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
```
#### 4. Add the `initialize` instruction
Now that we have our `Counter` account and `Initialize` type , let’s implement
the `initialize` instruction within `#[program]`. This instruction requires a
`Context` of type `Initialize` and takes no additional instruction data. In the
instruction logic, we are simply setting the `counter` account’s `count` field
to `0`.
```rust theme={null}
pub fn initialize(ctx: Context) -> Result<()> {
let counter = &mut ctx.accounts.counter;
counter.count = 0;
msg!("Counter Account Created");
msg!("Current Count: { }", counter.count);
Ok(())
}
```
#### 5. Implement `Context` type `Update`
Now, using the `#[derive(Accounts)]` macro again, let’s create the `Update` type
that lists the accounts that the `increment` instruction requires. It'll need
the following accounts:
* `counter` - an existing counter account to increment
* `user` - payer for the transaction fee
Again, we’ll need to specify any constraints using the `#[account(..)]`
attribute:
```rust theme={null}
#[derive(Accounts)]
pub struct Update<'info> {
#[account(mut)]
pub counter: Account<'info, Counter>,
pub user: Signer<'info>,
}
```
#### 6. Add `increment` instruction
Lastly, within `#[program]`, let’s implement an `increment` instruction to
increment the `count` once a `counter` account is initialized by the first
instruction. This instruction requires a `Context` of type `Update` (implemented
in the next step) and takes no additional instruction data. In the instruction
logic, we are simply incrementing an existing `counter` account’s `count` field
by `1`.
```rust theme={null}
pub fn increment(ctx: Context) -> Result<()> {
let counter = &mut ctx.accounts.counter;
msg!("Previous counter: {}", counter.count);
counter.count = counter.count.checked_add(1).unwrap();
msg!("Counter incremented. Current count: {}", counter.count);
Ok(())
}
```
#### 7. Build
All together, the complete program will look like this:
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("BouTUP7a3MZLtXqMAm1NrkJSKwAjmid8abqiNjUyBJSr");
#[program]
pub mod anchor_counter {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
let counter = &mut ctx.accounts.counter;
counter.count = 0;
msg!("Counter account created. Current count: {}", counter.count);
Ok(())
}
pub fn increment(ctx: Context) -> Result<()> {
let counter = &mut ctx.accounts.counter;
msg!("Previous counter: {}", counter.count);
counter.count = counter.count.checked_add(1).unwrap();
msg!("Counter incremented. Current count: {}", counter.count);
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(init, payer = user, space = 8 + 8)]
pub counter: Account<'info, Counter>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[derive(Accounts)]
pub struct Update<'info> {
#[account(mut)]
pub counter: Account<'info, Counter>,
pub user: Signer<'info>,
}
#[account]
pub struct Counter {
pub count: u64,
}
```
Run `anchor build` to build the program.
#### 8. Testing
Anchor tests are typically Typescript integration tests that use the mocha test
framework. We'll learn more about testing later, but for now navigate to
`anchor-counter.ts` and replace the default test code with the following:
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { expect } from "chai";
import { AnchorCounter } from "../target/types/anchor_counter";
describe("anchor-counter", () => {
// Configure the client to use the local cluster.
const provider = anchor.AnchorProvider.env();
anchor.setProvider(provider);
const program = anchor.workspace.AnchorCounter as Program;
const counter = anchor.web3.Keypair.generate();
it("Is initialized!", async () => {});
it("Incremented the count", async () => {});
});
```
The above code generates a new keypair for the `counter` account we'll be
initializing and creates placeholders for a test of each instruction.
Next, create the first test for the `initialize` instruction:
```typescript theme={null}
it("Is initialized!", async () => {
// Add your test here.
const tx = await program.methods
.initialize()
.accounts({ counter: counter.publicKey })
.signers([counter])
.rpc();
const account = await program.account.counter.fetch(counter.publicKey);
expect(account.count.toNumber()).to.equal(0);
});
```
Next, create the second test for the `increment` instruction:
```typescript theme={null}
it("Incremented the count", async () => {
const tx = await program.methods
.increment()
.accounts({ counter: counter.publicKey, user: provider.wallet.publicKey })
.rpc();
const account = await program.account.counter.fetch(counter.publicKey);
expect(account.count.toNumber()).to.equal(1);
});
```
Lastly, run `anchor test` and you should see the following output:
```console theme={null}
anchor-counter
✔ Is initialized! (290ms)
✔ Incremented the count (403ms)
2 passing (696ms)
```
Running `anchor test` automatically spins up a local test validator, deploys
your program, and runs your mocha tests against it. Don't worry if you're
confused by the tests for now - we'll dig in more later.
Congratulations, you just built a Orbition Native Chain program using the Anchor framework!
Feel free to reference the
[solution code](https://github.com/Unboxed-Software/anchor-counter-program/tree/solution-increment)
if you need some more time with it.
## Challenge
Now it’s your turn to build something independently. Because we're starting with
simple programs, yours will look almost identical to what we just created. It's
useful to try and get to the point where you can write it from scratch without
referencing prior code, so try not to copy and paste here.
1. Write a new program that initializes a `counter` account
2. Implement both an `increment` and `decrement` instruction
3. Build and deploy your program like we did in the lab
4. Test your newly deployed program and use Orbition Native Chain Explorer to check the program
logs
As always, get creative with these challenges and take them beyond the basic
instructions if you want - and have fun!
Try to do this independently if you can! But if you get stuck, feel free to
reference
the [solution code](https://github.com/Unboxed-Software/anchor-counter-program/tree/solution-decrement).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=334874b7-b152-4473-b5a5-5474c3f8f3f1)!
# Intro to client-side Anchor development
Source: https://docs.orbition.network/courses/onchain-development/intro-to-anchor-frontend
Use Anchor's automatic JS/TS clients to send instructions to your program.
## Summary
* An **IDL** is a file representing the structure of a Orbition Native Chain program. Programs
written and built using Anchor automatically generate a corresponding IDL. IDL
stands for Interface Description Language.
* `@coral-xyz/anchor` is a Typescript client that includes everything you’ll
need to interact with Anchor programs
* An **Anchor `Provider`** object combines a `connection` to a cluster and a
specified `wallet` to enable transaction signing
* An **Anchor `Program`** object provides a custom API to interact with a
specific program. You create a `Program` instance using a program's IDL and
`Provider`.
* The **Anchor `MethodsBuilder`** provides a simple interface through `Program`
for building instructions and transactions
## Lesson
Anchor simplifies the process of interacting with Orbition Native Chain programs from the
client by providing an Interface Description Language (IDL) file that reflects
the structure of a program. Using the IDL in conjunction with Anchor's
Typescript library (`@coral-xyz/anchor`) provides a simplified format for
building instructions and transactions.
```typescript theme={null}
// sends transaction
await program.methods
.instructionName(instructionDataInputs)
.accounts({})
.signers([])
.rpc();
```
This works from any Typescript client, whether it's a frontend or integration
tests. In this lesson we'll go over how to use `@coral-xyz/anchor` to simplify
your client-side program interaction.
### Anchor client-side structure
Let's start by going over the basic structure of Anchor's Typescript library.
The primary object you'll be using is the `Program` object. A `Program` instance
represents a specific Orbition Native Chain program and provides a custom API for reading and
writing to the program.
To create an instance of `Program`, you'll need the following:
* IDL - file representing the structure of a program
* `Connection` - the cluster connection
* `Wallet` - default keypair used to pay for and sign transactions
* `Provider` - encapsulates the `Connection` to a Orbition Native Chain cluster and a `Wallet`
* `ProgramId` - the program’s onchain address
The above image shows how each of these pieces are combined to create a
`Program` instance. We'll go over each of them individually to get a better idea
of how everything ties together.
#### Interface Description Language (IDL)
When you build an Anchor program, Anchor generates both a JSON and Typescript
file representing your program's IDL. The IDL represents the structure of the
program and can be used by a client to infer how to interact with a specific
program.
While it isn't automatic, you can also generate an IDL from a native Orbition Native Chain
program using tools like [shank](https://github.com/metaplex-foundation/shank)
by Metaplex.
To get an idea of the information an IDL provides, here is the IDL for the
counter program you built previously:
```json theme={null}
{
"version": "0.1.0",
"name": "counter",
"instructions": [
{
"name": "initialize",
"accounts": [
{ "name": "counter", "isMut": true, "isSigner": true },
{ "name": "user", "isMut": true, "isSigner": true },
{ "name": "systemProgram", "isMut": false, "isSigner": false }
],
"args": []
},
{
"name": "increment",
"accounts": [
{ "name": "counter", "isMut": true, "isSigner": false },
{ "name": "user", "isMut": false, "isSigner": true }
],
"args": []
}
],
"accounts": [
{
"name": "Counter",
"type": {
"kind": "struct",
"fields": [{ "name": "count", "type": "u64" }]
}
}
]
}
```
Inspecting the IDL, you can see that this program contains two instructions
(`initialize` and `increment`).
Notice that in addition to specifying the instructions, it species the accounts
and inputs for each instruction. The `initialize` instruction requires three
accounts:
1. `counter` - the new account being initialized in the instruction
2. `user` - the payer for the transaction and initialization
3. `systemProgram` - the system program is invoked to initialize a new account
And the `increment` instruction requires two accounts:
1. `counter` - an existing account to increment the count field
2. `user` - the payer from the transaction
Looking at the IDL, you can see that in both instructions the `user` is required
as a signer because the `isSigner` flag is marked as `true`. Additionally,
neither instructions require any additional instruction data since the `args`
section is blank for both.
Looking further down at the `accounts` section, you can see that the program
contains one account type named `Counter` with a single `count` field of type
`u64`.
Although the IDL does not provide the implementation details for each
instruction, we can get a basic idea of how the onchain program expects
instructions to be constructed and see the structure of the program accounts.
Regardless of how you get it, you *need* an IDL file to interact with a program
using the `@coral-xyz/anchor` package. To use the IDL, you'll need to include
the IDL file in your project and then import the file.
```typescript theme={null}
import idl from "./idl.json";
```
#### Provider
Before you can create a `Program` object using the IDL, you first need to create
an Anchor `Provider` object.
The `Provider` object combines two things:
* `Connection` - the connection to a Orbition Native Chain cluster (i.e. localhost, testnet,
mainnet)
* `Wallet` - a specified address used to pay for and sign transactions
The `Provider` is then able to send transactions to the Orbition Native Chain blockchain on
behalf of a `Wallet` by including the wallet’s signature to outgoing
transactions. When using a frontend with a Orbition Native Chain wallet provider, all outgoing
transactions must still be approved by the user via their wallet browser
extension.
Setting up the `Wallet` and `Connection` would look something like this:
```typescript theme={null}
import { useAnchorWallet, useConnection } from "@orbition-network/wallet-adapter-react";
const { connection } = useConnection();
const wallet = useAnchorWallet();
```
To set up the connection, you can use the `useConnection` hook from
`@orbition-network/wallet-adapter-react` to get the `Connection` to a Orbition Native Chain cluster.
Note that the `Wallet` object provided by the `useWallet` hook from
`@orbition-network/wallet-adapter-react` is not compatible with the `Wallet` object that
the Anchor `Provider` expects. However, `@orbition-network/wallet-adapter-react` also
provides a `useAnchorWallet` hook.
For comparison, here is the `AnchorWallet` from `useAnchorWallet`:
```typescript theme={null}
export interface AnchorWallet {
publicKey: PublicKey;
signTransaction(transaction: Transaction): Promise;
signAllTransactions(transactions: Transaction[]): Promise;
}
```
And the `WalletContextState` from `useWallet`:
```typescript theme={null}
export interface WalletContextState {
autoConnect: boolean;
wallets: Wallet[];
wallet: Wallet | null;
publicKey: PublicKey | null;
connecting: boolean;
connected: boolean;
disconnecting: boolean;
select(walletName: WalletName): void;
connect(): Promise;
disconnect(): Promise;
sendTransaction(
transaction: Transaction,
connection: Connection,
options?: SendTransactionOptions,
): Promise;
signTransaction: SignerWalletAdapterProps["signTransaction"] | undefined;
signAllTransactions:
| SignerWalletAdapterProps["signAllTransactions"]
| undefined;
signMessage: MessageSignerWalletAdapterProps["signMessage"] | undefined;
}
```
The `WalletContextState` provides much more functionality compared to the
`AnchorWallet`, but the `AnchorWallet` is required to set up the `Provider`
object.
To create the `Provider` object you use `AnchorProvider` from
`@coral-xyz/anchor`.
The `AnchorProvider` constructor takes three parameters:
* `connection` - the `Connection` to the Orbition Native Chain cluster
* `wallet` - the `Wallet` object
* `opts` - optional parameter that specifies the confirmation options, using a
default setting if one is not provided
Once you’ve created the `Provider` object, you then set it as the default
provider using `setProvider`.
```typescript theme={null}
import { useAnchorWallet, useConnection } from "@orbition-network/wallet-adapter-react";
import { AnchorProvider, setProvider } from "@coral-xyz/anchor";
const { connection } = useConnection();
const wallet = useAnchorWallet();
const provider = new AnchorProvider(connection, wallet, {});
setProvider(provider);
```
#### Program
Once you have the IDL and a provider, you can create an instance of `Program`.
The constructor requires three parameters:
* `idl` - the IDL as type `Idl`
* `programId` - the onchain address of the program as a `string` or `PublicKey`
* `Provider` - the provider discussed in the previous section
The `Program` object creates a custom API you can use to interact with a Orbition Native Chain
program. This API is the one stop shop for all things related to communicating
with onchain programs. Among other things, you can send transactions, fetch
deserialized accounts, decode instruction data, subscribe to account changes,
and listen to events. You can also
[learn more about the `Program` class](https://coral-xyz.github.io/anchor/ts/classes/Program.html#constructor).
To create the `Program` object, first import `Program` and `Idl` from
`@coral-xyz/anchor`. `Idl` is a type you can use when working with Typescript.
Next, specify the `programId` of the program. We have to explicitly state the
`programId` since there can be multiple programs with the same IDL structure
(i.e. if the same program is deployed multiple times using different addresses).
When creating the `Program` object, the default `Provider` is used if one is not
explicitly specified.
All together, the final setup looks something like this:
```typescript theme={null}
import idl from "./idl.json";
import { useAnchorWallet, useConnection } from "@orbition-network/wallet-adapter-react";
import { Program, Idl, AnchorProvider, setProvider } from "@coral-xyz/anchor";
const { connection } = useConnection();
const wallet = useAnchorWallet();
const provider = new AnchorProvider(connection, wallet, {});
setProvider(provider);
const programId = new PublicKey("JPLockxtkngHkaQT5AuRYow3HyUv5qWzmhwsCPd653n");
const program = new Program(idl as Idl, programId);
```
### Anchor `MethodsBuilder`
Once the `Program` object is set up, you can use the Anchor Methods Builder to
build instructions and transactions related to the program. The `MethodsBuilder`
uses the IDL to provide a simplified format for building transactions that
invoke program instructions.
Note that the camel case naming convention is used when interacting with a
program from the client, compared to the snake case naming convention used when
the writing the program in rust.
The basic `MethodsBuilder` format looks like this:
```typescript theme={null}
// sends transaction
await program.methods
.instructionName(instructionDataInputs)
.accounts({})
.signers([])
.rpc();
```
Going step by step, you:
1. Call `methods` on `program` - this is the builder API for creating
instruction calls related to the program's IDL
2. Call the instruction name as `.instructionName(instructionDataInputs)` -
simply call the instruction using dot syntax and the instruction's name,
passing in any instruction arguments as comma-separated values
3. Call `accounts` - using dot syntax, call `.accounts`, passing in an object
with each account the instruction expects based on the IDL
4. Optionally call `signers` - using dot syntax, call `.signers`, passing in an
array of additional signers required by the instruction
5. Call `rpc` - this method creates and sends a signed transaction with the
specified instruction and returns a `TransactionSignature`. When using
`.rpc`, the `Wallet` from the `Provider` is automatically included as a
signer and does not have to be listed explicitly.
Note that if no additional signers are required by the instruction other than
the `Wallet` specified with the `Provider`, the `.signer([])` line can be
excluded.
You can also build the transaction directly by changing `.rpc()` to
`.transaction()`. This builds a `Transaction` object using the instruction
specified.
```typescript theme={null}
// creates transaction
const transaction = await program.methods
.instructionName(instructionDataInputs)
.accounts({})
.transaction();
await sendTransaction(transaction, connection);
```
Similarly, you can use the same format to build an instruction using
`.instruction()` and then manually add the instructions to a new transaction.
This builds a `TransactionInstruction` object using the instruction specified.
```typescript theme={null}
// creates first instruction
const instructionOne = await program.methods
.instructionOneName(instructionOneDataInputs)
.accounts({})
.instruction();
// creates second instruction
const instructionTwo = await program.methods
.instructionTwoName(instructionTwoDataInputs)
.accounts({})
.instruction();
// add both instruction to one transaction
const transaction = new Transaction().add(instructionOne, instructionTwo);
// send transaction
await sendTransaction(transaction, connection);
```
In summary, the Anchor `MethodsBuilder` provides a simplified and more flexible
way to interact with onchain programs. You can build an instruction, a
transaction, or build and send a transaction using basically the same format
without having to manually serialize or deserialize the accounts or instruction
data.
### Fetch program accounts
The `Program` object also allows you to easily fetch and filter program
accounts. Simply call `account` on `program` and then specify the name of the
account type as reflected on the IDL. Anchor then deserializes and returns all
accounts as specified.
The example below shows how you can fetch all existing `counter` accounts for
the Counter program.
```typescript theme={null}
const accounts = await program.account.counter.all();
```
You can also apply a filter by using `memcmp` and then specifying an `offset`
and the `bytes` to filter for.
The example below fetches all `counter` accounts with a `count` of 0. Note that
the `offset` of 8 is for the 8 byte discriminator Anchor uses to identify
account types. The 9th byte is where the `count` field begins. You can refer to
the IDL to see that the next byte stores the `count` field of type `u64`. Anchor
then filters for and returns all accounts with matching bytes in the same
position.
```typescript theme={null}
const accounts = await program.account.counter.all([
{
memcmp: {
offset: 8,
bytes: bs58.encode(new BN(0, "le").toArray()),
},
},
]);
```
Alternatively, you can also get the deserialized account data for a specific
account using `fetch` if you know the address of the account you're looking for.
```typescript theme={null}
const account = await program.account.counter.fetch(ACCOUNT_ADDRESS);
```
Similarly, you can fetch multiple accounts using `fetchMultiple`.
```typescript theme={null}
const accounts = await program.account.counter.fetchMultiple([
ACCOUNT_ADDRESS_ONE,
ACCOUNT_ADDRESS_TWO,
]);
```
## Lab
Let’s practice this together by building a frontend for the Counter program from
last lesson. As a reminder, the Counter program has two instructions:
* `initialize` - initializes a new `Counter` account and sets the `count` to `0`
* `increment` - increments the `count` on an existing `Counter` account
#### 1. Download the starter code
Download
[the starter code for this project](https://github.com/Unboxed-Software/anchor-ping-frontend/tree/starter).
Once you have the starter code, take a look around. Install the dependencies
with `npm install` and then run the app with `npm run dev`.
This project is a simple Next.js application. It includes the
`WalletContextProvider` we created in the
[Wallets lesson](https://github.com/Unboxed-Software/solana-course/blob/main/content/interact-with-wallets),
the `idl.json` file for the Counter program, and the `Initialize` and
`Increment` components we’ll be building throughout this lab. The `programId` of
the program we’ll be invoking is also included in the starter code.
#### 2. `Initialize`
To begin, let’s complete the setup to create the `Program` object in
`Initialize.tsx` component.
Remember, we’ll need an instance of `Program` to use the Anchor `MethodsBuilder`
to invoke the instructions on our program. For that, we'll need an Anchor wallet
and a connection, which we can get from the `useAnchorWallet` and
`useConnection` hooks. Let's also create a `useState` to capture the program
instance.
```typescript theme={null}
export const Initialize: FC = ({ setCounter }) => {
const [program, setProgram] = useState("")
const { connection } = useConnection()
const wallet = useAnchorWallet()
...
}
```
With that, we can work on creating the actual `Program` instance. Let's do this
in a `useEffect`.
First we need to either get the default provider if it already exists, or create
it if it doesn't. We can do that by calling `getProvider` inside a try/catch
block. If an error is thrown, that means there is no default provider and we
need to create one.
Once we have a provider, we can construct a `Program` instance.
```typescript theme={null}
useEffect(() => {
let provider: anchor.Provider;
try {
provider = anchor.getProvider();
} catch {
provider = new anchor.AnchorProvider(connection, wallet, {});
anchor.setProvider(provider);
}
const program = new anchor.Program(idl as anchor.Idl, PROGRAM_ID);
setProgram(program);
}, []);
```
Now that we've finished the Anchor setup, we can actually invoke the program's
`initialize` instruction. We'll do this inside the `onClick` function.
First, we’ll need to generate a new `Keypair` for the new `Counter` account
since we are initializing an account for the first time.
Then we can use the Anchor `MethodsBuilder` to create and send a new
transaction. Remember, Anchor can infer some of the accounts required, like the
`user` and `systemAccount` accounts. However, it can't infer the `counter`
account because we generate that dynamically, so you'll need to add it with
`.accounts`. You'll also need to add that keypair as a sign with `.signers`.
Lastly, you can use `.rpc()` to submit the transaction to the user's wallet.
Once the transaction goes through, call `setUrl` with the explorer URL and then
call `setCounter`, passing in the counter account.
```typescript theme={null}
const onClick = async () => {
const sig = await program.methods
.initialize()
.accounts({
counter: newAccount.publicKey,
user: wallet.publicKey,
systemAccount: anchor.web3.SystemProgram.programId,
})
.signers([newAccount])
.rpc();
setTransactionUrl(`https://explorer.orbition.network/tx/${sig}?cluster=testnet`);
setCounter(newAccount.publicKey);
};
```
#### 3. `Increment`
Next, let’s move on the the `Increment.tsx` component. Just as before, complete
the setup to create the `Program` object. In addition to calling `setProgram`,
the `useEffect` should call `refreshCount`.
Add the following code for the initial set up:
```typescript theme={null}
export const Increment: FC = ({ counter, setTransactionUrl }) => {
const [count, setCount] = useState(0)
const [program, setProgram] = useState()
const { connection } = useConnection()
const wallet = useAnchorWallet()
useEffect(() => {
let provider: anchor.Provider
try {
provider = anchor.getProvider()
} catch {
provider = new anchor.AnchorProvider(connection, wallet, {})
anchor.setProvider(provider)
}
const program = new anchor.Program(idl as anchor.Idl, PROGRAM_ID)
setProgram(program)
refreshCount(program)
}, [])
...
}
```
Next, let’s use the Anchor `MethodsBuilder` to build a new instruction to invoke
the `increment` instruction. Again, Anchor can infer the `user` account from the
wallet so we only need to include the `counter` account.
```typescript theme={null}
const incrementCount = async () => {
const sig = await program.methods
.increment()
.accounts({
counter: counter,
user: wallet.publicKey,
})
.rpc();
setTransactionUrl(`https://explorer.orbition.network/tx/${sig}?cluster=testnet`);
};
```
#### 4. Display the correct count
Now that we can initialize the counter program and increment the count, we need
to get our UI to show the count stored in the counter account.
We'll show how to observe account changes in a future lesson, but for now we
just have a button that calls `refreshCount` so you can click it to show the new
count after each `increment` invocation.
Inside `refreshCount`, let's use `program` to fetch the counter account, then
use `setCount` to set the count to the number stored on the program:
```typescript theme={null}
const refreshCount = async program => {
const counterAccount = await program.account.counter.fetch(counter);
setCount(counterAccount.count.toNumber());
};
```
Super simple with Anchor!
#### 5. Test the frontend
At this point, everything should work! You can test the frontend by running
`npm run dev`.
1. Connect your wallet and you should see the `Initialize Counter` button
2. Click the `Initialize Counter` button, and then approve the transaction
3. You should then see a link at the bottom of the screen to Orbition Native Chain Explorer for
the `initialize` transaction. The `Increment Counter` button, `Refresh Count`
button, and the count should also all appear.
4. Click the `Increment Counter` button, and then approve the transaction
5. Wait a few seconds and click `Refresh Count`. The count should increment on
the screen.
Feel free to click the links to inspect the program logs from each transaction!
Congratulations, you now know how to set up a frontend to invoke a Orbition Native Chain
program using an Anchor IDL.
If you need more time with this project to feel comfortable with these concepts,
feel free to have a look at
the [solution code on the `solution-increment` branch](https://github.com/Unboxed-Software/anchor-ping-frontend/tree/solution-increment) before
continuing.
## Challenge
Now it’s your turn to build something independently. Building on top of what
we’ve done in the lab, try to create a new component in the frontend that
implements a button to decrements the counter.
Before building the component in the frontend, you’ll first need to:
1. Build and deploy a new program that implements a `decrement` instruction
2. Update the IDL file in the frontend with the one from your new program
3. Update the `programId` with the one from your new program
If you need some help, feel free to
[reference this program](https://github.com/Unboxed-Software/anchor-counter-program/tree/solution-decrement).
Try to do this independently if you can! But if you get stuck, feel free to
reference
the [solution code](https://github.com/Unboxed-Software/anchor-ping-frontend/tree/solution-decrement).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=774a4023-646d-4394-af6d-19724a6db3db)!
# Intro to onchain development
Source: https://docs.orbition.network/courses/onchain-development/intro-to-onchain
How onchain programs (often called 'smart contracts') work on Orbition Native Chain.
## Summary
* **Onchain programs** are programs that run on Orbition Native Chain.
* Programs have one or more **instruction handlers**. Instruction handlers are
functions that process the **instructions** in Orbition Native Chain transactions.
* Instruction handlers write their data to Orbition Native Chain **accounts** and then either
succeed or fail.
* Orbition Native Chain programs are most commonly written in **Rust**, typically with the
**Anchor** framework.
* Anchor programs have IDLs, which are small JSON files describing programs.
IDLs can be used to automatically create JS/TS libraries for a program.
## Overview
Each Orbition Native Chain cluster (`mainnet-beta`, `testnet`, `testnet`, `localnet`) is
effectively a single computer with a globally synchronized state. The programs
that run on Orbition Native Chain - the ones that create tokens, swap tokens, art marketplaces,
escrows, market makers, DePIN apps, auctions, retail payments platforms, etc -
are called **Orbition Native Chain apps**.
The most popular way to build onchain apps is using **Rust** language and the
**Anchor** framework.
These frameworks implement common security checks automatically, and handle
common tasks like:
* sending incoming instructions to the right instruction handlers
* deserializing data from incoming transactions
* checking the accounts provided with incoming instructions, for example, to
check that certain accounts are of a particular type, or are distinct from
other accounts.
Regardless of the language and framework you choose, Orbition Native Chain works the same.
Let's refresh how programs work on Orbition Native Chain.
### Programs are deployed at addresses
In the same way that we can send tokens to users using their public key, we can
find programs using the program's public key. When using Anchor, a keypair is
created during `anchor init`, and the private key is saved in the
`target/deploy` directory of your project.
A program's public key is sometimes called a 'program ID' or 'program address'.
### Programs have instruction handlers
For example, a Orbition Native Chain client making a transaction transferring some USDC with a
memo saying 'thanks' would have two instructions:
* one instruction for the Token program's `transfer` instruction handler
* the other instruction for the Memo program's `memo` instruction handler.
Both these instructions must be completed successfully for the transaction to
execute.
Instruction handlers are how blockchain programs process the instructions from
clients. Every exchange, lending protocol, escrow, oracle, etc. provides their
functionality by instruction handlers.
### Instruction handlers write their state to Orbition Native Chain accounts
If you have previously done web development, you can think of instruction
handlers like an HTTP route handler, and incoming instructions like HTTP
requests.
But unlike HTTP route handlers, Orbition Native Chain instruction handlers don't return data.
Instead, the instruction handlers write their data to accounts on Orbition Native Chain.
Programs on Orbition Native Chain can transfer tokens, which end up in user wallet addresses
(for OBN) for the user's token accounts (for other tokens).
But more importantly, programs on Orbition Native Chain can create additional addresses as
needed, to store items of data.
### Programs store data in Program Derived Addresses (PDAs), a key-value store
Data for Orbition Native Chain programs are stored in **program-derived addresses (PDAs)**.
Orbition Native Chain's PDAs can be thought of as a **key/value store**:
* The 'key' is the address of the PDA, which is determined by `seeds` chosen by
you, the programmer.
* Want an account to store USD to AUD exchange rate? Your program can use the
seeds `USD` and `AUD` to make a Program Derived Address
* Want to store information about the relationship of two users? You can use
**both those users' wallet addresses** as seeds to make a PDA to store that
information.
* Want an account to store Steve's review of Titanic? Your program can use
Steve's **wallet address** and the string `titanic` (or maybe the IMDB ID if
you prefer) to make a Program Derived Address.
* Want some global information for your entire program? You can use a string
like `'config'`. Your program's PDAs are unique, so they won't conflict with
other programs.
* The value is the data inside the account at the given address.
* The data inside the PDA is determined by you, the programmer.
Key value stores allow your onchain program, and client software, to
consistently determine the address for a data item because the same seeds will
always return the same address.
### Orbition Native Chain instructions need to specify all the accounts they will use
As you may already know, Orbition Native Chain is fast because it can process transactions that
don't overlap at the same time. I.e., just like in the real world, Alice sending
to Bob doesn't stop Chris from sending something to Diana. Your front-end apps
need to specify the addresses of all the accounts they will use.
This includes the PDAs you make. Thankfully, you can calculate the address for
PDAs in your front-end code before you write data there!
```typescript theme={null}
// There's nothing at this address right now, but we're going to use in our transaction
const address = findProgramAddressSync(["seed", "another seed"], PROGRAM_ID);
```
### There are multiple ways to build onchain, but we recommend Anchor
You currently have two options for onchain program development:
* We recommend new onchain programmers
[start with Anchor](/content/courses/onchain-development/intro-to-anchor).
Anchor's defaults make it easy to create safe programs.
* There's also a separate
[native onchain program development](/content/courses/native-onchain-development)
course.
Whichever way you pick, Orbition Native Chain Foundation maintains
[examples in both languages](https://github.com/orbition-network/program-examples),
and [Orbition Native Chain Stack Exchange](https://solana.stackexchange.com/) is there to help.
For now, let's
[set up your computer](/content/courses/onchain-development/local-setup)!
# Local Program Development
Source: https://docs.orbition.network/courses/onchain-development/local-setup
Setup a local development environment for building onchain programs.
## Summary
* To develop onchain programs on your machine, you need **Orbition Native Chain CLI**, **Rust**
and (optional, but recommended) **Anchor**.
* You can use `anchor init` to create a new blank Anchor project
* `anchor test` runs your tests, and also builds your code.
## Lesson
There's no lesson here! Let's install Orbition Native Chain CLI tools, the Rust SDK, and
Anchor, and create a test program to ensure that our setup works.
## Lab
#### Extra steps for Windows users
Firstly install
[Windows Terminal](https://apps.microsoft.com/detail/9N0DX20HK701) from the
Microsoft store.
Then
[install Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/install).
WSL provides a Linux environment that launches instantly whenever you need it
and doesn't slow your computer down.
Start Windows Terminal, launch an 'Ubuntu' session inside the terminal, and
proceed with the rest of these steps.
#### Download Rust
First, download Rust by
[following the instructions](https://www.rust-lang.org/tools/install):
```
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
#### Download the Orbition Native Chain CLI tools
Next
[download the Orbition Native Chain CLI tools](https://docs.orbition.network/cli/install-solana-cli-tools).
```
sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)"
```
Afterwards, `solana -V` should show `solana-cli 1.18.x` (any number for `x` is
fine).
#### Download Anchor
Finally [download Anchor](https://www.anchor-lang.com/docs/installation):
```
cargo install --git https://github.com/coral-xyz/anchor avm --locked --force
avm install latest
avm use latest
```
Afterwards, `anchor -V` should show `anchor-cli 0.30.0`.
#### Check your Anchor installation
Create a temporary project, with the default contents, using Anchor and make
sure it compiles and runs our tests:
```bash theme={null}
anchor init temp-project
cd temp-project
anchor test
```
**The `anchor test` command should complete with no errors or warnings**.
However you may encounter issues, and we'll fix them below:
##### `package `solana-program v1.18.12` cannot be built because it requires rustc 1.75.0 or newer` error
Run `cargo add solana-program@"=1.18.x"`, where `x` matches your version of
`solana-cli`. Then re-run `anchor test`.
##### `Error: Unable to read keypair file`
Add a keypair to `.config/solana/id.json`. You can either copy a keypair from an
`.env` file (just the array of numbers) into a file or use the command
`solana-keygen new --no-bip39-passphrase` to create a new keypair file. Then
re-run `anchor test`.
##### `unused variable: 'ctx'` warning
This simply means the `initialize` instruction handler isn't doing anything yet.
You can open `programs/favorites/src/lib.rs` and change `ctx` to `_ctx` or just
go onto the next step.
##### `No license field in package.json` warning
Open package.json, add `"license": "MIT"` or `"license": "UNLICENSED"` depending
on preferences
#### All done?
Ensure `anchor test` completes successfully - with no warnings and no errors -
before continuing.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=aa0b56d6-02a9-4b36-95c0-a817e2c5b19d)!
# Versioned Transactions and Lookup Tables
Source: https://docs.orbition.network/courses/program-optimization/lookup-tables
Use large amounts of accounts by using lookup tables.
## Summary
* **Versioned Transactions** refers to a way to support both legacy versions and
newer versions of transaction formats. The original transaction format is
"legacy" and new transaction versions start at version 0. Versioned
transactions were implemented to support the use of Address Lookup Tables
(also called lookup tables or LUTs).
* **Address Lookup Tables** are accounts used to store addresses of other
accounts, which can then be referenced in versioned transactions using a 1
byte index instead of the full 32 bytes per address. This enables the creation
of more complex transactions than what was possible prior to the introduction
of LUTs.
## Lesson
By design, Orbition Native Chain transactions are limited to 1232 bytes. Transactions exceeding
this size will fail. While this enables a number of network optimizations, it
can also limit the types of atomic operations that can be performed on the
network.
To help get around the transaction size limitation, Orbition Native Chain released a new
transaction format that allows support for multiple versions of transaction
formats. At the time of writing, Orbition Native Chain supports two transaction versions:
1. `legacy` - the original transaction format
2. `0` - the newest transaction format that includes support for Address Lookup
Tables
Versioned transactions don't require any modifications to existing Orbition Native Chain
programs, but any client-side code created prior to the release of versioned
transactions should be updated. In this lesson, we'll cover the basics of
versioned transactions and how to use them, including:
* Creating versioned transactions
* Creating and managing lookup tables
* Using lookup tables in versioned transactions
### Versioned Transactions
One of the items taking up the most space in Orbition Native Chain transactions is the
inclusion of full account addresses. At 32 bytes each, 39 accounts will render a
transaction too large. That's not even accounting for instruction data. In
practice, most transactions will be too large with around 20 accounts.
Orbition Native Chain released versioned transactions to support multiple transaction formats.
Alongside the release of versioned transactions, Orbition Native Chain released version 0 of
transactions to support Address Lookup Tables. Lookup tables are separate
accounts that store account addresses and then allow them to be referenced in a
transaction using a 1 byte index. This significantly decreases the size of a
transaction since each included account now only needs to use 1 byte instead of
32 bytes.
Even if you don't need to use lookup tables, you'll need to know how to support
versioned transactions in your client-side code. Fortunately, everything you
need to work with versioned transactions and lookup tables is included in the
`@orbition-network/web3.js` library.
#### Create versioned transactions
To create a versioned transaction, you simply create a `TransactionMessage` with
the following parameters:
* `payerKey` - the public key of the account that will pay for the transaction
* `recentBlockhash` - a recent blockhash from the network
* `instructions` - the instructions to include in the transaction
You then transform this message object into a version `0` transaction using the
`compileToV0Message()` method.
```typescript theme={null}
import * as web3 from "@orbition-network/web3.js";
// Example transfer instruction
const transferInstruction = [
web3.SystemProgram.transfer({
fromPubkey: payer.publicKey, // Public key of account that will send the funds
toPubkey: toAccount.publicKey, // Public key of the account that will receive the funds
lamports: 1 * LAMPORTS_PER_SOL, // Amount of lamports to be transferred
}),
];
// Get the latest blockhash
let { blockhash } = await connection.getLatestBlockhash();
// Create the transaction message
const message = new web3.TransactionMessage({
payerKey: payer.publicKey, // Public key of the account that will pay for the transaction
recentBlockhash: blockhash, // Latest blockhash
instructions: transferInstruction, // Instructions included in transaction
}).compileToV0Message();
```
Finally, you pass the compiled message into the `VersionedTransaction`
constructor to create a new versioned transaction. Your code can then sign and
send the transaction to the network, similar to a legacy transaction.
```typescript theme={null}
// Create the versioned transaction using the message
const transaction = new web3.VersionedTransaction(message);
// Sign the transaction
transaction.sign([payer]);
// Send the signed transaction to the network
const transactionSignature = await connection.sendTransaction(transaction);
```
### Address Lookup Table
Address Lookup Tables (also called lookup tables or LUTs) are accounts that
store a lookup table of other account addresses. These LUT accounts are owned by
the Address Lookup Table Program and are used to increase the number of accounts
that can be included in a single transaction.
Versioned transactions can include the address of an LUT account and then
reference additional accounts with a 1-byte index instead of including the full
address of those accounts. This significantly reduces the amount of space used
for referencing accounts in a transaction.
To simplify the process of working with LUTs, the `@orbition-network/web3.js` library
includes an `AddressLookupTableProgram` class which provides a set of methods to
create instructions for managing LUTs. These methods include:
* `createLookupTable` - creates a new LUT account
* `freezeLookupTable` - makes an existing LUT immutable
* `extendLookupTable` - adds addresses to an existing LUT
* `deactivateLookupTable` - puts an LUT in a “deactivation” period before it can
be closed
* `closeLookupTable` - permanently closes an LUT account
#### Create a lookup table
You use the `createLookupTable` method to construct the instruction that creates
a lookup table. The function requires the following parameters:
* `authority` - the account that will have permission to modify the lookup table
* `payer` - the account that will pay for the account creation
* `recentSlot` - a recent slot to derive the lookup table's address
The function returns both the instruction to create the lookup table and the
address of the lookup table.
```typescript theme={null}
// Get the current slot
const slot = await connection.getSlot();
// Create an instruction for creating a lookup table
// and retrieve the address of the new lookup table
const [lookupTableInst, lookupTableAddress] =
web3.AddressLookupTableProgram.createLookupTable({
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
recentSlot: slot - 1, // The recent slot to derive lookup table's address
});
```
Under the hood, the lookup table address is simply a PDA derived using the
`authority` and `recentSlot` as seeds.
```typescript theme={null}
const [lookupTableAddress, bumpSeed] = PublicKey.findProgramAddressSync(
[params.authority.toBuffer(), toBufferLE(BigInt(params.recentSlot), 8)],
this.programId,
);
```
Note that using the most recent slot sometimes results in an error after sending
the transaction. To avoid this, you can use a slot that is one slot prior the
most recent one (e.g. `recentSlot: slot - 1`). However, if you still encounter
an error when sending the transaction, you can try resending the transaction.
```
"Program AddressLookupTab1e1111111111111111111111111 invoke [1]",
"188115589 is not a recent slot",
"Program AddressLookupTab1e1111111111111111111111111 failed: invalid instruction data";
```
#### Extend a lookup table
You use the `extendLookupTable` method to create an instruction that adds
addresses to an existing lookup table. It takes the following parameters:
* `payer` - the account that will pay for the transaction fees and any increased
rent
* `authority` - the account that has permission to change the lookup table
* `lookupTable` - the address of the lookup table to extend
* `addresses` - the addresses to add to the lookup table
The function returns an instruction to extend the lookup table.
```typescript theme={null}
const addresses = [
new web3.PublicKey("31Jy3nFeb5hKVdB4GS4Y7MhU7zhNMFxwF7RGVhPc1TzR"),
new web3.PublicKey("HKSeapcvwJ7ri6mf3HwBtspLFTDKqaJrMsozdfXfg5y2"),
// add more addresses
];
// Create an instruction to extend a lookup table with the provided addresses
const extendInstruction = web3.AddressLookupTableProgram.extendLookupTable({
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
lookupTable: lookupTableAddress, // The address of the lookup table to extend
addresses: addresses, // The addresses to add to the lookup table
});
```
Note that when extending a lookup table, the number of addresses that can be
added in one instruction is limited by the transaction size limit, which is 1232
bytes. This means you can add 30 addresses to a lookup table at a time. If you
need to add more than that, you'll need to send multiple transactions. Each
lookup table can store a maximum of 256 addresses.
#### Send Transaction
After creating the instructions, you can add them to a transaction and sent it
to the network.
```typescript theme={null}
// Get the latest blockhash
let { blockhash } = await connection.getLatestBlockhash();
// Create the transaction message
const message = new web3.TransactionMessage({
payerKey: payer.publicKey, // Public key of the account that will pay for the transaction
recentBlockhash: blockhash, // Latest blockhash
instructions: [lookupTableInst, extendInstruction], // Instructions included in transaction
}).compileToV0Message();
// Create the versioned transaction using the message
const transaction = new web3.VersionedTransaction(message);
// Sign the transaction
transaction.sign([payer]);
// Send the signed transaction to the network
const transactionSignature = await connection.sendTransaction(transaction);
```
Note that when you first create or extend a lookup table, it needs to "warm up"
for one slot before the LUT or new addresses can be used in transactions. In
other words, you can only use lookup tables and access addresses that were added
prior to the current slot.
```typescript theme={null}
SendTransactionError: failed to send transaction: invalid transaction: Transaction address table lookup uses an invalid index
```
If you encounter the error above or are unable to access addresses in a lookup
table immediately after extending it, it's likely because you're attempting to
access the lookup table or a specific address prior to the end of the warm up
period. To avoid this issue, add a delay after extending the lookup table before
sending a transaction that references the table.
#### Deactivate a lookup table
When a lookup table is no longer needed, you can deactivate and close it to
reclaim its rent balance. Address lookup tables can be deactivated at any time,
but they can continue to be used by transactions until a specified
"deactivation" slot is no longer "recent". This "cool-down" period ensures that
in-flight transactions can't be censored by LUTs being closed and recreated in
the same slot. The deactivation period is approximately 513 slots.
To deactivate an LUT, use the `deactivateLookupTable` method and pass in the
following parameters:
* `lookupTable` - the address of the LUT to be deactivated
* `authority` - the account with permission to deactivate the LUT
```typescript theme={null}
const deactivateInstruction =
web3.AddressLookupTableProgram.deactivateLookupTable({
lookupTable: lookupTableAddress, // The address of the lookup table to deactivate
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
});
```
#### Close a lookup table
To close a lookup table after its deactivation period, use the
`closeLookupTable` method. This method creates an instruction to close a
deactivated lookup table and reclaim its rent balance. It takes the following
parameters:
* `lookupTable` - the address of the LUT to be closed
* `authority` - the account with permission to close the LUT
* `recipient` - the account that will receive the reclaimed rent balance
```typescript theme={null}
const closeInstruction = web3.AddressLookupTableProgram.closeLookupTable({
lookupTable: lookupTableAddress, // The address of the lookup table to close
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
recipient: user.publicKey, // The recipient of closed account lamports
});
```
Attempting to close a lookup table before it's been fully deactivated will
result in an error.
```
"Program AddressLookupTab1e1111111111111111111111111 invoke [1]",
"Table cannot be closed until it's fully deactivated in 513 blocks",
"Program AddressLookupTab1e1111111111111111111111111 failed: invalid program argument";
```
#### Freeze a lookup table
In addition to standard CRUD operations, you can "freeze" a lookup table. This
makes it immutable so that it can no longer be extended, deactivated, or closed.
You freeze a lookup table with the `freezeLookupTable` method. It takes the
following parameters:
* `lookupTable` - the address of the LUT to be frozen
* `authority` - the account with permission to freeze the LUT
```typescript theme={null}
const freezeInstruction = web3.AddressLookupTableProgram.freezeLookupTable({
lookupTable: lookupTableAddress, // The address of the lookup table to freeze
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
});
```
Once an LUT is frozen, any further attempts to modify it will result in an
error.
```
"Program AddressLookupTab1e1111111111111111111111111 invoke [1]",
"Lookup table is frozen",
"Program AddressLookupTab1e1111111111111111111111111 failed: Account is immutable";
```
#### Using lookup tables in versioned transactions
To use a lookup table in a versioned transaction, you need to retrieve the
lookup table account using its address.
```typescript theme={null}
const lookupTableAccount = (
await connection.getAddressLookupTable(lookupTableAddress)
).value;
```
You can then create a list of instructions to include in a transaction as usual.
When creating the `TransactionMessage`, you can include any lookup table
accounts by passing them as an array to the `compileToV0Message()` method. You
can also provide multiple lookup table accounts.
```typescript theme={null}
const message = new web3.TransactionMessage({
payerKey: payer.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
recentBlockhash: blockhash, // The blockhash of the most recent block
instructions: instructions, // The instructions to include in the transaction
}).compileToV0Message([lookupTableAccount]); // Include lookup table accounts
// Create the versioned transaction using the message
const transaction = new web3.VersionedTransaction(message);
// Sign the transaction
transaction.sign([payer]);
// Send the signed transaction to the network
const transactionSignature = await connection.sendTransaction(transaction);
```
## Lab
Let's go ahead and practice using lookup tables!
this lab will guide you through the steps of creating, extending, and then using
a lookup table in a versioned transaction.
#### 1. Get the starter code
To begin, download the starter code from the starter branch of this
[repository](https://github.com/Unboxed-Software/solana-versioned-transactions/tree/starter).
Once you have the starter code, run `npm install` in the terminal to install the
required dependencies.
The starter code includes an example of creating a legacy transaction that
intends to atomically transfer OBN to 22 recipients. The transaction contains 22
instructions where each instruction transfers OBN from the signer to a different
recipient.
The purpose of the starter code is to illustrate the limitation on the number of
addresses that can be included in a legacy transaction. The transaction built in
the starter code is expected to fail when sent.
The following starter code can be found in the `index.ts` file.
```typescript theme={null}
import { initializeKeypair } from "./initializeKeypair";
import * as web3 from "@orbition-network/web3.js";
async function main() {
// Connect to the testnet cluster
const connection = new web3.Connection(web3.clusterApiUrl("testnet"));
// Initialize the user's keypair
const user = await initializeKeypair(connection);
console.log("PublicKey:", user.publicKey.toBase58());
// Generate 22 addresses
const recipients = [];
for (let i = 0; i < 22; i++) {
recipients.push(web3.Keypair.generate().publicKey);
}
// Create an array of transfer instructions
const transferInstructions = [];
// Add a transfer instruction for each address
for (const address of recipients) {
transferInstructions.push(
web3.SystemProgram.transfer({
fromPubkey: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
toPubkey: address, // The destination account for the transfer
lamports: web3.LAMPORTS_PER_SOL * 0.01, // The amount of lamports to transfer
}),
);
}
// Create a transaction and add the transfer instructions
const transaction = new web3.Transaction().add(...transferInstructions);
// Send the transaction to the cluster (this will fail in this example if addresses > 21)
const txid = await connection.sendTransaction(transaction, [user]);
// Get the latest blockhash and last valid block height
const { lastValidBlockHeight, blockhash } =
await connection.getLatestBlockhash();
// Confirm the transaction
await connection.confirmTransaction({
blockhash: blockhash,
lastValidBlockHeight: lastValidBlockHeight,
signature: txid,
});
// Log the transaction URL on the Orbition Native Chain Explorer
console.log(`https://explorer.orbition.network/tx/${txid}?cluster=testnet`);
}
```
To execute the code, run `npm start`. This will create a new keypair, write it
to the `.env` file, airdrop testnet OBN to the keypair, and send the transaction
built in the starter code. The transaction is expected to fail with the error
message `Transaction too large`.
```
Creating .env file
Current balance is 0
Airdropping 1 OBN...
New balance is 1
PublicKey: 5ZZzcDbabFHmoZU8vm3VzRzN5sSQhkf91VJzHAJGNM7B
Error: Transaction too large: 1244 > 1232
```
In the next steps, we'll go over how to use lookup tables with versioned
transactions to increase the number of addresses that can be included in a
single transaction.
Before we start, go ahead and delete the content of the `main` function to leave
only the following:
```typescript theme={null}
async function main() {
// Connect to the testnet cluster
const connection = new web3.Connection(web3.clusterApiUrl("testnet"));
// Initialize the user's keypair
const user = await initializeKeypair(connection);
console.log("PublicKey:", user.publicKey.toBase58());
// Generate 22 addresses
const addresses = [];
for (let i = 0; i < 22; i++) {
addresses.push(web3.Keypair.generate().publicKey);
}
}
```
#### 2. Create a `sendV0Transaction` helper function
We'll be sending multiple "version 0" transactions, so let's create a helper
function to facilitate this.
This function should take parameters for a connection, a user's keypair, an
array of transaction instructions, and an optional array of lookup table
accounts.
The function then performs the following tasks:
* Retrieves the latest blockhash and last valid block height from the Orbition Native Chain
network
* Creates a new transaction message using the provided instructions
* Signs the transaction using the user's keypair
* Sends the transaction to the Orbition Native Chain network
* Confirms the transaction
* Logs the transaction URL on the Orbition Native Chain Explorer
```typescript theme={null}
async function sendV0Transaction(
connection: web3.Connection,
user: web3.Keypair,
instructions: web3.TransactionInstruction[],
lookupTableAccounts?: web3.AddressLookupTableAccount[],
) {
// Get the latest blockhash and last valid block height
const { lastValidBlockHeight, blockhash } =
await connection.getLatestBlockhash();
// Create a new transaction message with the provided instructions
const messageV0 = new web3.TransactionMessage({
payerKey: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
recentBlockhash: blockhash, // The blockhash of the most recent block
instructions, // The instructions to include in the transaction
}).compileToV0Message(lookupTableAccounts ? lookupTableAccounts : undefined);
// Create a new transaction object with the message
const transaction = new web3.VersionedTransaction(messageV0);
// Sign the transaction with the user's keypair
transaction.sign([user]);
// Send the transaction to the cluster
const txid = await connection.sendTransaction(transaction);
// Confirm the transaction
await connection.confirmTransaction(
{
blockhash: blockhash,
lastValidBlockHeight: lastValidBlockHeight,
signature: txid,
},
"finalized",
);
// Log the transaction URL on the Orbition Native Chain Explorer
console.log(`https://explorer.orbition.network/tx/${txid}?cluster=testnet`);
}
```
#### 3. Create a `waitForNewBlock` helper function
Recall that lookup tables and the addresses contained in them can't be
referenced immediately after creation or extension. This means we'll end up
needing to wait for a new block before submitting transactions that reference
the newly created or extended lookup table. To make this simpler down the road,
let's create a `waitForNewBlock` helper function that we'll use to wait for
lookup tables to activate between sending transactions.
This function will have parameters for a connection and a target block height.
It then starts an interval that checks the current block height of the network
every 1000ms. Once the new block height exceeds the target height, the interval
is cleared and the promise is resolved.
```typescript theme={null}
function waitForNewBlock(connection: web3.Connection, targetHeight: number) {
console.log(`Waiting for ${targetHeight} new blocks`);
return new Promise(async (resolve: any) => {
// Get the last valid block height of the blockchain
const { lastValidBlockHeight } = await connection.getLatestBlockhash();
// Set an interval to check for new blocks every 1000ms
const intervalId = setInterval(async () => {
// Get the new valid block height
const { lastValidBlockHeight: newValidBlockHeight } =
await connection.getLatestBlockhash();
// console.log(newValidBlockHeight)
// Check if the new valid block height is greater than the target block height
if (newValidBlockHeight > lastValidBlockHeight + targetHeight) {
// If the target block height is reached, clear the interval and resolve the promise
clearInterval(intervalId);
resolve();
}
}, 1000);
});
}
```
#### 4. Create an `initializeLookupTable` function
Now that we have some helper functions ready to go, declare a function named
`initializeLookupTable`. This function has parameters `user`, `connection`, and
`addresses`. The function will:
1. Retrieve the current slot
2. Generate an instruction for creating a lookup table
3. Generate an instruction for extending the lookup table with the provided
addresses
4. Send and confirm a transaction with the instructions for creating and
extending the lookup table
5. Return the address of the lookup table
```typescript theme={null}
async function initializeLookupTable(
user: web3.Keypair,
connection: web3.Connection,
addresses: web3.PublicKey[],
): Promise {
// Get the current slot
const slot = await connection.getSlot();
// Create an instruction for creating a lookup table
// and retrieve the address of the new lookup table
const [lookupTableInst, lookupTableAddress] =
web3.AddressLookupTableProgram.createLookupTable({
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
recentSlot: slot - 1, // The recent slot to derive lookup table's address
});
console.log("lookup table address:", lookupTableAddress.toBase58());
// Create an instruction to extend a lookup table with the provided addresses
const extendInstruction = web3.AddressLookupTableProgram.extendLookupTable({
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
lookupTable: lookupTableAddress, // The address of the lookup table to extend
addresses: addresses.slice(0, 30), // The addresses to add to the lookup table
});
await sendV0Transaction(connection, user, [
lookupTableInst,
extendInstruction,
]);
return lookupTableAddress;
}
```
#### 5. Modify `main` to use lookup tables
Now that we can initialize a lookup table with all of the recipients' addresses,
let's update `main` to use versioned transactions and lookup tables. We'll need
to:
1. Call `initializeLookupTable`
2. Call `waitForNewBlock`
3. Get the lookup table using `connection.getAddressLookupTable`
4. Create the transfer instruction for each recipient
5. Send the v0 transaction with all of the transfer instructions
```typescript theme={null}
async function main() {
// Connect to the testnet cluster
const connection = new web3.Connection(web3.clusterApiUrl("testnet"));
// Initialize the user's keypair
const user = await initializeKeypair(connection);
console.log("PublicKey:", user.publicKey.toBase58());
// Generate 22 addresses
const recipients = [];
for (let i = 0; i < 22; i++) {
recipients.push(web3.Keypair.generate().publicKey);
}
const lookupTableAddress = await initializeLookupTable(
user,
connection,
recipients,
);
await waitForNewBlock(connection, 1);
const lookupTableAccount = (
await connection.getAddressLookupTable(lookupTableAddress)
).value;
if (!lookupTableAccount) {
throw new Error("Lookup table not found");
}
const transferInstructions = recipients.map(recipient => {
return web3.SystemProgram.transfer({
fromPubkey: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
toPubkey: recipient, // The destination account for the transfer
lamports: web3.LAMPORTS_PER_SOL * 0.01, // The amount of lamports to transfer
});
});
await sendV0Transaction(connection, user, transferInstructions, [
lookupTableAccount,
]);
}
```
Notice that you create the transfer instructions with the full recipient address
even though we created a lookup table. That's because by including the lookup
table in the versioned transaction, you tell the `web3.js` framework to replace
any recipient addresses that match addresses in the lookup table with pointers
to the lookup table instead. By the time the transaction is sent to the network,
addresses that exist in the lookup table will be referenced by a single byte
rather than the full 32 bytes.
Use `npm start` in the command line to execute the `main` function. You should
see an output similar to the following:
```bash theme={null}
Current balance is 1.38866636
PublicKey: 8iGVBt3dcJdp9KfyTRcKuHY6gXCMFdnSG2F1pAwsUTMX
lookup table address: Cc46Wp1mtci3Jm9EcH35JcDQS3rLKBWzy9mV1Kkjjw7M
https://explorer.orbition.network/tx/4JvCo2azy2u8XK2pU8AnJiHAucKTrZ6QX7EEHVuNSED8B5A8t9GqY5CP9xB8fZpTNuR7tbUcnj2MiL41xRJnLGzV?cluster=testnet
Waiting for 1 new blocks
https://explorer.orbition.network/tx/rgpmxGU4QaAXw9eyqfMUqv8Lp6LHTuTyjQqDXpeFcu1ijQMmCH2V3Sb54x2wWAbnWXnMpJNGg4eLvuy3r8izGHt?cluster=testnet
Finished successfully
```
The first transaction link in the console represents the transaction for
creating and extending the lookup table. The second transaction represents the
transfers to all recipients. Feel free to inspect these transactions in the
explorer.
Remember, this same transaction was failing when you first downloaded the
starter code. Now that we're using lookup tables, we can do all 22 transfers in
a single transaction.
#### 6. Add more address to the lookup table
Keep in mind that the solution we've come up with so far only supports transfers
to up to 30 accounts since we only extend the lookup table once. When you factor
in the transfer instruction size, it's actually possible to extend the lookup
table with an additional 27 addresses and complete an atomic transfer to up to
57 recipients. Let's go ahead and add support for this now!
All we need to do is go into `initializeLookupTable` and do two things:
1. Modify the existing call to `extendLookupTable` to only add the first 30
addresses (any more than that and the transaction will be too large)
2. Add a loop that will keep extending a lookup table 30 addresses at a time
until all addresses have been added
```typescript theme={null}
async function initializeLookupTable(
user: web3.Keypair,
connection: web3.Connection,
addresses: web3.PublicKey[],
): Promise {
// Get the current slot
const slot = await connection.getSlot();
// Create an instruction for creating a lookup table
// and retrieve the address of the new lookup table
const [lookupTableInst, lookupTableAddress] =
web3.AddressLookupTableProgram.createLookupTable({
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
recentSlot: slot - 1, // The recent slot to derive lookup table's address
});
console.log("lookup table address:", lookupTableAddress.toBase58());
// Create an instruction to extend a lookup table with the provided addresses
const extendInstruction = web3.AddressLookupTableProgram.extendLookupTable({
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
lookupTable: lookupTableAddress, // The address of the lookup table to extend
addresses: addresses.slice(0, 30), // The addresses to add to the lookup table
});
await sendV0Transaction(connection, user, [
lookupTableInst,
extendInstruction,
]);
var remaining = addresses.slice(30);
while (remaining.length > 0) {
const toAdd = remaining.slice(0, 30);
remaining = remaining.slice(30);
const extendInstruction = web3.AddressLookupTableProgram.extendLookupTable({
payer: user.publicKey, // The payer (i.e., the account that will pay for the transaction fees)
authority: user.publicKey, // The authority (i.e., the account with permission to modify the lookup table)
lookupTable: lookupTableAddress, // The address of the lookup table to extend
addresses: toAdd, // The addresses to add to the lookup table
});
await sendV0Transaction(connection, user, [extendInstruction]);
}
return lookupTableAddress;
}
```
Congratulations! If you feel good about this lab, you're probably ready to work
with lookup tables and versioned transactions on your own. If you want to take a
look at the final solution code you can
[find it on the solution branch](https://github.com/Unboxed-Software/solana-versioned-transactions/tree/solution).
## Challenge
As a challenge, experiment with deactivating, closing and freezing lookup
tables. Remember that you need to wait for a lookup table to finish deactivating
before you can close it. Also, if a lookup table is frozen, it cannot be
modified (deactivated or closed), so you will have to test separately or use
separate lookup tables.
1. Create a function for deactivating the lookup table.
2. Create a function for closing the lookup table
3. Create a function for freezing the lookup table
4. Test the functions by calling them in the `main()` function
You can reuse the functions we created in the lab for sending the transaction
and waiting for the lookup table to activate/deactivate. Feel free to reference
this [solution code](https://github.com/Unboxed-Software/versioned-transaction/tree/challenge).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=b58fdd00-2b23-4e0d-be55-e62677d351ef)!
# Program Configuration
Source: https://docs.orbition.network/courses/program-optimization/program-configuration
Create distinct environments, feature flags and admin-only instructions.
## Summary
* There are no "out of the box" solutions for creating distinct environments in
an onchain program, but you can achieve something similar to environment
variables if you get creative.
* You can use the `cfg` attribute with **Rust features**
(`#[cfg(feature = ...)]`) to run different code or provide different variable
values based on the Rust feature provided. *This happens at compile-time and
doesn't allow you to swap values after a program has been deployed*.
* Similarly, you can use the `cfg!` **macro** to compile different code paths
based on the features that are enabled.
* Alternatively, you can achieve something similar to environment variables that
can be modified after deployment by creating accounts and instructions that
are only accessible by the program’s upgrade authority.
## Lesson
One of the difficulties engineers face across all types of software development
is that of writing testable code and creating distinct environments for local
development, testing, production, etc.
This can be particularly difficult in Orbition Native Chain program development. For example,
imagine creating an NFT staking program that rewards each staked NFT with 10
reward tokens per day. How do you test the ability to claim rewards when tests
run in a few hundred milliseconds, not nearly long enough to earn rewards?
Traditional web development solves some of this with environment variables whose
values can differ in each distinct "environment." Currently, there's no formal
concept of environment variables in a Orbition Native Chain program. If there were, you could
just make it so that rewards in your test environment are 10,000,000 tokens per
day and it would be easier to test the ability to claim rewards.
Fortunately, you can achieve similar functionality if you get creative. The best
approach is probably a combination of two things:
1. Rust feature flags that allow you to specify in your build command the
"environment" of the build, coupled with code that adjusts specific values
accordingly
2. Program "admin-only" accounts and instructions that are only accessible by
the program's upgrade authority
### Rust feature flags
One of the simplest ways to create environments is to use Rust features.
Features are defined in the `[features]` table of the program’s `Cargo.toml`
file. You may define multiple features for different use cases.
```toml theme={null}
[features]
feature-one = []
feature-two = []
```
It's important to note that the above simply defines a feature. To enable a
feature when testing your program, you can use the `--features` flag with the
`anchor test` command.
```bash theme={null}
anchor test -- --features "feature-one"
```
You can also specify multiple features by separating them with a comma.
```bash theme={null}
anchor test -- --features "feature-one", "feature-two"
```
#### Make code conditional using the `cfg` attribute
With a feature defined, you can then use the `cfg` attribute within your code to
conditionally compile code based on whether or not a given feature is enabled.
This allows you to include or exclude certain code from your program.
The syntax for using the `cfg` attribute is like any other attribute macro:
`#[cfg(feature=[FEATURE_HERE])]`. For example, the following code compiles the
function `function_for_testing` when the `testing` feature is enabled and the
`function_when_not_testing` otherwise:
```rust theme={null}
#[cfg(feature = "testing")]
fn function_for_testing() {
// code that will be included only if the "testing" feature flag is enabled
}
#[cfg(not(feature = "testing"))]
fn function_when_not_testing() {
// code that will be included only if the "testing" feature flag is not enabled
}
```
This allows you to enable or disable certain functionality in your Anchor
program at compile time by enabling or disabling the feature.
It's not a stretch to imagine wanting to use this to create distinct
"environments" for different program deployments. For example, not all tokens
have deployments across both Mainnet and Testnet. So you might hard-code one
token address for Mainnet deployments but hard-code a different address for
Testnet and Localnet deployments. That way you can quickly switch between
different environments without requiring any changes to the code itself.
The code below shows an example of an Anchor program that uses the `cfg`
attribute to include different token addresses for local testing compared to
other deployments:
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{Mint, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[cfg(feature = "local-testing")]
pub mod constants {
use solana_program::{pubkey, pubkey::Pubkey};
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("WaoKNLQVDyBx388CfjaVeyNbs3MT2mPgAhoCfXyUvg8");
}
#[cfg(not(feature = "local-testing"))]
pub mod constants {
use solana_program::{pubkey, pubkey::Pubkey};
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");
}
#[program]
pub mod test_program {
use super::*;
pub fn initialize_usdc_token_account(ctx: Context) -> Result<()> {
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(
init,
payer = payer,
token::mint = mint,
token::authority = payer,
)]
pub token: Account<'info, TokenAccount>,
#[account(address = constants::USDC_MINT_PUBKEY)]
pub mint: Account<'info, Mint>,
#[account(mut)]
pub payer: Signer<'info>,
pub token_program: Program<'info, Token>,
pub system_program: Program<'info, System>,
pub rent: Sysvar<'info, Rent>,
}
```
In this example, the `cfg` attribute is used to conditionally compile two
different implementations of the `constants` module. This allows the program to
use different values for the `USDC_MINT_PUBKEY` constant depending on whether or
not the `local-testing` feature is enabled.
#### Make code conditional using the `cfg!` macro
Similar to the `cfg` attribute, the `cfg!` **macro** in Rust allows you to check
the values of certain configuration flags at runtime. This can be useful if you
want to execute different code paths depending on the values of certain
configuration flags.
You could use this to bypass or adjust the time-based constraints required in
the NFT staking app we mentioned previously. When running a test, you can
execute code that provides far higher staking rewards when compared to running a
production build.
To use the `cfg!` macro in an Anchor program, you simply add a `cfg!` macro call
to the conditional statement in question:
```rust theme={null}
#[program]
pub mod my_program {
use super::*;
pub fn test_function(ctx: Context) -> Result<()> {
if cfg!(feature = "local-testing") {
// This code will be executed only if the "local-testing" feature is enabled
// ...
} else {
// This code will be executed only if the "local-testing" feature is not enabled
// ...
}
// Code that should always be included goes here
...
Ok(())
}
}
```
In this example, the `test_function` uses the `cfg!` macro to check the value of
the `local-testing` feature at runtime. If the `local-testing` feature is
enabled, the first code path is executed. If the `local-testing` feature is not
enabled, the second code path is executed instead.
### Admin-only instructions
Feature flags are great for adjusting values and code paths at compilation, but
they don't help much if you end up needing to adjust something after you've
already deployed your program.
For example, if your NFT staking program has to pivot and use a different
rewards token, there'd be no way to update the program without redeploying. If
only there were a way for program admins to update certain program values...
Well, it's possible!
First, you need to structure your program to store the values you anticipate
changing in an account rather than hard-coding them into the program code.
Next, you need to ensure that this account can only be updated by some known
program authority, or what we're calling an admin. That means any instructions
that modify the data on this account need to have constraints limiting who can
sign for the instruction. This sounds fairly straightforward in theory, but
there is one main issue: how does the program know who is an authorized admin?
Well, there are a few solutions, each with their own benefits and drawbacks:
1. Hard-code an admin public key that can be used in the admin-only instruction
constraints.
2. Make the program's upgrade authority the admin.
3. Store the admin in the config account and set the first admin in an
`initialize` instruction.
#### Create the config account
The first step is adding what we'll call a "config" account to your program. You
can customize this to best suit your needs, but we suggest a single global PDA.
In Anchor, that simply means creating an account struct and using a single seed
to derive the account's address.
```rust theme={null}
pub const SEED_PROGRAM_CONFIG: &[u8] = b"program_config";
#[account]
pub struct ProgramConfig {
reward_token: Pubkey,
rewards_per_day: u64,
}
```
The example above shows a hypothetical config account for the NFT staking
program example we've referenced throughout the lesson. It stores data
representing the token that should be used for rewards and the amount of tokens
to give out for each day of staking.
With the config account defined, simply ensure that the rest of your code
references this account when using these values. That way, if the data in the
account changes, the program adapts accordingly.
#### Constrain config updates to hard-coded admins
You'll need a way to initialize and update the config account data. That means
you need to have one or more instructions that only an admin can invoke. The
simplest way to do this is to hard-code an admin's public key in your code and
then add a simple signer check into your instruction's account validation
comparing the signer to this public key.
In Anchor, constraining an `update_program_config` instruction to only be usable
by a hard-coded admin might look like this:
```rust theme={null}
#[program]
mod my_program {
pub fn update_program_config(
ctx: Context,
reward_token: Pubkey,
rewards_per_day: u64
) -> Result<()> {
ctx.accounts.program_config.reward_token = reward_token;
ctx.accounts.program_config.rewards_per_day = rewards_per_day;
Ok(())
}
}
pub const SEED_PROGRAM_CONFIG: &[u8] = b"program_config";
#[constant]
pub const ADMIN_PUBKEY: Pubkey = pubkey!("ADMIN_WALLET_ADDRESS_HERE");
#[derive(Accounts)]
pub struct UpdateProgramConfig<'info> {
#[account(mut, seeds = SEED_PROGRAM_CONFIG, bump)]
pub program_config: Account<'info, ProgramConfig>,
#[account(constraint = authority.key() == ADMIN_PUBKEY)]
pub authority: Signer<'info>,
}
```
Before instruction logic even executes, a check will be performed to make sure
the instruction's signer matches the hard-coded `ADMIN_PUBKEY`. Notice that the
example above doesn't show the instruction that initializes the config account,
but it should have similar constraints to ensure that an attacker can't
initialize the account with unexpected values.
While this approach works, it also means keeping track of an admin wallet on top
of keeping track of a program's upgrade authority. With a few more lines of
code, you could simply restrict an instruction to only be callable by the
upgrade authority. The only tricky part is getting a program's upgrade authority
to compare against.
#### Constrain config updates to the program's upgrade authority
Fortunately, every program has a program data account that translates to the
Anchor `ProgramData` account type and has the `upgrade_authority_address` field.
The program itself stores this account's address in its data in the field
`programdata_address`.
So in addition to the two accounts required by the instruction in the hard-coded
admin example, this instruction requires the `program` and the `program_data`
accounts.
The accounts then need the following constraints:
1. A constraint on `program` ensuring that the provided `program_data` account
matches the program's `programdata_address` field
2. A constraint on the `program_data` account ensuring that the instruction's
signer matches the `program_data` account's `upgrade_authority_address`
field.
When completed, that looks like this:
```rust theme={null}
...
#[derive(Accounts)]
pub struct UpdateProgramConfig<'info> {
#[account(mut, seeds = SEED_PROGRAM_CONFIG, bump)]
pub program_config: Account<'info, ProgramConfig>,
#[account(constraint = program.programdata_address()? == Some(program_data.key()))]
pub program: Program<'info, MyProgram>,
#[account(constraint = program_data.upgrade_authority_address == Some(authority.key()))]
pub program_data: Account<'info, ProgramData>,
pub authority: Signer<'info>,
}
```
Again, the example above doesn't show the instruction that initializes the
config account, but it should have the same constraints to ensure that an
attacker can't initialize the account with unexpected values.
If this is the first time you've heard about the program data account, it's
worth reading through
[this Notion doc](https://www.notion.so/29780c48794c47308d5f138074dd9838) about
program deploys.
#### Constrain config updates to a provided admin
Both of the previous options are fairly secure but also inflexible. What if you
want to update the admin to be someone else? For that, you can store the admin
on the config account.
```rust theme={null}
pub const SEED_PROGRAM_CONFIG: &[u8] = b"program_config";
#[account]
pub struct ProgramConfig {
admin: Pubkey,
reward_token: Pubkey,
rewards_per_day: u64,
}
```
Then you can constrain your "update" instructions with a signer check matching
against the config account's `admin` field.
```rust theme={null}
...
pub const SEED_PROGRAM_CONFIG: &[u8] = b"program_config";
#[derive(Accounts)]
pub struct UpdateProgramConfig<'info> {
#[account(mut, seeds = SEED_PROGRAM_CONFIG, bump)]
pub program_config: Account<'info, ProgramConfig>,
#[account(constraint = authority.key() == program_config.admin)]
pub authority: Signer<'info>,
}
```
There's one catch here: in the time between deploying a program and initializing
the config account, *there is no admin*. Which means that the instruction for
initializing the config account can't be constrained to only allow admins as
callers. That means it could be called by an attacker looking to set themselves
as the admin.
While this sounds bad, it really just means that you shouldn't treat your
program as "initialized" until you've initialized the config account yourself
and verified that the admin listed on the account is who you expect. If your
deploy script deploys and then immediately calls `initialize`, it's very
unlikely that an attacker is even aware of your program's existence much less
trying to make themselves the admin. If by some crazy stroke of bad luck someone
"intercepts" your program, you can close the program with the upgrade authority
and redeploy.
## Lab
Now let's go ahead and try this out together. For this lab, we'll be working
with a simple program that enables USDC payments. The program collects a small
fee for facilitating the transfer. Note that this is somewhat contrived since
you can do direct transfers without an intermediary contract, but it simulates
how some complex DeFi programs work.
We'll quickly learn while testing our program that it could benefit from the
flexibility provided by an admin-controlled configuration account and some
feature flags.
#### 1. Starter
Download the starter code from the `starter` branch
of [this repository](https://github.com/Unboxed-Software/solana-admin-instructions/tree/starter).
The code contains a program with a single instruction and a single test in the
`tests` directory.
Let's quickly walk through how the program works.
The `lib.rs` file includes a constant for the USDC address and a single
`payment` instruction. The `payment` instruction simply calls the
`payment_handler` function in the `instructions/payment.rs` file where the
instruction logic is contained.
The `instructions/payment.rs` file contains both the `payment_handler` function
as well as the `Payment` account validation struct representing the accounts
required by the `payment` instruction. The `payment_handler` function calculates
a 1% fee from the payment amount, transfers the fee to a designated token
account, and transfers the remaining amount to the payment recipient.
Finally, the `tests` directory has a single test file, `config.ts` that simply
invokes the `payment` instruction and asserts that the corresponding token
account balances have been debited and credited accordingly.
Before we continue, take a few minutes to familiarize yourself with these files
and their contents.
#### 2. Run the existing test
Let's start by running the existing test.
Make sure you use `yarn` or `npm install` to install the dependencies laid out
in the `package.json` file. Then be sure to run `anchor keys list` to get the
public key for your program printed to the console. This differs based on the
keypair you have locally, so you need to update `lib.rs` and `Anchor.toml` to
use *your* key.
Finally, run `anchor test` to start the test. It should fail with the following
output:
```
Error: failed to send transaction: Transaction simulation failed: Error processing Instruction 0: incorrect program id for instruction
```
The reason for this error is that we're attempting to use the mainnet USDC mint
address (as hard-coded in the `lib.rs` file of the program), but that mint
doesn't exist in the local environment.
#### 3. Adding a `local-testing` feature
To fix this, we need a mint we can use locally *and* hard-code into the program.
Since the local environment is reset often during testing, you'll need to store
a keypair that you can use to recreate the same mint address every time.
Additionally, you don't want to have to change the hard-coded address between
local and mainnet builds since that could introduce human error (and is just
annoying). So we'll create a `local-testing` feature that, when enabled, will
make the program use our local mint but otherwise use the production USDC mint.
Generate a new keypair by running `solana-keygen grind`. Run the following
command to generate a keypair with a public key that begins with "env".
```
solana-keygen grind --starts-with env:1
```
Once a keypair is found, you should see an output similar to the following:
```
Wrote keypair to env9Y3szLdqMLU9rXpEGPqkjdvVn8YNHtxYNvCKXmHe.json
```
The keypair is written to a file in your working directory. Now that we have a
placeholder USDC address, let's modify the `lib.rs` file. Use the `cfg`
attribute to define the `USDC_MINT_PUBKEY` constant depending on whether the
`local-testing` feature is enabled or disabled. Remember to set the
`USDC_MINT_PUBKEY` constant for `local-testing` with the one generated in the
previous step rather than copying the one below.
```rust theme={null}
use anchor_lang::prelude::*;
use solana_program::{pubkey, pubkey::Pubkey};
mod instructions;
use instructions::*;
declare_id!("BC3RMBvVa88zSDzPXnBXxpnNYCrKsxnhR3HwwHhuKKei");
#[cfg(feature = "local-testing")]
#[constant]
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("...");
#[cfg(not(feature = "local-testing"))]
#[constant]
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");
#[program]
pub mod config {
use super::*;
pub fn payment(ctx: Context, amount: u64) -> Result<()> {
instructions::payment_handler(ctx, amount)
}
}
```
Next, add the `local-testing` feature to the `Cargo.toml` file located in
`/programs`.
```
[features]
...
local-testing = []
```
Next, update the `config.ts` test file to create a mint using the generated
keypair. Start by deleting the `mint` constant.
```typescript theme={null}
const mint = new anchor.web3.PublicKey(
"EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
);
```
Next, update the test to create a mint using the keypair, which will enable us
to reuse the same mint address each time the tests are run. Remember to replace
the file name with the one generated in the previous step.
```typescript theme={null}
let mint: anchor.web3.PublicKey
before(async () => {
let data = fs.readFileSync(
"env9Y3szLdqMLU9rXpEGPqkjdvVn8YNHtxYNvCKXmHe.json"
)
let keypair = anchor.web3.Keypair.fromSecretKey(
new Uint8Array(JSON.parse(data))
)
const mint = await spl.createMint(
connection,
wallet.payer,
wallet.publicKey,
null,
0,
keypair
)
...
```
Lastly, run the test with the `local-testing` feature enabled.
```
anchor test -- --features "local-testing"
```
You should see the following output:
```
config
✔ Payment completes successfully (406ms)
1 passing (3s)
```
Boom. Just like that, you've used features to run two different code paths for
different environments.
#### 4. Program Config
Features are great for setting different values at compilation, but what if you
wanted to be able to dynamically update the fee percentage used by the program?
Let's make that possible by creating a Program Config account that allows us to
update the fee without upgrading the program.
To begin, let's first update the `lib.rs` file to:
1. Include a `SEED_PROGRAM_CONFIG` constant, which will be used to generate the
PDA for the program config account.
2. Include an `ADMIN` constant, which will be used as a constraint when
initializing the program config account. Run the `solana address` command to
get your address to use as the constant's value.
3. Include a `state` module that we'll implement shortly.
4. Include the `initialize_program_config` and `update_program_config`
instructions and calls to their "handlers," both of which we'll implement in
another step.
```rust theme={null}
use anchor_lang::prelude::*;
use solana_program::{pubkey, pubkey::Pubkey};
mod instructions;
mod state;
use instructions::*;
declare_id!("BC3RMBvVa88zSDzPXnBXxpnNYCrKsxnhR3HwwHhuKKei");
#[cfg(feature = "local-testing")]
#[constant]
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("envgiPXWwmpkHFKdy4QLv2cypgAWmVTVEm71YbNpYRu");
#[cfg(not(feature = "local-testing"))]
#[constant]
pub const USDC_MINT_PUBKEY: Pubkey = pubkey!("EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v");
pub const SEED_PROGRAM_CONFIG: &[u8] = b"program_config";
#[constant]
pub const ADMIN: Pubkey = pubkey!("...");
#[program]
pub mod config {
use super::*;
pub fn initialize_program_config(ctx: Context) -> Result<()> {
instructions::initialize_program_config_handler(ctx)
}
pub fn update_program_config(
ctx: Context,
new_fee: u64,
) -> Result<()> {
instructions::update_program_config_handler(ctx, new_fee)
}
pub fn payment(ctx: Context, amount: u64) -> Result<()> {
instructions::payment_handler(ctx, amount)
}
}
```
#### 5. Program Config State
Next, let's define the structure for the `ProgramConfig` state. This account
will store the admin, the token account where fees are sent, and the fee rate.
We'll also specify the number of bytes required to store this structure.
Create a new file called `state.rs` in the `/src` directory and add the
following code.
```rust theme={null}
use anchor_lang::prelude::*;
#[account]
pub struct ProgramConfig {
pub admin: Pubkey,
pub fee_destination: Pubkey,
pub fee_basis_points: u64,
}
impl ProgramConfig {
pub const LEN: usize = 8 + 32 + 32 + 8;
}
```
#### 6. Add Initialize Program Config Account Instruction
Now let's create the instruction logic for initializing the program config
account. It should only be callable by a transaction signed by the `ADMIN` key
and should set all the properties on the `ProgramConfig` account.
Create a folder called `program_config` at the path
`/src/instructions/program_config`. This folder will store all instructions
related to the program config account.
Within the `program_config` folder, create a file called
`initialize_program_config.rs` and add the following code.
```rust theme={null}
use crate::state::ProgramConfig;
use crate::ADMIN;
use crate::SEED_PROGRAM_CONFIG;
use crate::USDC_MINT_PUBKEY;
use anchor_lang::prelude::*;
use anchor_spl::token::TokenAccount;
#[derive(Accounts)]
pub struct InitializeProgramConfig<'info> {
#[account(init, seeds = [SEED_PROGRAM_CONFIG], bump, payer = authority, space = ProgramConfig::LEN)]
pub program_config: Account<'info, ProgramConfig>,
#[account( token::mint = USDC_MINT_PUBKEY)]
pub fee_destination: Account<'info, TokenAccount>,
#[account(mut, address = ADMIN)]
pub authority: Signer<'info>,
pub system_program: Program<'info, System>,
}
pub fn initialize_program_config_handler(ctx: Context) -> Result<()> {
ctx.accounts.program_config.admin = ctx.accounts.authority.key();
ctx.accounts.program_config.fee_destination = ctx.accounts.fee_destination.key();
ctx.accounts.program_config.fee_basis_points = 100;
Ok(())
}
```
#### 7. Add Update Program Config Fee Instruction
Next, implement the instruction logic for updating the config account. The
instruction should require that the signer match the `admin` stored in the
`program_config` account.
Within the `program_config` folder, create a file called
`update_program_config.rs` and add the following code.
```rust theme={null}
use crate::state::ProgramConfig;
use crate::SEED_PROGRAM_CONFIG;
use crate::USDC_MINT_PUBKEY;
use anchor_lang::prelude::*;
use anchor_spl::token::TokenAccount;
#[derive(Accounts)]
pub struct UpdateProgramConfig<'info> {
#[account(mut, seeds = [SEED_PROGRAM_CONFIG], bump)]
pub program_config: Account<'info, ProgramConfig>,
#[account( token::mint = USDC_MINT_PUBKEY)]
pub fee_destination: Account<'info, TokenAccount>,
#[account(
mut,
address = program_config.admin,
)]
pub admin: Signer<'info>,
/// CHECK: arbitrarily assigned by existing admin
pub new_admin: UncheckedAccount<'info>,
}
pub fn update_program_config_handler(
ctx: Context,
new_fee: u64,
) -> Result<()> {
ctx.accounts.program_config.admin = ctx.accounts.new_admin.key();
ctx.accounts.program_config.fee_destination = ctx.accounts.fee_destination.key();
ctx.accounts.program_config.fee_basis_points = new_fee;
Ok(())
}
```
#### 8. Add mod.rs and update instructions.rs
Next, let's expose the instruction handlers we created so that the call from
`lib.rs` doesn't show an error. Start by adding a file `mod.rs` in the
`program_config` folder. Add the code below to make the two modules,
`initialize_program_config` and `update_program_config` accessible.
```rust theme={null}
mod initialize_program_config;
pub use initialize_program_config::*;
mod update_program_config;
pub use update_program_config::*;
```
Now, update `instructions.rs` at the path `/src/instructions.rs`. Add the code
below to make the two modules, `program_config` and `payment` accessible.
```rust theme={null}
mod program_config;
pub use program_config::*;
mod payment;
pub use payment::*;
```
#### 9. Update Payment Instruction
Lastly, let's update the payment instruction to check that the `fee_destination`
account in the instruction matches the `fee_destination` stored in the program
config account. Then update the instruction's fee calculation to be based on the
`fee_basis_point` stored in the program config account.
```rust theme={null}
use crate::state::ProgramConfig;
use crate::SEED_PROGRAM_CONFIG;
use crate::USDC_MINT_PUBKEY;
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Token, TokenAccount};
#[derive(Accounts)]
pub struct Payment<'info> {
#[account(
seeds = [SEED_PROGRAM_CONFIG],
bump,
has_one = fee_destination
)]
pub program_config: Account<'info, ProgramConfig>,
#[account(
mut,
token::mint = USDC_MINT_PUBKEY
)]
pub fee_destination: Account<'info, TokenAccount>,
#[account(
mut,
token::mint = USDC_MINT_PUBKEY
)]
pub sender_token_account: Account<'info, TokenAccount>,
#[account(
mut,
token::mint = USDC_MINT_PUBKEY
)]
pub receiver_token_account: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
#[account(mut)]
pub sender: Signer<'info>,
}
pub fn payment_handler(ctx: Context, amount: u64) -> Result<()> {
let fee_amount = amount
.checked_mul(ctx.accounts.program_config.fee_basis_points)
.unwrap()
.checked_div(10000)
.unwrap();
let remaining_amount = amount.checked_sub(fee_amount).unwrap();
msg!("Amount: {}", amount);
msg!("Fee Amount: {}", fee_amount);
msg!("Remaining Transfer Amount: {}", remaining_amount);
token::transfer(
CpiContext::new(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.sender_token_account.to_account_info(),
authority: ctx.accounts.sender.to_account_info(),
to: ctx.accounts.fee_destination.to_account_info(),
},
),
fee_amount,
)?;
token::transfer(
CpiContext::new(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.sender_token_account.to_account_info(),
authority: ctx.accounts.sender.to_account_info(),
to: ctx.accounts.receiver_token_account.to_account_info(),
},
),
remaining_amount,
)?;
Ok(())
}
```
#### 10. Test
Now that we're done implementing our new program configuration struct and
instructions, let's move on to testing our updated program. To begin, add the
PDA for the program config account to the test file.
```typescript theme={null}
describe("config", () => {
...
const programConfig = findProgramAddressSync(
[Buffer.from("program_config")],
program.programId
)[0]
...
```
Next, update the test file with three more tests testing that:
1. The program config account is initialized correctly
2. The payment instruction is functioning as intended
3. The config account can be updated successfully by the admin
4. The config account cannot be updated by someone other than the admin
The first test initializes the program config account and verifies that the
correct fee is set and that the correct admin is stored on the program config
account.
```typescript theme={null}
it("Initialize Program Config Account", async () => {
const tx = await program.methods
.initializeProgramConfig()
.accounts({
programConfig: programConfig,
feeDestination: feeDestination,
authority: wallet.publicKey,
systemProgram: anchor.web3.SystemProgram.programId,
})
.rpc();
assert.strictEqual(
(
await program.account.programConfig.fetch(programConfig)
).feeBasisPoints.toNumber(),
100,
);
assert.strictEqual(
(await program.account.programConfig.fetch(programConfig)).admin.toString(),
wallet.publicKey.toString(),
);
});
```
The second test verifies that the payment instruction is working correctly, with
the fee being sent to the fee destination and the remaining balance being
transferred to the receiver. Here we update the existing test to include the
`programConfig` account.
```typescript theme={null}
it("Payment completes successfully", async () => {
const tx = await program.methods
.payment(new anchor.BN(10000))
.accounts({
programConfig: programConfig,
feeDestination: feeDestination,
senderTokenAccount: senderTokenAccount,
receiverTokenAccount: receiverTokenAccount,
sender: sender.publicKey,
})
.transaction();
await anchor.web3.sendAndConfirmTransaction(connection, tx, [sender]);
assert.strictEqual(
(await connection.getTokenAccountBalance(senderTokenAccount)).value
.uiAmount,
0,
);
assert.strictEqual(
(await connection.getTokenAccountBalance(feeDestination)).value.uiAmount,
100,
);
assert.strictEqual(
(await connection.getTokenAccountBalance(receiverTokenAccount)).value
.uiAmount,
9900,
);
});
```
The third test attempts to update the fee on the program config account, which
should be successful.
```typescript theme={null}
it("Update Program Config Account", async () => {
const tx = await program.methods
.updateProgramConfig(new anchor.BN(200))
.accounts({
programConfig: programConfig,
admin: wallet.publicKey,
feeDestination: feeDestination,
newAdmin: sender.publicKey,
})
.rpc();
assert.strictEqual(
(
await program.account.programConfig.fetch(programConfig)
).feeBasisPoints.toNumber(),
200,
);
});
```
The fourth test tries to update the fee on the program config account, where the
admin is not the one stored on the program config account, and this should fail.
```typescript theme={null}
it("Update Program Config Account with unauthorized admin (expect fail)", async () => {
try {
const tx = await program.methods
.updateProgramConfig(new anchor.BN(300))
.accounts({
programConfig: programConfig,
admin: sender.publicKey,
feeDestination: feeDestination,
newAdmin: sender.publicKey,
})
.transaction();
await anchor.web3.sendAndConfirmTransaction(connection, tx, [sender]);
} catch (err) {
expect(err);
}
});
```
Finally, run the test using the following command:
```
anchor test -- --features "local-testing"
```
You should see the following output:
```
config
✔ Initialize Program Config Account (199ms)
✔ Payment completes successfully (405ms)
✔ Update Program Config Account (403ms)
✔ Update Program Config Account with unauthorized admin (expect fail)
4 passing (8s)
```
And that's it! You've made the program a lot easier to work with moving forward.
If you want to take a look at the final solution code you can find it on
the `solution` branch
of [the same repository](https://github.com/Unboxed-Software/solana-admin-instructions/tree/solution).
## Challenge
Now it's time for you to do some of this on your own. We mentioned being able to
use the program's upgrade authority as the initial admin. Go ahead and update
the lab's `initialize_program_config` so that only the upgrade authority can
call it rather than having a hardcoded `ADMIN`.
Note that the `anchor test` command, when run on a local network, starts a new
test validator using `solana-test-validator`. This test validator uses a
non-upgradeable loader. The non-upgradeable loader makes it so the program's
`program_data` account isn't initialized when the validator starts. You'll
recall from the lesson that this account is how we access the upgrade authority
from the program.
To work around this, you can add a `deploy` function to the test file that runs
the deploy command for the program with an upgradeable loader. To use it, run
`anchor test --skip-deploy`, and call the `deploy` function within the test to
run the deploy command after the test validator has started.
```typescript theme={null}
import { execSync } from "child_process"
...
const deploy = () => {
const deployCmd = `solana program deploy --url localhost -v --program-id $(pwd)/target/deploy/config-keypair.json $(pwd)/target/deploy/config.so`
execSync(deployCmd)
}
...
before(async () => {
...
deploy()
})
```
For example, the command to run the test with features would look like this:
```
anchor test --skip-deploy -- --features "local-testing"
```
Try doing this on your own, but if you get stuck, feel free to reference the
`challenge` branch of
[the same repository](https://github.com/Unboxed-Software/solana-admin-instructions/tree/challenge)
to see one possible solution.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=02a7dab7-d9c1-495b-928c-a4412006ec20)!
# Rust Procedural Macros
Source: https://docs.orbition.network/courses/program-optimization/rust-macros
Use Rust macros to generate code at compile time.
## Summary
* **Procedural macros** are a special kind of Rust macros that allow the
programmer to generate code at compile time based on custom input.
* In the Anchor framework, procedural macros are used to generate code that
reduces the amount of boilerplate required when writing Orbition Native Chain programs.
* An **Abstract Syntax Tree (AST)** is a representation of the syntax and
structure of the input code that is passed to a procedural macro. When
creating a macro, you use elements of the AST like tokens and items to
generate the appropriate code.
* A **Token** is the smallest unit of source code that can be parsed by the
compiler in Rust.
* An **Item** is a declaration that defines something that can be used in a Rust
program, such as a struct, an enum, a trait, a function, or a method.
* A **TokenStream** is a sequence of tokens that represents a piece of source
code, and can be passed to a procedural macro to allow it to access and
manipulate the individual tokens in the code.
## Lesson
In Rust, a macro is a piece of code that you can write once and then "expand" to
generate code at compile time. This can be useful when you need to generate code
that is repetitive or complex, or when you want to use the same code in multiple
places in your program.
There are two different types of macros: declarative macros and procedural
macros.
* Declarative macros are defined using the `macro_rules!` macro, which allows
you to match against patterns of code and generate code based on the matching
pattern.
* Procedural macros in Rust are defined using Rust code and operate on the
abstract syntax tree (AST) of the input TokenStream, which allows them to
manipulate and generate code at a finer level of detail.
In this lesson, we'll focus on procedural macros, which are commonly used in the
Anchor framework.
### Rust concepts
Before we dig into macros, specifically, let's talk about some of the important
terminology, concepts, and tools we'll be using throughout the lesson.
#### Token
In the context of Rust programming, a
[token](https://doc.rust-lang.org/reference/tokens.html) is a basic element of
the language syntax like an identifier or literal value. Tokens represent the
smallest unit of source code that are recognized by the Rust compiler, and they
are used to build up more complex expressions and statements in a program.
Examples of Rust tokens include:
* [Keywords](https://doc.rust-lang.org/reference/keywords.html), such as `fn`,
`let`, and `match`, are reserved words in the Rust language that have special
meanings.
* [Identifiers](https://doc.rust-lang.org/reference/identifiers.html), such as
variable and function names, are used to refer to values and functions.
* [Punctuation](https://doc.rust-lang.org/reference/tokens.html#punctuation)
marks, such as `{`, `}`, and `;`, are used to structure and delimit blocks of
code.
* [Literals](https://doc.rust-lang.org/reference/tokens.html#literals), such as
numbers and strings, represent constant values in a Rust program.
You can
[read more about Rust tokens](https://doc.rust-lang.org/reference/tokens.html).
#### Item
Items are named, self-contained pieces of code in Rust. They provide a way to
group related code together and give it a name by which the group can be
referenced. This allows you to reuse and organize your code in a modular way.
There are several different kinds of items, such as:
* Functions
* Structs
* Enums
* Traits
* Modules
* Macros
You can
[read more about Rust items](https://doc.rust-lang.org/reference/items.html).
#### Token Streams
The `TokenStream` type is a data type that represents a sequence of tokens. This
type is defined in the `proc_macro` crate and is surfaced as a way for you to
write macros based on other code in the codebase.
When defining a procedural macro, the macro input is passed to the macro as a
`TokenStream`, which can then be parsed and transformed as needed. The resulting
`TokenStream` can then be expanded into the final code output by the macro.
```rust theme={null}
use proc_macro::TokenStream;
#[proc_macro]
pub fn my_macro(input: TokenStream) -> TokenStream {
...
}
```
#### Abstract syntax tree
In the context of a Rust procedural macro, an abstract syntax tree (AST) is a
data structure that represents the hierarchical structure of the input tokens
and their meaning in the Rust language. It's typically used as an intermediate
representation of the input that can be easily processed and transformed by the
procedural macro.
The macro can use the AST to analyze the input code and make changes to it, such
as adding or removing tokens, or transforming the meaning of the code in some
way. It can then use this transformed AST to generate new code, which can be
returned as the output of the proc macro.
#### The `syn` crate
The `syn` crate is available to help parse a token stream into an AST that macro
code can traverse and manipulate. When a procedural macro is invoked in a Rust
program, the macro function is called with a token stream as the input. Parsing
this input is the first step to virtually any macro.
Take as an example a proc macro that you invoke using `my_macro!`as follows:
```rust theme={null}
my_macro!("hello, world");
```
When the above code is executed, the Rust compiler passes the input tokens
(`"hello, world"`) as a `TokenStream` to the `my_macro` proc macro.
```rust theme={null}
use proc_macro::TokenStream;
use syn::parse_macro_input;
#[proc_macro]
pub fn my_macro(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as syn::LitStr);
eprintln! {"{:#?}", ast};
...
}
```
Inside the proc macro, the code uses the `parse_macro_input!` macro from the
`syn` crate to parse the input `TokenStream` into an abstract syntax tree (AST).
Specifically, this example parses it as an instance of `LitStr` that represents
a string literal in Rust. The `eprintln!` macro is then used to print the
`LitStr` AST for debugging purposes.
```rust theme={null}
LitStr {
token: Literal {
kind: Str,
symbol: "hello, world",
suffix: None,
span: #0 bytes(172..186),
},
}
```
The output of the `eprintln!` macro shows the structure of the `LitStr` AST that
was generated from the input tokens. It shows the string literal value
(`"hello, world"`) and other metadata about the token, such as its kind (`Str`),
suffix (`None`), and span.
#### The `quote` crate
Another important crate is the `quote` crate. This crate is pivotal in the code
generation portion of the macro.
Once a proc macro has finished analyzing and transforming the AST, it can use
the `quote` crate or a similar code generation library to convert the AST back
into a token stream. After that, it returns the `TokenStream`, which the Rust
compiler uses to replace the original stream in the source code.
Take the below example of `my_macro`:
```rust theme={null}
use proc_macro::TokenStream;
use syn::parse_macro_input;
use quote::quote;
#[proc_macro]
pub fn my_macro(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as syn::LitStr);
eprintln! {"{:#?}", ast};
let expanded = {
quote! {println!("The input is: {}", #ast)}
};
expanded.into()
}
```
This example uses the `quote!` macro to generate a new `TokenStream` consisting
of a `println!` macro call with the `LitStr` AST as its argument.
Note that the `quote!` macro generates a `TokenStream` of type
`proc_macro2::TokenStream`. To return this `TokenStream` to the Rust compiler,
you need to use the `.into()` method to convert it to `proc_macro::TokenStream`.
The Rust compiler will then use this `TokenStream` to replace the original proc
macro call in the source code.
```text theme={null}
The input is: hello, world
```
This allows you to create procedural macros that perform powerful code
generation and metaprogramming tasks.
### Procedural Macro
Procedural macros in Rust are a powerful way to extend the language and create
custom syntax. These macros are written in Rust and are compiled along with the
rest of the code. There are three types of procedural macros:
* Function-like macros - `custom!(...)`
* Derive macros - `#[derive(CustomDerive)]`
* Attribute macros - `#[CustomAttribute]`
This section will discuss the three types of procedural macros and provide an
example implementation of one. The process of writing a procedural macro is
consistent across all three types, so the example provided can be adapted to the
other types.
#### Function-like macros
Function-like procedural macros are the simplest of the three types of
procedural macros. These macros are defined using a function preceded by the
`#[proc_macro]` attribute. The function must take a `TokenStream` as input and
return a new `TokenStream` as output to replace the original code.
```rust theme={null}
#[proc_macro]
pub fn my_macro(input: TokenStream) -> TokenStream {
...
}
```
These macros are invoked using the name of the function followed by the `!`
operator. They can be used in various places in a Rust program, such as in
expressions, statements, and function definitions.
```rust theme={null}
my_macro!(input);
```
Function-like procedural macros are best suited for simple code generation tasks
that require only a single input and output stream. They are easy to understand
and use, and they provide a straightforward way to generate code at compile
time.
#### Attribute macros
Attribute macros define new attributes that are attached to items in a Rust
program such as functions and structs.
```rust theme={null}
#[my_macro]
fn my_function() {
...
}
```
Attribute macros are defined with a function preceded by the
`#[proc_macro_attribute]` attribute. The function requires two token streams as
input and returns a single `TokenStream` as output that replaces the original
item with an arbitrary number of new items.
```rust theme={null}
#[proc_macro_attribute]
pub fn my_macro(attr: TokenStream, input: TokenStream) -> TokenStream {
...
}
```
The first token stream input represents attribute arguments. The second token
stream is the rest of the item that the attribute is attached to, including any
other attributes that may be present.
```rust theme={null}
#[my_macro(arg1, arg2)]
fn my_function() {
...
}
```
For example, an attribute macro could process the arguments passed to the
attribute to enable or disable certain features, and then use the second token
stream to modify the original item in some way. By having access to both token
streams, attribute macros can provide greater flexibility and functionality
compared to using only a single token stream.
#### Derive macros
Derive macros are invoked using the `#[derive]` attribute on a struct, enum, or
union. They are typically used to automatically implement traits for the input
types.
```rust theme={null}
#[derive(MyMacro)]
struct Input {
field: String
}
```
Derive macros are defined with a function preceded by the `#[proc_macro_derive]`
attribute. They're limited to generating code for structs, enums, and unions.
They take a single token stream as input and return a single token stream as
output.
Unlike the other procedural macros, the returned token stream doesn't replace
the original code. Rather, the returned token stream gets appended to the module
or block that the original item belongs to. This allows developers to extend the
functionality of the original item without modifying the original code.
```rust theme={null}
#[proc_macro_derive(MyMacro)]
pub fn my_macro(input: TokenStream) -> TokenStream {
...
}
```
In addition to implementing traits, derive macros can define helper attributes.
Helper attributes can be used in the scope of the item that the derive macro is
applied to and customize the code generation process.
```rust theme={null}
#[proc_macro_derive(MyMacro, attributes(helper))]
pub fn my_macro(body: TokenStream) -> TokenStream {
...
}
```
Helper attributes are inert, which means they do not have any effect on their
own, and their only purpose is to be used as input to the derive macro that
defined them.
```rust theme={null}
#[derive(MyMacro)]
struct Input {
#[helper]
field: String
}
```
For example, a derive macro could define a helper attribute to perform
additional operations depending on the presence of the attribute. This allows
developers to further extend the functionality of derive macros and customize
the code they generate in a more flexible way.
#### Example of a procedural macro
This example shows how to use a derive procedural macro to automatically
generate an implementation of a `describe()` method for a struct.
```rust theme={null}
use example_macro::Describe;
#[derive(Describe)]
struct MyStruct {
my_string: String,
my_number: u64,
}
fn main() {
MyStruct::describe();
}
```
The `describe()` method will print a description of the struct's fields to the
console.
```text theme={null}
MyStruct is a struct with these named fields: my_string, my_number.
```
The first step is to define the procedural macro using the using the
`#[proc_macro_derive]` attribute. The input `TokenStream` is parsed using the
`parse_macro_input!()` macro to extract the struct's identifier and data.
```rust theme={null}
use proc_macro::{self, TokenStream};
use quote::quote;
use syn::{parse_macro_input, DeriveInput, FieldsNamed};
#[proc_macro_derive(Describe)]
pub fn describe_struct(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
...
}
```
The next step is to use the `match` keyword to perform pattern matching on the
`data` value to extract the names of the fields in the struct.
The first `match` has two arms: one for the `syn::Data::Struct` variant, and one
for the "catch-all" `_` arm that handles all other variants of `syn::Data`.
The second `match` has two arms as well: one for the `syn::Fields::Named`
variant, and one for the "catch-all" `_` arm that handles all other variants of
`syn::Fields`.
The `#(#idents), *` syntax specifies that the `idents` iterator will be
"expanded" to create a comma-separated list of the elements in the iterator.
```rust theme={null}
use proc_macro::{self, TokenStream};
use quote::quote;
use syn::{parse_macro_input, DeriveInput, FieldsNamed};
#[proc_macro_derive(Describe)]
pub fn describe_struct(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
let field_names = match data {
syn::Data::Struct(s) => match s.fields {
syn::Fields::Named(FieldsNamed { named, .. }) => {
let idents = named.iter().map(|f| &f.ident);
format!(
"a struct with these named fields: {}",
quote! {#(#idents), *},
)
}
_ => panic!("The syn::Fields variant is not supported"),
},
_ => panic!("The syn::Data variant is not supported"),
};
...
}
```
The last step is to implement a `describe()` method for a struct. The `expanded`
variable is defined using the `quote!` macro and the `impl` keyword to create an
implementation for the struct name stored in the `#ident` variable.
This implementation defines the `describe()` method that uses the `println!`
macro to print the name of the struct and its field names.
Finally, the `expanded` variable is converted into a `TokenStream` using the
`into()` method.
```rust theme={null}
use proc_macro::{self, TokenStream};
use quote::quote;
use syn::{parse_macro_input, DeriveInput, FieldsNamed};
#[proc_macro_derive(Describe)]
pub fn describe(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
let field_names = match data {
syn::Data::Struct(s) => match s.fields {
syn::Fields::Named(FieldsNamed { named, .. }) => {
let idents = named.iter().map(|f| &f.ident);
format!(
"a struct with these named fields: {}",
quote! {#(#idents), *},
)
}
_ => panic!("The syn::Fields variant is not supported"),
},
_ => panic!("The syn::Data variant is not supported"),
};
let expanded = quote! {
impl #ident {
fn describe() {
println!("{} is {}.", stringify!(#ident), #field_names);
}
}
};
expanded.into()
}
```
Now, when the `#[derive(Describe)]` attribute is added to a struct, the Rust
compiler automatically generates an implementation of the `describe()` method
that can be called to print the name of the struct and the names of its fields.
```rust theme={null}
#[derive(Describe)]
struct MyStruct {
my_string: String,
my_number: u64,
}
```
The `cargo expand` command from the `cargo-expand` crate can be used to expand
Rust code that uses procedural macros. For example, the code for the `MyStruct`
struct generated using the the `#[derive(Describe)]` attribute looks like this:
```rust theme={null}
struct MyStruct {
my_string: String,
my_number: f64,
}
impl MyStruct {
fn describe() {
{
::std::io::_print(
::core::fmt::Arguments::new_v1(
&["", " is ", ".\n"],
&[
::core::fmt::ArgumentV1::new_display(&"MyStruct"),
::core::fmt::ArgumentV1::new_display(
&"a struct with these named fields: my_string, my_number",
),
],
),
);
};
}
}
```
### Anchor procedural macros
Procedural macros are the magic behind the Anchor library that is commonly used
in Orbition Native Chain development. Anchor macros allow for more succinct code, common
security checks, and more. Let's go through a few examples of how Anchor uses
procedural macros.
#### Function-like macro
The `declare_id` macro shows how function-like macros are used in Anchor. This
macro takes in a string of characters representing a program's ID as input and
converts it into a `Pubkey` type that can be used in the Anchor program.
```rust theme={null}
declare_id!("G839pmstFmKKGEVXRGnauXxFgzucvELrzuyk6gHTiK7a");
```
The `declare_id` macro is defined using the `#[proc_macro]` attribute,
indicating that it's a function-like proc macro.
```rust theme={null}
#[proc_macro]
pub fn declare_id(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let id = parse_macro_input!(input as id::Id);
proc_macro::TokenStream::from(quote! {#id})
}
```
#### Derive macro
The `#[derive(Accounts)]` is an example of just one of many derive macros that
are used in Anchor.
The `#[derive(Accounts)]` macro generates code that implements the `Accounts`
trait for the given struct. This trait does a number of things, including
validating and deserializing the accounts passed into an instruction. This
allows the struct to be used as a list of accounts required by an instruction in
an Anchor program.
Any constraints specified on fields by the `#[account(..)]` attribute are
applied during deserialization. The `#[instruction(..)]` attribute can also be
added to specify the instruction's arguments and make them accessible to the
macro.
```rust theme={null}
#[derive(Accounts)]
#[instruction(input: String)]
pub struct Initialize<'info> {
#[account(init, payer = payer, space = 8 + input.len())]
pub data_account: Account<'info, MyData>,
#[account(mut)]
pub payer: Signer<'info>,
pub system_program: Program<'info, System>,
}
```
This macro is defined using the `proc_macro_derive` attribute, which allows it
to be used as a derive macro that can be applied to a struct. The line
`#[proc_macro_derive(Accounts, attributes(account, instruction))]` indicates
that this is a derive macro that processes `account` and `instruction` helper
attributes.
```rust theme={null}
#[proc_macro_derive(Accounts, attributes(account, instruction))]
pub fn derive_anchor_deserialize(item: TokenStream) -> TokenStream {
parse_macro_input!(item as anchor_syn::AccountsStruct)
.to_token_stream()
.into()
}
```
#### Attribute macro `#[program]`
The `#[program]` attribute macro is an example of an attribute macro used in
Anchor to define the module containing instruction handlers for a Orbition Native Chain
program.
```rust theme={null}
#[program]
pub mod my_program {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
...
}
}
```
In this case, the `#[program]` attribute is applied to a module, and it is used
to specify that the module contains instruction handlers for a Orbition Native Chain program.
```rust theme={null}
#[proc_macro_attribute]
pub fn program(
_args: proc_macro::TokenStream,
input: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
parse_macro_input!(input as anchor_syn::Program)
.to_token_stream()
.into()
}
```
Overall, the use of proc macros in Anchor greatly reduces the amount of
repetitive code that Orbition Native Chain developers have to write. By reducing the amount of
boilerplate code, developers can focus on their program's core functionality and
avoid mistakes caused by manual repetition. This ultimately results in a faster
and more efficient development process.
## Lab
Let's practice this by creating a new derive macro! Our new macro will let us
automatically generate instruction logic for updating each field on an account
in an Anchor program.
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/anchor-custom-macro/tree/starter).
The starter code includes a simple Anchor program that allows you to initialize
and update a `Config` account. This is similar to what we did with the
[Program Configuration lesson](/content/courses/program-optimization/program-configuration).
The account in question is structured as follows:
```rust theme={null}
use anchor_lang::prelude::*;
#[account]
pub struct Config {
pub auth: Pubkey,
pub bool: bool,
pub first_number: u8,
pub second_number: u64,
}
impl Config {
pub const LEN: usize = 8 + 32 + 1 + 1 + 8;
}
```
The `programs/admin/src/lib.rs` file contains the program entrypoint with the
definitions of the program's instructions. Currently, the program has
instructions to initialize this account and then one instruction per account
field for updating the field.
The `programs/admin/src/admin_config` directory contains the program's
instruction logic and state. Take a look through each of these files. You'll
notice that instruction logic for each field is duplicated for each instruction.
The goal of this lab is to implement a procedural macro that will allow us to
replace all of the instruction logic functions and automatically generate
functions for each instruction.
#### 2. Set up the custom macro declaration
Let's get started by creating a separate crate for our custom macro. In the
project's root directory, run `cargo new custom-macro`. This will create a new
`custom-macro` directory with its own `Cargo.toml`. Update the new `Cargo.toml`
file to be the following:
```text theme={null}
[package]
name = "custom-macro"
version = "0.1.0"
edition = "2021"
[lib]
proc-macro = true
[dependencies]
syn = "1.0.105"
quote = "1.0.21"
proc-macro2 = "0.4"
anchor-lang = "0.25.0"
```
The `proc-macro = true` line defines this crate as containing a procedural
macro. The dependencies are all crates we'll be using to create our derive
macro.
Next, change `src/main.rs` to `src/lib.rs`.
Next, update the project root's `Cargo.toml` file's `members` field to include
`"custom-macro"`:
```text theme={null}
[workspace]
members = [
"programs/*",
"custom-macro"
]
```
Now our crate is set up and ready to go. But before we move on, let's create one
more crate at the root level that we can use to test out our macro as we create
it. Use `cargo new custom-macro-test` at the project root. Then update the newly
created `Cargo.toml` to add `anchor-lang` and the `custom-macro` crates as
dependencies:
```text theme={null}
[package]
name = "custom-macro-test"
version = "0.1.0"
edition = "2021"
[dependencies]
anchor-lang = "0.25.0"
custom-macro = { path = "../custom-macro" }
```
Next, update the root project's `Cargo.toml` to include the new
`custom-macro-test` crate as before:
```text theme={null}
[workspace]
members = [
"programs/*",
"custom-macro",
"custom-macro-test"
]
```
Finally, replace the code in `custom-macro-test/src/main.rs` with the following
code. We'll use this later for testing:
```rust theme={null}
use anchor_lang::prelude::*;
use custom_macro::InstructionBuilder;
#[derive(InstructionBuilder)]
pub struct Config {
pub auth: Pubkey,
pub bool: bool,
pub first_number: u8,
pub second_number: u64,
}
```
#### 3. Define the custom macro
Now, in the `custom-macro/src/lib.rs` file, let's add our new macro's
declaration. In this file, we’ll use the `parse_macro_input!` macro to parse the
input `TokenStream` and extract the `ident` and `data` fields from a
`DeriveInput` struct. Then, we’ll use the `eprintln!` macro to print the values
of `ident` and `data`. For now, we will use `TokenStream::new()` to return an
empty `TokenStream`.
```rust theme={null}
use proc_macro::TokenStream;
use quote::*;
use syn::*;
#[proc_macro_derive(InstructionBuilder)]
pub fn instruction_builder(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
eprintln! {"{:#?}", ident};
eprintln! {"{:#?}", data};
TokenStream::new()
}
```
Let's test what this prints. To do this, you first need to install the
`cargo-expand` command by running `cargo install cargo-expand`. You'll also need
to install the nightly version of Rust by running `rustup install nightly`.
Once you've done this, you can see the output of the code described above by
navigating to the `custom-macro-test` directory and running `cargo expand`.
This command expands macros in the crate. Since the `main.rs` file uses the
newly created `InstructionBuilder` macro, this will print the syntax tree for
the `ident` and `data` of the struct to the console. Once you have confirmed
that the input `TokenStream` is parsing correctly, feel free to remove the
`eprintln!` statements.
#### 4. Get the struct's fields
Next, let’s use `match` statements to get the named fields from the `data` of
the struct. Then we'll use the `eprintln!` macro to print the values of the
fields.
```rust theme={null}
use proc_macro::TokenStream;
use quote::*;
use syn::*;
#[proc_macro_derive(InstructionBuilder)]
pub fn instruction_builder(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
let fields = match data {
syn::Data::Struct(s) => match s.fields {
syn::Fields::Named(n) => n.named,
_ => panic!("The syn::Fields variant is not supported: {:#?}", s.fields),
},
_ => panic!("The syn::Data variant is not supported: {:#?}", data),
};
eprintln! {"{:#?}", fields};
TokenStream::new()
}
```
Once again, use `cargo expand` in the terminal to see the output of this code.
Once you have confirmed that the fields are being extracted and printed
correctly, you can remove the `eprintln!` statement.
#### 5. Build update instructions
Next, let’s iterate over the fields of the struct and generate an update
instruction for each field. The instruction will be generated using the `quote!`
macro and will include the field's name and type, as well as a new function name
for the update instruction.
```rust theme={null}
use proc_macro::TokenStream;
use quote::*;
use syn::*;
#[proc_macro_derive(InstructionBuilder)]
pub fn instruction_builder(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
let fields = match data {
syn::Data::Struct(s) => match s.fields {
syn::Fields::Named(n) => n.named,
_ => panic!("The syn::Fields variant is not supported: {:#?}", s.fields),
},
_ => panic!("The syn::Data variant is not supported: {:#?}", data),
};
let update_instruction = fields.into_iter().map(|f| {
let name = &f.ident;
let ty = &f.ty;
let fname = format_ident!("update_{}", name.clone().unwrap());
quote! {
pub fn #fname(ctx: Context, new_value: #ty) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.#name = new_value;
Ok(())
}
}
});
TokenStream::new()
}
```
#### 6. Return new `TokenStream`
Lastly, let’s use the `quote!` macro to generate an implementation for the
struct with the name specified by the `ident` variable. The implementation
includes the update instructions that were generated for each field in the
struct. The generated code is then converted to a `TokenStream` using the
`into()` method and returned as the result of the macro.
```rust theme={null}
use proc_macro::TokenStream;
use quote::*;
use syn::*;
#[proc_macro_derive(InstructionBuilder)]
pub fn instruction_builder(input: TokenStream) -> TokenStream {
let DeriveInput { ident, data, .. } = parse_macro_input!(input);
let fields = match data {
syn::Data::Struct(s) => match s.fields {
syn::Fields::Named(n) => n.named,
_ => panic!("The syn::Fields variant is not supported: {:#?}", s.fields),
},
_ => panic!("The syn::Data variant is not supported: {:#?}", data),
};
let update_instruction = fields.into_iter().map(|f| {
let name = &f.ident;
let ty = &f.ty;
let fname = format_ident!("update_{}", name.clone().unwrap());
quote! {
pub fn #fname(ctx: Context, new_value: #ty) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.#name = new_value;
Ok(())
}
}
});
let expanded = quote! {
impl #ident {
#(#update_instruction)*
}
};
expanded.into()
}
```
To verify that the macro is generating the correct code, use the `cargo expand`
command to see the expanded form of the macro. The output of this look like the
following:
```rust theme={null}
use anchor_lang::prelude::*;
use custom_macro::InstructionBuilder;
pub struct Config {
pub auth: Pubkey,
pub bool: bool,
pub first_number: u8,
pub second_number: u64,
}
impl Config {
pub fn update_auth(
ctx: Context,
new_value: Pubkey,
) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.auth = new_value;
Ok(())
}
pub fn update_bool(ctx: Context, new_value: bool) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.bool = new_value;
Ok(())
}
pub fn update_first_number(
ctx: Context,
new_value: u8,
) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.first_number = new_value;
Ok(())
}
pub fn update_second_number(
ctx: Context,
new_value: u64,
) -> Result<()> {
let admin_account = &mut ctx.accounts.admin_account;
admin_account.second_number = new_value;
Ok(())
}
}
```
#### 7. Update the program to use your new macro
To use the new macro to generate update instructions for the `Config` struct,
first add the `custom-macro` crate as a dependency to the program in its
`Cargo.toml`:
```text theme={null}
[dependencies]
anchor-lang = "0.25.0"
custom-macro = { path = "../../custom-macro" }
```
Then, navigate to the `state.rs` file in the Anchor program and update it with
the following code:
```rust theme={null}
use crate::admin_update::UpdateAdminAccount;
use anchor_lang::prelude::*;
use custom_macro::InstructionBuilder;
#[derive(InstructionBuilder)]
#[account]
pub struct Config {
pub auth: Pubkey,
pub bool: bool,
pub first_number: u8,
pub second_number: u64,
}
impl Config {
pub const LEN: usize = 8 + 32 + 1 + 1 + 8;
}
```
Next, navigate to the `admin_update.rs` file and delete the existing update
instructions. This should leave only the `UpdateAdminAccount` context struct in
the file.
```rust theme={null}
use crate::state::Config;
use anchor_lang::prelude::*;
#[derive(Accounts)]
pub struct UpdateAdminAccount<'info> {
pub auth: Signer<'info>,
#[account(
mut,
has_one = auth,
)]
pub admin_account: Account<'info, Config>,
}
```
Next, update `lib.rs` in the Anchor program to use the update instructions
generated by the `InstructionBuilder` macro.
```rust theme={null}
use anchor_lang::prelude::*;
mod admin_config;
use admin_config::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod admin {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
Initialize::initialize(ctx)
}
pub fn update_auth(ctx: Context, new_value: Pubkey) -> Result<()> {
Config::update_auth(ctx, new_value)
}
pub fn update_bool(ctx: Context, new_value: bool) -> Result<()> {
Config::update_bool(ctx, new_value)
}
pub fn update_first_number(ctx: Context, new_value: u8) -> Result<()> {
Config::update_first_number(ctx, new_value)
}
pub fn update_second_number(ctx: Context, new_value: u64) -> Result<()> {
Config::update_second_number(ctx, new_value)
}
}
```
Lastly, navigate to the `admin` directory and run `anchor test` to verify that
the update instructions generated by the `InstructionBuilder` macro are working
correctly.
```
admin
✔ Is initialized! (160ms)
✔ Update bool! (409ms)
✔ Update u8! (403ms)
✔ Update u64! (406ms)
✔ Update Admin! (405ms)
5 passing (2s)
```
Nice work! At this point, you can create procedural macros to help in your
development process. We encourage you to make the most of the Rust language and
use macros where they make sense. But even if you don't, knowing how they work
helps to understand what's happening with Anchor under the hood.
If you need to spend more time with the solution code, feel free to reference
the `solution` branch of
[the repository](https://github.com/Unboxed-Software/anchor-custom-macro/tree/solution).
## Challenge
To solidify what you've learned, go ahead and create another procedural macro on
your own. Think about code you've written that could be reduced or improved by a
macro and try it out! Since this is still practice, it's okay if it doesn't work
out the way you want or expect. Just jump in and experiment!
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=eb892157-3014-4635-beac-f562af600bf8)!
# Account Data Matching
Source: https://docs.orbition.network/courses/program-security/account-data-matching
How to check your program's data accounts in both Anchor and Native Rust.
## Summary
* Use **data validation checks** to verify that account data matches an expected
value. Without appropriate data validation checks, unexpected accounts may be
used in an instruction.
* To implement data validation checks in Rust, simply compare the data stored on
an account to an expected value.
```rust theme={null}
if ctx.accounts.user.key() != ctx.accounts.user_data.user {
return Err(ProgramError::InvalidAccountData.into());
}
```
* In Anchor, you can use `constraint` to checks whether the given expression
evaluates to true. Alternatively, you can use `has_one` to check that a target
account field stored on the account matches the key of an account in the
`Accounts` struct.
## Lesson
Account data matching refers to data validation checks used to verify the data
stored on an account matches an expected value. Data validation checks provide a
way to include additional constraints to ensure the appropriate accounts are
passed into an instruction.
This can be useful when accounts required by an instruction have dependencies on
values stored in other accounts or if an instruction is dependent on the data
stored in an account.
#### Missing data validation check
The example below includes an `update_admin` instruction that updates the
`admin` field stored on an `admin_config` account.
The instruction is missing a data validation check to verify the `admin` account
signing the transaction matches the `admin` stored on the `admin_config`
account. This means any account signing the transaction and passed into the
instruction as the `admin` account can update the `admin_config` account.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod data_validation {
use super::*;
...
pub fn update_admin(ctx: Context) -> Result<()> {
ctx.accounts.admin_config.admin = ctx.accounts.new_admin.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAdmin<'info> {
#[account(mut)]
pub admin_config: Account<'info, AdminConfig>,
#[account(mut)]
pub admin: Signer<'info>,
pub new_admin: SystemAccount<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
#### Add data validation check
The basic Rust approach to solve this problem is to simply compare the passed in
`admin` key to the `admin` key stored in the `admin_config` account, throwing an
error if they don’t match.
```rust theme={null}
if ctx.accounts.admin.key() != ctx.accounts.admin_config.admin {
return Err(ProgramError::InvalidAccountData.into());
}
```
By adding a data validation check, the `update_admin` instruction would only
process if the `admin` signer of the transaction matched the `admin` stored on
the `admin_config` account.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod data_validation {
use super::*;
...
pub fn update_admin(ctx: Context) -> Result<()> {
if ctx.accounts.admin.key() != ctx.accounts.admin_config.admin {
return Err(ProgramError::InvalidAccountData.into());
}
ctx.accounts.admin_config.admin = ctx.accounts.new_admin.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAdmin<'info> {
#[account(mut)]
pub admin_config: Account<'info, AdminConfig>,
#[account(mut)]
pub admin: Signer<'info>,
pub new_admin: SystemAccount<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
#### Use Anchor constraints
Anchor simplifies this with the `has_one` constraint. You can use the `has_one`
constraint to move the data validation check from the instruction logic to the
`UpdateAdmin` struct.
In the example below, `has_one = admin` specifies that the `admin` account
signing the transaction must match the `admin` field stored on the
`admin_config` account. To use the `has_one` constraint, the naming convention
of the data field on the account must be consistent with the naming on the
account validation struct.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod data_validation {
use super::*;
...
pub fn update_admin(ctx: Context) -> Result<()> {
ctx.accounts.admin_config.admin = ctx.accounts.new_admin.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAdmin<'info> {
#[account(
mut,
has_one = admin
)]
pub admin_config: Account<'info, AdminConfig>,
#[account(mut)]
pub admin: Signer<'info>,
pub new_admin: SystemAccount<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
Alternatively, you can use `constraint` to manually add an expression that must
evaluate to true in order for execution to continue. This is useful when for
some reason naming can’t be consistent or when you need a more complex
expression to fully validate the incoming data.
```rust theme={null}
#[derive(Accounts)]
pub struct UpdateAdmin<'info> {
#[account(
mut,
constraint = admin_config.admin == admin.key()
)]
pub admin_config: Account<'info, AdminConfig>,
#[account(mut)]
pub admin: Signer<'info>,
pub new_admin: SystemAccount<'info>,
}
```
## Lab
For this lab we’ll create a simple “vault” program similar to the program we
used in the Signer Authorization lesson and the Owner Check lesson. Similar to
those labs, we’ll show in this lab how a missing data validation check could
allow the vault to be drained.
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-account-data-matching).
The starter code includes a program with two instructions and the boilerplate
setup for the test file.
The `initialize_vault` instruction initializes a new `Vault` account and a new
`TokenAccount`. The `Vault` account will store the address of a token account,
the authority of the vault, and a withdraw destination token account.
The authority of the new token account will be set as the `vault`, a PDA of the
program. This allows the `vault` account to sign for the transfer of tokens from
the token account.
The `insecure_withdraw` instruction transfers all the tokens in the `vault`
account’s token account to a `withdraw_destination` token account.
Notice that this instruction \***\*does\*\*** have a signer check for
`authority` and an owner check for `vault`. However, nowhere in the account
validation or instruction logic is there code that checks that the `authority`
account passed into the instruction matches the `authority` account on the
`vault`.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod account_data_matching {
use super::*;
pub fn initialize_vault(ctx: Context) -> Result<()> {
ctx.accounts.vault.token_account = ctx.accounts.token_account.key();
ctx.accounts.vault.authority = ctx.accounts.authority.key();
ctx.accounts.vault.withdraw_destination = ctx.accounts.withdraw_destination.key();
Ok(())
}
pub fn insecure_withdraw(ctx: Context) -> Result<()> {
let amount = ctx.accounts.token_account.amount;
let seeds = &[b"vault".as_ref(), &[*ctx.bumps.get("vault").unwrap()]];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.vault.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
#[derive(Accounts)]
pub struct InitializeVault<'info> {
#[account(
init,
payer = authority,
space = 8 + 32 + 32 + 32,
seeds = [b"vault"],
bump,
)]
pub vault: Account<'info, Vault>,
#[account(
init,
payer = authority,
token::mint = mint,
token::authority = vault,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
pub withdraw_destination: Account<'info, TokenAccount>,
pub mint: Account<'info, Mint>,
#[account(mut)]
pub authority: Signer<'info>,
pub token_program: Program<'info, Token>,
pub system_program: Program<'info, System>,
pub rent: Sysvar<'info, Rent>,
}
#[derive(Accounts)]
pub struct InsecureWithdraw<'info> {
#[account(
seeds = [b"vault"],
bump,
)]
pub vault: Account<'info, Vault>,
#[account(
mut,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
pub authority: Signer<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
withdraw_destination: Pubkey,
}
```
#### 2. Test `insecure_withdraw` instruction
To prove that this is a problem, let’s write a test where an account other than
the vault’s `authority` tries to withdraw from the vault.
The test file includes the code to invoke the `initialize_vault` instruction
using the provider wallet as the `authority` and then mints 100 tokens to the
`vault` token account.
Add a test to invoke the `insecure_withdraw` instruction. Use
`withdrawDestinationFake` as the `withdrawDestination` account and `walletFake`
as the `authority`. Then send the transaction using `walletFake`.
Since there are no checks the verify the `authority` account passed into the
instruction matches the values stored on the `vault` account initialized in the
first test, the instruction will process successfully and the tokens will be
transferred to the `withdrawDestinationFake` account.
```typescript theme={null}
describe("account-data-matching", () => {
...
it("Insecure withdraw", async () => {
const tx = await program.methods
.insecureWithdraw()
.accounts({
vault: vaultPDA,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestinationFake,
authority: walletFake.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
const balance = await connection.getTokenAccountBalance(tokenPDA)
expect(balance.value.uiAmount).to.eq(0)
})
})
```
Run `anchor test` to see that both transactions will complete successfully.
```bash theme={null}
account-data-matching
✔ Initialize Vault (811ms)
✔ Insecure withdraw (403ms)
```
#### 3. Add `secure_withdraw` instruction
Let’s go implement a secure version of this instruction called
`secure_withdraw`.
This instruction will be identical to the `insecure_withdraw` instruction,
except we’ll use the `has_one` constraint in the account validation struct
(`SecureWithdraw`) to check that the `authority` account passed into the
instruction matches the `authority` account on the `vault` account. That way
only the correct authority account can withdraw the vault’s tokens.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod account_data_matching {
use super::*;
...
pub fn secure_withdraw(ctx: Context) -> Result<()> {
let amount = ctx.accounts.token_account.amount;
let seeds = &[b"vault".as_ref(), &[*ctx.bumps.get("vault").unwrap()]];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.vault.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
#[derive(Accounts)]
pub struct SecureWithdraw<'info> {
#[account(
seeds = [b"vault"],
bump,
has_one = token_account,
has_one = authority,
has_one = withdraw_destination,
)]
pub vault: Account<'info, Vault>,
#[account(
mut,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
pub authority: Signer<'info>,
}
```
#### 4. Test `secure_withdraw` instruction
Now let’s test the `secure_withdraw` instruction with two tests: one that uses
`walletFake` as the authority and one that uses `wallet` as the authority. We
expect the first invocation to return an error and the second to succeed.
```typescript theme={null}
describe("account-data-matching", () => {
...
it("Secure withdraw, expect error", async () => {
try {
const tx = await program.methods
.secureWithdraw()
.accounts({
vault: vaultPDA,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestinationFake,
authority: walletFake.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
} catch (err) {
expect(err)
console.log(err)
}
})
it("Secure withdraw", async () => {
await spl.mintTo(
connection,
wallet.payer,
mint,
tokenPDA,
wallet.payer,
100
)
await program.methods
.secureWithdraw()
.accounts({
vault: vaultPDA,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestination,
authority: wallet.publicKey,
})
.rpc()
const balance = await connection.getTokenAccountBalance(tokenPDA)
expect(balance.value.uiAmount).to.eq(0)
})
})
```
Run `anchor test` to see that the transaction using an incorrect authority
account will now return an Anchor Error while the transaction using correct
accounts completes successfully.
```bash theme={null}
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS invoke [1]',
'Program log: Instruction: SecureWithdraw',
'Program log: AnchorError caused by account: vault. Error Code: ConstraintHasOne. Error Number: 2001. Error Message: A has one constraint was violated.',
'Program log: Left:',
'Program log: DfLZV18rD7wCQwjYvhTFwuvLh49WSbXFeJFPQb5czifH',
'Program log: Right:',
'Program log: 5ovvmG5ntwUC7uhNWfirjBHbZD96fwuXDMGXiyMwPg87',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS consumed 10401 of 200000 compute units',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS failed: custom program error: 0x7d1'
```
Note that Anchor specifies in the logs the account that causes the error
(`AnchorError caused by account: vault`).
```bash theme={null}
✔ Secure withdraw, expect error (77ms)
✔ Secure withdraw (10073ms)
```
And just like that, you've closed up the security loophole. The theme across
most of these potential exploits is that they're quite simple. However, as your
programs grow in scope and complexity, it becomse increasingly easy to miss
possible exploits. It's great to get in a habit of writing tests that send
instructions that *shouldn't* work. The more the better. That way you catch
problems before you deploy.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the repository](https://github.com/Unboxed-Software/solana-account-data-matching/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that proper data checks
are in place to avoid security exploits.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=a107787e-ad33-42bb-96b3-0592efc1b92f)!
# Arbitrary CPI
Source: https://docs.orbition.network/courses/program-security/arbitrary-cpi
How to safely invoke Orbition Native Chain programs from other Orbition Native Chain programs.
## Summary
* To generate a CPI, the target program must be passed into the invoking
instruction as an account. This means that any target program could be passed
into the instruction. Your program should check for incorrect or unexpected
programs.
* Perform program checks in native programs by simply comparing the public key
of the passed-in program to the progam you expected.
* If a program is written in Anchor, then it may have a publicly available CPI
module. This makes invoking the program from another Anchor program simple and
secure. The Anchor CPI module automatically checks that the address of the
program passed in matches the address of the program stored in the module.
## Lesson
A cross program invocation (CPI) is when one program invokes an instruction on
another program. An “arbitrary CPI” is when a program is structured to issue a
CPI to whatever program is passed into the instruction rather than expecting to
perform a CPI to one specific program. Given that the callers of your program's
instruction can pass any program they'd like into the instruction's list of
accounts, failing to verify the address of a passed-in program results in your
program performing CPIs to arbitrary programs.
This lack of program checks creates an opportunity for a malicious user to pass
in a different program than expected, causing the original program to call an
instruction on this mystery program. There’s no telling what the consequences of
this CPI could be. It depends on the program logic (both that of the original
program and the unexpected program), as well as what other accounts are passed
into the original instruction.
### Missing program checks
Take the following program as an example. The `cpi` instruction invokes the
`transfer` instruction on `token_program`, but there is no code that checks
whether or not the `token_program` account passed into the instruction is, in
fact, the SPL Token Program.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_lang::solana_program;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod arbitrary_cpi_insecure {
use super::*;
pub fn cpi(ctx: Context, amount: u64) -> ProgramResult {
solana_program::program::invoke(
&spl_token::instruction::transfer(
ctx.accounts.token_program.key,
ctx.accounts.source.key,
ctx.accounts.destination.key,
ctx.accounts.authority.key,
&[],
amount,
)?,
&[
ctx.accounts.source.clone(),
ctx.accounts.destination.clone(),
ctx.accounts.authority.clone(),
],
)
}
}
#[derive(Accounts)]
pub struct Cpi<'info> {
source: UncheckedAccount<'info>,
destination: UncheckedAccount<'info>,
authority: UncheckedAccount<'info>,
token_program: UncheckedAccount<'info>,
}
```
An attacker could easily call this instruction and pass in a duplicate token
program that they created and control.
### Add program checks
It's possible to fix this vulnerabilty by simply adding a few lines to the `cpi`
instruction to check whether or not `token_program`'s public key is that of the
SPL Token Program.
```rust theme={null}
pub fn cpi_secure(ctx: Context, amount: u64) -> ProgramResult {
if &spl_token::ID != ctx.accounts.token_program.key {
return Err(ProgramError::IncorrectProgramId);
}
solana_program::program::invoke(
&spl_token::instruction::transfer(
ctx.accounts.token_program.key,
ctx.accounts.source.key,
ctx.accounts.destination.key,
ctx.accounts.authority.key,
&[],
amount,
)?,
&[
ctx.accounts.source.clone(),
ctx.accounts.destination.clone(),
ctx.accounts.authority.clone(),
],
)
}
```
Now, if an attacker passes in a different token program, the instruction will
return the `ProgramError::IncorrectProgramId` error.
Depending on the program you’re invoking with your CPI, you can either hard code
the address of the expected program ID or use the program’s Rust crate to get
the address of the program, if available. In the example above, the `spl_token`
crate provides the address of the SPL Token Program.
### Use an Anchor CPI module
A simpler way to manage program checks is to use Anchor CPI modules. We learned
in a
[previous lesson](https://github.com/Unboxed-Software/solana-course/blob/main/content/anchor-cpi)
that Anchor can automatically generate CPI modules to make CPIs into the program
simpler. These modules also enhance security by verifying the public key of the
program that’s passed into one of its public instructions.
Every Anchor program uses the `declare_id()` macro to define the address of the
program. When a CPI module is generated for a specific program, it uses the
address passed into this macro as the "source of truth" and will automatically
verify that all CPIs made using its CPI module target this program id.
While at the core no different than manual program checks, using CPI modules
avoids the possibility of forgetting to perform a program check or accidentally
typing in the wrong program ID when hard-coding it.
The program below shows an example of using a CPI module for the SPL Token
Program to perform the transfer shown in the previous examples.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod arbitrary_cpi_recommended {
use super::*;
pub fn cpi(ctx: Context, amount: u64) -> ProgramResult {
token::transfer(ctx.accounts.transfer_ctx(), amount)
}
}
#[derive(Accounts)]
pub struct Cpi<'info> {
source: Account<'info, TokenAccount>,
destination: Account<'info, TokenAccount>,
authority: Signer<'info>,
token_program: Program<'info, Token>,
}
impl<'info> Cpi<'info> {
pub fn transfer_ctx(&self) -> CpiContext<'_, '_, '_, 'info, token::Transfer<'info>> {
let program = self.token_program.to_account_info();
let accounts = token::Transfer {
from: self.source.to_account_info(),
to: self.destination.to_account_info(),
authority: self.authority.to_account_info(),
};
CpiContext::new(program, accounts)
}
}
```
Note that, like the example above, Anchor has created a few
[wrappers for popular native programs](https://github.com/coral-xyz/anchor/tree/master/spl/src)
that allow you to issue CPIs into them as if they were Anchor programs.
Additionally and depending on the program you’re making the CPI to, you may be
able to use Anchor’s
[`Program` account type](https://docs.rs/anchor-lang/latest/anchor_lang/accounts/program/struct.Program.html)
to validate the passed-in program in your account validation struct. Between
the [`anchor_lang`](https://docs.rs/anchor-lang/latest/anchor_lang) and [`anchor_spl`](https://docs.rs/anchor_spl/latest/) crates,
the following `Program` types are provided out of the box:
* [`System`](https://docs.rs/anchor-lang/latest/anchor_lang/system_program/struct.System.html)
* [`AssociatedToken`](https://docs.rs/anchor-spl/latest/anchor_spl/associated_token/struct.AssociatedToken.html)
* [`Token`](https://docs.rs/anchor-spl/latest/anchor_spl/token/struct.Token.html)
If you have access to an Anchor program's CPI module, you typically can import
its program type with the following, replacing the program name with the name of
the actual program:
```rust theme={null}
use other_program::program::OtherProgram;
```
## Lab
To show the importance of checking with program you use for CPIs, we're going to
work with a simplified and somewhat contrived game. This game represents
characters with PDA accounts, and uses a separate "metadata" program to manage
character metadata and attributes like health and power.
While this example is somewhat contrived, it's actually almost identical
architecture to how NFTs on Orbition Native Chain work: the SPL Token Program manages the token
mints, distribution, and transfers, and a separate metadata program is used to
assign metadata to tokens. So the vulnerability we go through here could also be
applied to real tokens.
#### 1. Setup
We'll start with the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-arbitrary-cpi/tree/starter).
Clone the repository and then open it on the `starter` branch.
Notice that there are three programs:
1. `gameplay`
2. `character-metadata`
3. `fake-metadata`
Additionally, there is already a test in the `tests` directory.
The first program, `gameplay`, is the one that our test directly uses. Take a
look at the program. It has two instructions:
1. `create_character_insecure` - creates a new character and CPI's into the
metadata program to set up the character's initial attributes
2. `battle_insecure` - pits two characters against each other, assigning a "win"
to the character with the highest attributes
The second program, `character-metadata`, is meant to be the "approved" program
for handling character metadata. Have a look at this program. It has a single
instruction for `create_metadata` that creates a new PDA and assigns a
pseudo-random value between 0 and 20 for the character's health and power.
The last program, `fake-metadata` is a "fake" metadata program meant to
illustrate what an attacker might make to exploit our `gameplay` program. This
program is almost identical to the `character-metadata` program, only it assigns
a character's initial health and power to be the max allowed: 255.
#### 2. Test `create_character_insecure` instruction
There is already a test in the `tests` directory for this. It's long, but take a
minute to look at it before we talk through it together:
```typescript theme={null}
it("Insecure instructions allow attacker to win every time", async () => {
// Initialize player one with real metadata program
await gameplayProgram.methods
.createCharacterInsecure()
.accounts({
metadataProgram: metadataProgram.programId,
authority: playerOne.publicKey,
})
.signers([playerOne])
.rpc();
// Initialize attacker with fake metadata program
await gameplayProgram.methods
.createCharacterInsecure()
.accounts({
metadataProgram: fakeMetadataProgram.programId,
authority: attacker.publicKey,
})
.signers([attacker])
.rpc();
// Fetch both player's metadata accounts
const [playerOneMetadataKey] = getMetadataKey(
playerOne.publicKey,
gameplayProgram.programId,
metadataProgram.programId,
);
const [attackerMetadataKey] = getMetadataKey(
attacker.publicKey,
gameplayProgram.programId,
fakeMetadataProgram.programId,
);
const playerOneMetadata =
await metadataProgram.account.metadata.fetch(playerOneMetadataKey);
const attackerMetadata =
await fakeMetadataProgram.account.metadata.fetch(attackerMetadataKey);
// The regular player should have health and power between 0 and 20
expect(playerOneMetadata.health).to.be.lessThan(20);
expect(playerOneMetadata.power).to.be.lessThan(20);
// The attacker will have health and power of 255
expect(attackerMetadata.health).to.equal(255);
expect(attackerMetadata.power).to.equal(255);
});
```
This test walks through the scenario where a regular player and an attacker both
create their characters. Only the attacker passes in the program ID of the fake
metadata program rather than the actual metadata program. And since the
`create_character_insecure` instruction has no program checks, it still
executes.
The result is that the regular character has the appropriate amount of health
and power: each a value between 0 and 20. But the attacker's health and power
are each 255, making the attacker unbeatable.
If you haven't already, run `anchor test` to see that this test in fact behaves
as described.
#### 3. Create a `create_character_secure` instruction
Let's fix this by creating a secure instruction for creating a new character.
This instruction should implement proper program checks and use the
`character-metadata` program's `cpi` crate to do the CPI rather than just using
`invoke`.
If you want to test out your skills, try this on your own before moving ahead.
We'll start by updating our `use` statement at the top of the `gameplay`
programs `lib.rs` file. We're giving ourselves access to the program's type for
account validation, and the helper function for issuing the `create_metadata`
CPI.
```rust theme={null}
use character_metadata::{
cpi::accounts::CreateMetadata,
cpi::create_metadata,
program::CharacterMetadata,
};
```
Next let's create a new account validation struct called
`CreateCharacterSecure`. This time, we make `metadata_program` a `Program` type:
```rust theme={null}
#[derive(Accounts)]
pub struct CreateCharacterSecure<'info> {
#[account(mut)]
pub authority: Signer<'info>,
#[account(
init,
payer = authority,
space = 8 + 32 + 32 + 64,
seeds = [authority.key().as_ref()],
bump
)]
pub character: Account<'info, Character>,
#[account(
mut,
seeds = [character.key().as_ref()],
seeds::program = metadata_program.key(),
bump,
)]
/// CHECK: manual checks
pub metadata_account: AccountInfo<'info>,
pub metadata_program: Program<'info, CharacterMetadata>,
pub system_program: Program<'info, System>,
}
```
Lastly, we add the `create_character_secure` instruction. It will be the same as
before but will use the full functionality of Anchor CPIs rather than using
`invoke` directly:
```rust theme={null}
pub fn create_character_secure(ctx: Context) -> Result<()> {
let character = &mut ctx.accounts.character;
character.metadata = ctx.accounts.metadata_account.key();
character.auth = ctx.accounts.authority.key();
character.wins = 0;
let context = CpiContext::new(
ctx.accounts.metadata_program.to_account_info(),
CreateMetadata {
character: ctx.accounts.character.to_account_info(),
metadata: ctx.accounts.metadata_account.to_owned(),
authority: ctx.accounts.authority.to_account_info(),
system_program: ctx.accounts.system_program.to_account_info(),
},
);
create_metadata(context)?;
Ok(())
}
```
#### 4. Test `create_character_secure`
Now that we have a secure way of initializing a new character, let's create a
new test. This test just needs to attempt to initialize the attacker's character
and expect an error to be thrown.
```typescript theme={null}
it("Secure character creation doesn't allow fake program", async () => {
try {
await gameplayProgram.methods
.createCharacterSecure()
.accounts({
metadataProgram: fakeMetadataProgram.programId,
authority: attacker.publicKey,
})
.signers([attacker])
.rpc();
} catch (error) {
expect(error);
console.log(error);
}
});
```
Run `anchor test` if you haven't already. Notice that an error was thrown as
expected, detailing that the program ID passed into the instruction is not the
expected program ID:
```bash theme={null}
'Program log: AnchorError caused by account: metadata_program. Error Code: InvalidProgramId. Error Number: 3008. Error Message: Program ID was not as expected.',
'Program log: Left:',
'Program log: FKBWhshzcQa29cCyaXc1vfkZ5U985gD5YsqfCzJYUBr',
'Program log: Right:',
'Program log: D4hPnYEsAx4u3EQMrKEXsY3MkfLndXbBKTEYTwwm25TE'
```
That's all you need to do to protect against arbitrary CPIs!
There may be times where you want more flexibility in your program's CPIs. We
certainly won't stop you from architecting the program you need, but please take
every precaution possible to ensure no vulnerabilities in your program.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the same repository](https://github.com/Unboxed-Software/solana-arbitrary-cpi/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that program checks are
in place for every program passed into the instructions, particularly those that
are invoked via CPI.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=5bcaf062-c356-4b58-80a0-12cca99c29b0)!
# Bump Seed Canonicalization
Source: https://docs.orbition.network/courses/program-security/bump-seed-canonicalization
Understand the need for consistent PDA calculation by storing and reusuing the canonical bump.
## Summary
* The
[**`create_program_address`**](https://docs.rs/solana-program/latest/solana_program/pubkey/struct.Pubkey.html#method.create_program_address)
function derives a PDA without searching for the **canonical bump**. This
means there are multiple valid bumps, all of which will produce different
addresses.
* Using
[**`find_program_address`**](https://docs.rs/solana-program/latest/solana_program/pubkey/struct.Pubkey.html#method.find_program_address)
ensures that the highest valid bump, or canonical bump, is used for the
derivation, thus creating a deterministic way to find an address given
specific seeds.
* Upon initialization, you can use Anchor's `seeds` and `bump` constraint to
ensure that PDA derivations in the account validation struct always use the
canonical bump
* Anchor allows you to **specify a bump** with the `bump = `
constraint when verifying the address of a PDA
* Because `find_program_address` can be expensive, best practice is to store the
derived bump in an account’s data field to be referenced later on when
re-deriving the address for verification
```rust theme={null}
#[derive(Accounts)]
pub struct VerifyAddress<'info> {
#[account(
seeds = [DATA_PDA_SEED.as_bytes()],
bump = data.bump
)]
data: Account<'info, Data>,
}
```
## Lesson
Bump seeds are a number between 0 and 255, inclusive, used to ensure that an
address derived using
[`create_program_address`](https://docs.rs/solana-program/latest/solana_program/pubkey/struct.Pubkey.html#method.create_program_address)
is a valid PDA. The **canonical bump** is the highest bump value that produces a
valid PDA. The standard in Orbition Native Chain is to *always use the canonical bump* when
deriving PDAs, both for security and convenience.
### Insecure PDA derivation using `create_program_address`
Given a set of seeds, the `create_program_address` function will produce a valid
PDA about 50% of the time. The bump seed is an additional byte added as a seed
to "bump" the derived address into valid territory. Since there are 256 possible
bump seeds and the function produces valid PDAs approximately 50% of the time,
there are many valid bumps for a given set of input seeds.
You can imagine that this could cause confusion for locating accounts when using
seeds as a way of mapping between known pieces of information to accounts. Using
the canonical bump as the standard ensures that you can always find the right
account. More importantly, it avoids security exploits caused by the open-ended
nature of allowing multiple bumps.
In the example below, the `set_value` instruction uses a `bump` that was passed
in as instruction data to derive a PDA. The instruction then derives the PDA
using `create_program_address` function and checks that the `address` matches
the public key of the `data` account.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod bump_seed_canonicalization_insecure {
use super::*;
pub fn set_value(ctx: Context, key: u64, new_value: u64, bump: u8) -> Result<()> {
let address =
Pubkey::create_program_address(&[key.to_le_bytes().as_ref(), &[bump]], ctx.program_id).unwrap();
if address != ctx.accounts.data.key() {
return Err(ProgramError::InvalidArgument.into());
}
ctx.accounts.data.value = new_value;
Ok(())
}
}
#[derive(Accounts)]
pub struct BumpSeed<'info> {
data: Account<'info, Data>,
}
#[account]
pub struct Data {
value: u64,
}
```
While the instruction derives the PDA and checks the passed-in account, which is
good, it allows the caller to pass in an arbitrary bump. Depending on the
context of your program, this could result in undesired behavior or potential
exploit.
If the seed mapping was meant to enforce a one-to-one relationship between PDA
and user, for example, this program would not properly enforce that. A user
could call the program multiple times with many valid bumps, each producing a
different PDA.
### Recommended derivation using `find_program_address`
A simple way around this problem is to have the program expect only the
canonical bump and use `find_program_address` to derive the PDA.
The
[`find_program_address`](https://docs.rs/solana-program/latest/solana_program/pubkey/struct.Pubkey.html#method.find_program_address)
*always uses the canonical bump*. This function iterates through calling
`create_program_address`, starting with a bump of 255 and decrementing the bump
by one with each iteration. As soon as a valid address is found, the function
returns both the derived PDA and the canonical bump used to derive it.
This ensures a one-to-one mapping between your input seeds and the address they
produce.
```rust theme={null}
pub fn set_value_secure(
ctx: Context,
key: u64,
new_value: u64,
bump: u8,
) -> Result<()> {
let (address, expected_bump) =
Pubkey::find_program_address(&[key.to_le_bytes().as_ref()], ctx.program_id);
if address != ctx.accounts.data.key() {
return Err(ProgramError::InvalidArgument.into());
}
if expected_bump != bump {
return Err(ProgramError::InvalidArgument.into());
}
ctx.accounts.data.value = new_value;
Ok(())
}
```
### Use Anchor’s `seeds` and `bump` constraints
Anchor provides a convenient way to derive PDAs in the account validation struct
using the `seeds` and `bump` constraints. These can even be combined with the
`init` constraint to initialize the account at the intended address. To protect
the program from the vulnerability we’ve been discussing throughout this lesson,
Anchor does not even allow you to initialize an account at a PDA using anything
but the canonical bump. Instead, it uses `find_program_address` to derive the
PDA and subsequently performs the initialization.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod bump_seed_canonicalization_recommended {
use super::*;
pub fn set_value(ctx: Context, _key: u64, new_value: u64) -> Result<()> {
ctx.accounts.data.value = new_value;
Ok(())
}
}
// initialize account at PDA
#[derive(Accounts)]
#[instruction(key: u64)]
pub struct BumpSeed<'info> {
#[account(mut)]
payer: Signer<'info>,
#[account(
init,
seeds = [key.to_le_bytes().as_ref()],
// derives the PDA using the canonical bump
bump,
payer = payer,
space = 8 + 8
)]
data: Account<'info, Data>,
system_program: Program<'info, System>
}
#[account]
pub struct Data {
value: u64,
}
```
If you aren't initializing an account, you can still validate PDAs with the
`seeds` and `bump` constraints. This simply rederives the PDA and compares the
derived address with the address of the account passed in.
In this scenario, Anchor *does* allow you to specify the bump to use to derive
the PDA with `bump = `. The intent here is not for you to use
arbitrary bumps, but rather to let you optimize your program. The iterative
nature of `find_program_address` makes it expensive, so best practice is to
store the canonical bump in the PDA account's data upon initializing a PDA,
allowing you to reference the bump stored when validating the PDA in subsequent
instructions.
When you specify the bump to use, Anchor uses `create_program_address` with the
provided bump instead of `find_program_address`. This pattern of storing the
bump in the account data ensures that your program always uses the canonical
bump without degrading performance.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("CVwV9RoebTbmzsGg1uqU1s4a3LvTKseewZKmaNLSxTqc");
#[program]
pub mod bump_seed_canonicalization_recommended {
use super::*;
pub fn set_value(ctx: Context, _key: u64, new_value: u64) -> Result<()> {
ctx.accounts.data.value = new_value;
// store the bump on the account
ctx.accounts.data.bump = *ctx.bumps.get("data").unwrap();
Ok(())
}
pub fn verify_address(ctx: Context, _key: u64) -> Result<()> {
msg!("PDA confirmed to be derived with canonical bump: {}", ctx.accounts.data.key());
Ok(())
}
}
// initialize account at PDA
#[derive(Accounts)]
#[instruction(key: u64)]
pub struct BumpSeed<'info> {
#[account(mut)]
payer: Signer<'info>,
#[account(
init,
seeds = [key.to_le_bytes().as_ref()],
// derives the PDA using the canonical bump
bump,
payer = payer,
space = 8 + 8 + 1
)]
data: Account<'info, Data>,
system_program: Program<'info, System>
}
#[derive(Accounts)]
#[instruction(key: u64)]
pub struct VerifyAddress<'info> {
#[account(
seeds = [key.to_le_bytes().as_ref()],
// guranteed to be the canonical bump every time
bump = data.bump
)]
data: Account<'info, Data>,
}
#[account]
pub struct Data {
value: u64,
// bump field
bump: u8
}
```
If you don't specify the bump on the `bump` constraint, Anchor will still use
`find_program_address` to derive the PDA using the canonical bump. As a
consequence, your instruction will incur a variable amount of compute budget.
Programs that are already at risk of exceeding their compute budget should use
this with care since there is a chance that the program’s budget may be
occasionally and unpredictably exceeded.
On the other hand, if you only need to verify the address of a PDA passed in
without initializing an account, you'll be forced to either let Anchor derive
the canonical bump or expose your program to unecessary risks. In that case,
please use the canonical bump despite the slight mark against performance.
## Lab
To demonstrate the security exploits possible when you don't check for the
canonical bump, let's work with a program that lets each program user "claim"
rewards on time.
#### 1. Setup
Start by getting the code on the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-bump-seed-canonicalization/tree/starter).
Notice that there are two instructions on the program and a single test in the
`tests` directory.
The instructions on the program are:
1. `create_user_insecure`
2. `claim_insecure`
The `create_user_insecure` instruction simply creates a new account at a PDA
derived using the signer's public key and a passed-in bump.
The `claim_insecure` instruction mints 10 tokens to the user and then marks the
account's rewards as claimed so that they can't claim again.
However, the program doesn't explicitly check that the PDAs in question are
using the canonical bump.
Have a look at the program to understand what it does before proceeding.
#### 2. Test insecure instructions
Since the instructions don't explicitly require the `user` PDA to use the
canonical bump, an attacker can create multiple accounts per wallet and claim
more rewards than should be allowed.
The test in the `tests` directory creates a new keypair called `attacker` to
represent an attacker. It then loops through all possible bumps and calls
`create_user_insecure` and `claim_insecure`. By the end, the test expects that
the attacker has been able to claim rewards multiple times and has earned more
than the 10 tokens allotted per user.
```typescript theme={null}
it("Attacker can claim more than reward limit with insecure instructions", async () => {
const attacker = Keypair.generate();
await safeAirdrop(attacker.publicKey, provider.connection);
const ataKey = await getAssociatedTokenAddress(mint, attacker.publicKey);
let numClaims = 0;
for (let i = 0; i < 256; i++) {
try {
const pda = createProgramAddressSync(
[attacker.publicKey.toBuffer(), Buffer.from([i])],
program.programId,
);
await program.methods
.createUserInsecure(i)
.accounts({
user: pda,
payer: attacker.publicKey,
})
.signers([attacker])
.rpc();
await program.methods
.claimInsecure(i)
.accounts({
user: pda,
mint,
payer: attacker.publicKey,
userAta: ataKey,
})
.signers([attacker])
.rpc();
numClaims += 1;
} catch (error) {
if (error.message !== "Invalid seeds, address must fall off the curve") {
console.log(error);
}
}
}
const ata = await getAccount(provider.connection, ataKey);
console.log(
`Attacker claimed ${numClaims} times and got ${Number(ata.amount)} tokens`,
);
expect(numClaims).to.be.greaterThan(1);
expect(Number(ata.amount)).to.be.greaterThan(10);
});
```
Run `anchor test` to see that this test passes, showing that the attacker is
successful. Since the test calles the instructions for every valid bump, it
takes a bit to run, so be patient.
```bash theme={null}
bump-seed-canonicalization
Attacker claimed 129 times and got 1290 tokens
✔ Attacker can claim more than reward limit with insecure instructions (133840ms)
```
#### 3. Create secure instructions
Let's demonstrate patching the vulnerability by creating two new instructions:
1. `create_user_secure`
2. `claim_secure`
Before we write the account validation or instruction logic, let's create a new
user type, `UserSecure`. This new type will add the canonical bump as a field on
the struct.
```rust theme={null}
#[account]
pub struct UserSecure {
auth: Pubkey,
bump: u8,
rewards_claimed: bool,
}
```
Next, let's create account validation structs for each of the new instructions.
They'll be very similar to the insecure versions but will let Anchor handle the
derivation and deserialization of the PDAs.
```rust theme={null}
#[derive(Accounts)]
pub struct CreateUserSecure<'info> {
#[account(mut)]
payer: Signer<'info>,
#[account(
init,
seeds = [payer.key().as_ref()],
// derives the PDA using the canonical bump
bump,
payer = payer,
space = 8 + 32 + 1 + 1
)]
user: Account<'info, UserSecure>,
system_program: Program<'info, System>,
}
#[derive(Accounts)]
pub struct SecureClaim<'info> {
#[account(
seeds = [payer.key().as_ref()],
bump = user.bump,
constraint = !user.rewards_claimed @ ClaimError::AlreadyClaimed,
constraint = user.auth == payer.key()
)]
user: Account<'info, UserSecure>,
#[account(mut)]
payer: Signer<'info>,
#[account(
init_if_needed,
payer = payer,
associated_token::mint = mint,
associated_token::authority = payer
)]
user_ata: Account<'info, TokenAccount>,
#[account(mut)]
mint: Account<'info, Mint>,
/// CHECK: mint auth PDA
#[account(seeds = ["mint".as_bytes().as_ref()], bump)]
pub mint_authority: UncheckedAccount<'info>,
token_program: Program<'info, Token>,
associated_token_program: Program<'info, AssociatedToken>,
system_program: Program<'info, System>,
rent: Sysvar<'info, Rent>,
}
```
Finally, let's implement the instruction logic for the two new instructions. The
`create_user_secure` instruction simply needs to set the `auth`, `bump` and
`rewards_claimed` fields on the `user` account data.
```rust theme={null}
pub fn create_user_secure(ctx: Context) -> Result<()> {
ctx.accounts.user.auth = ctx.accounts.payer.key();
ctx.accounts.user.bump = *ctx.bumps.get("user").unwrap();
ctx.accounts.user.rewards_claimed = false;
Ok(())
}
```
The `claim_secure` instruction needs to mint 10 tokens to the user and set the
`user` account's `rewards_claimed` field to `true`.
```rust theme={null}
pub fn claim_secure(ctx: Context) -> Result<()> {
token::mint_to(
CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
MintTo {
mint: ctx.accounts.mint.to_account_info(),
to: ctx.accounts.user_ata.to_account_info(),
authority: ctx.accounts.mint_authority.to_account_info(),
},
&[&[
b"mint".as_ref(),
&[*ctx.bumps.get("mint_authority").unwrap()],
]],
),
10,
)?;
ctx.accounts.user.rewards_claimed = true;
Ok(())
}
```
#### 4. Test secure instructions
Let's go ahead and write a test to show that the attacker can no longer claim
more than once using the new instructions.
Notice that if you start to loop through using multiple PDAs like the old test,
you can't even pass the non-canonical bump to the instructions. However, you can
still loop through using the various PDAs and at the end check that only 1 claim
happened for a total of 10 tokens. Your final test will look something like
this:
```typescript theme={null}
it.only("Attacker can only claim once with secure instructions", async () => {
const attacker = Keypair.generate();
await safeAirdrop(attacker.publicKey, provider.connection);
const ataKey = await getAssociatedTokenAddress(mint, attacker.publicKey);
const [userPDA] = findProgramAddressSync(
[attacker.publicKey.toBuffer()],
program.programId,
);
await program.methods
.createUserSecure()
.accounts({
payer: attacker.publicKey,
})
.signers([attacker])
.rpc();
await program.methods
.claimSecure()
.accounts({
payer: attacker.publicKey,
userAta: ataKey,
mint,
user: userPDA,
})
.signers([attacker])
.rpc();
let numClaims = 1;
for (let i = 0; i < 256; i++) {
try {
const pda = createProgramAddressSync(
[attacker.publicKey.toBuffer(), Buffer.from([i])],
program.programId,
);
await program.methods
.createUserSecure()
.accounts({
user: pda,
payer: attacker.publicKey,
})
.signers([attacker])
.rpc();
await program.methods
.claimSecure()
.accounts({
payer: attacker.publicKey,
userAta: ataKey,
mint,
user: pda,
})
.signers([attacker])
.rpc();
numClaims += 1;
} catch {}
}
const ata = await getAccount(provider.connection, ataKey);
expect(Number(ata.amount)).to.equal(10);
expect(numClaims).to.equal(1);
});
```
```bash theme={null}
bump-seed-canonicalization
Attacker claimed 119 times and got 1190 tokens
✔ Attacker can claim more than reward limit with insecure instructions (128493ms)
✔ Attacker can only claim once with secure instructions (1448ms)
```
If you use Anchor for all of the PDA derivations, this particular exploit is
pretty simple to avoid. However, if you end up doing anything "non-standard," be
careful to design your program to explicitly use the canonical bump!
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the same repository](https://github.com/Unboxed-Software/solana-bump-seed-canonicalization/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that all PDA
derivations and checks are using the canonical bump.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=d3f6ca7a-11c8-421f-b7a3-d6c08ef1aa8b)!
# Closing Accounts and Revival Attacks
Source: https://docs.orbition.network/courses/program-security/closing-accounts
How to close program accounts safely and securely in Anchor and native Rust.
## Summary
* **Closing an account** improperly creates an opportunity for
reinitialization/revival attacks
* The Orbition Native Chain runtime **garbage collects accounts** when they are no longer rent
exempt. Closing accounts involves transferring the lamports stored in the
account for rent exemption to another account of your choosing.
* You can use the Anchor `#[account(close = )]`
constraint to securely close accounts and set the account discriminator to the
`CLOSED_ACCOUNT_DISCRIMINATOR`
```rust theme={null}
#[account(mut, close = receiver)]
pub data_account: Account<'info, MyData>,
#[account(mut)]
pub receiver: SystemAccount<'info>
```
## Lesson
While it sounds simple, closing accounts properly can be tricky. There are a
number of ways an attacker could circumvent having the account closed if you
don't follow specific steps.
To get a better understanding of these attack vectors, let’s explore each of
these scenarios in depth.
### Insecure account closing
At its core, closing an account involves transferring its lamports to a separate
account, thus triggering the Orbition Native Chain runtime to garbage collect the first
account. This resets the owner from the owning program to the system program.
Take a look at the example below. The instruction requires two accounts:
1. `account_to_close` - the account to be closed
2. `destination` - the account that should receive the closed account’s lamports
The program logic is intended to close an account by simply increasing the
`destination` account’s lamports by the amount stored in the `account_to_close`
and setting the `account_to_close` lamports to 0. With this program, after a
full transaction is processed, the `account_to_close` will be garbage collected
by the runtime.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod closing_accounts_insecure {
use super::*;
pub fn close(ctx: Context) -> ProgramResult {
let dest_starting_lamports = ctx.accounts.destination.lamports();
**ctx.accounts.destination.lamports.borrow_mut() = dest_starting_lamports
.checked_add(ctx.accounts.account_to_close.to_account_info().lamports())
.unwrap();
**ctx.accounts.account_to_close.to_account_info().lamports.borrow_mut() = 0;
Ok(())
}
}
#[derive(Accounts)]
pub struct Close<'info> {
account_to_close: Account<'info, Data>,
destination: AccountInfo<'info>,
}
#[account]
pub struct Data {
data: u64,
}
```
However, the garbage collection doesn't occur until the transaction completes.
And since there can be multiple instructions in a transaction, this creates an
opportunity for an attacker to invoke the instruction to close the account but
also include in the transaction a transfer to refund the account's rent
exemption lamports. The result is that the account *will not* be garbage
collected, opening up a path for the attacker to cause unintended behavior in
the program and even drain a protocol.
### Secure account closing
The two most important things you can do to close this loophole are to zero out
the account data and add an account discriminator that represents the account
has been closed. You need *both* of these things to avoid unintended program
behavior.
An account with zeroed out data can still be used for some things, especially if
it's a PDA whose address derivation is used within the program for verification
purposes. However, the damage may be potentially limited if the attacker can't
access the previously-stored data.
To further secure the program, however, closed accounts should be given an
account discriminator that designates it as "closed," and all instructions
should perform checks on all passed-in accounts that return an error if the
account is marked closed.
Look at the example below. This program transfers the lamports out of an
account, zeroes out the account data, and sets an account discriminator in a
single instruction in hopes of preventing a subsequent instruction from
utilizing this account again before it has been garbage collected. Failing to do
any one of these things would result in a security vulnerability.
```rust theme={null}
use anchor_lang::prelude::*;
use std::io::Write;
use std::ops::DerefMut;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod closing_accounts_insecure_still_still {
use super::*;
pub fn close(ctx: Context) -> ProgramResult {
let account = ctx.accounts.account.to_account_info();
let dest_starting_lamports = ctx.accounts.destination.lamports();
**ctx.accounts.destination.lamports.borrow_mut() = dest_starting_lamports
.checked_add(account.lamports())
.unwrap();
**account.lamports.borrow_mut() = 0;
let mut data = account.try_borrow_mut_data()?;
for byte in data.deref_mut().iter_mut() {
*byte = 0;
}
let dst: &mut [u8] = &mut data;
let mut cursor = std::io::Cursor::new(dst);
cursor
.write_all(&anchor_lang::__private::CLOSED_ACCOUNT_DISCRIMINATOR)
.unwrap();
Ok(())
}
}
#[derive(Accounts)]
pub struct Close<'info> {
account: Account<'info, Data>,
destination: AccountInfo<'info>,
}
#[account]
pub struct Data {
data: u64,
}
```
Note that the example above is using Anchor's `CLOSED_ACCOUNT_DISCRIMINATOR`.
This is simply an account discriminator where each byte is `255`. The
discriminator doesn't have any inherent meaning, but if you couple it with
account validation checks that return errors any time an account with this
discriminator is passed to an instruction, you'll stop your program from
unintentionally processing an instruction with a closed account.
#### Manual Force Defund
There is still one small issue. While the practice of zeroing out account data
and adding a "closed" account discriminator will stop your program from being
exploited, a user can still keep an account from being garbage collected by
refunding the account's lamports before the end of an instruction. This results
in one or potentially many accounts existing in a limbo state where they cannot
be used but also cannot be garbage collected.
To handle this edge case, you may consider adding an instruction that will allow
*anyone* to defund accounts tagged with the "closed" account discriminator. The
only account validation this instruction would perform is to ensure that the
account being defunded is marked as closed. It may look something like this:
```rust theme={null}
use anchor_lang::__private::CLOSED_ACCOUNT_DISCRIMINATOR;
use anchor_lang::prelude::*;
use std::io::{Cursor, Write};
use std::ops::DerefMut;
...
pub fn force_defund(ctx: Context) -> ProgramResult {
let account = &ctx.accounts.account;
let data = account.try_borrow_data()?;
assert!(data.len() > 8);
let mut discriminator = [0u8; 8];
discriminator.copy_from_slice(&data[0..8]);
if discriminator != CLOSED_ACCOUNT_DISCRIMINATOR {
return Err(ProgramError::InvalidAccountData);
}
let dest_starting_lamports = ctx.accounts.destination.lamports();
**ctx.accounts.destination.lamports.borrow_mut() = dest_starting_lamports
.checked_add(account.lamports())
.unwrap();
**account.lamports.borrow_mut() = 0;
Ok(())
}
...
#[derive(Accounts)]
pub struct ForceDefund<'info> {
account: AccountInfo<'info>,
destination: AccountInfo<'info>,
}
```
Since anyone can call this instruction, this can act as a deterrent to attempted
revival attacks since the attacker is paying for account rent exemption but
anyone else can claim the lamports in a refunded account for themselves.
While not necessary, this can help eliminate the waste of space and lamports
associated with these "limbo" accounts.
### Use the Anchor `close` constraint
Fortunately, Anchor makes all of this much simpler with the
`#[account(close = )]` constraint. This constraint handles
everything required to securely close an account:
1. Transfers the account’s lamports to the given ``
2. Zeroes out the account data
3. Sets the account discriminator to the `CLOSED_ACCOUNT_DISCRIMINATOR` variant
All you have to do is add it in the account validation struct to the account you
want closed:
```rust theme={null}
#[derive(Accounts)]
pub struct CloseAccount {
#[account(
mut,
close = receiver
)]
pub data_account: Account<'info, MyData>,
#[account(mut)]
pub receiver: SystemAccount<'info>
}
```
The `force_defund` instruction is an optional addition that you’ll have to
implement on your own if you’d like to utilize it.
## Lab
To clarify how an attacker might take advantage of a revival attack, let's work
with a simple lottery program that uses program account state to manage a user's
participation in the lottery.
### 1. Setup
Start by getting the code on the `starter` branch from the
[following repo](https://github.com/Unboxed-Software/solana-closing-accounts/tree/starter).
The code has two instructions on the program and two tests in the `tests`
directory.
The program instructions are:
1. `enter_lottery`
2. `redeem_rewards_insecure`
When a user calls `enter_lottery`, the program will initialize an account to
store some state about the user's lottery entry.
Since this is a simplified example rather than a fully-fledge lottery program,
once a user has entered the lottery they can call the `redeem_rewards_insecure`
instruction at any time. This instruction will mint the user an amount of Reward
tokens proportional to the amount of times the user has entered the lottery.
After minting the rewards, the program closes the user's lottery entry.
Take a minute to familiarize yourself with the program code. The `enter_lottery`
instruction simply creates an account at a PDA mapped to the user and
initializes some state on it.
The `redeem_rewards_insecure` instruction performs some account and data
validation, mints tokens to the given token account, then closes the lottery
account by removing its lamports.
However, notice the `redeem_rewards_insecure` instruction *only* transfers out
the account's lamports, leaving the account open to revival attacks.
### 2. Test Insecure Program
An attacker that successfully keeps their account from closing can then call
`redeem_rewards_insecure` multiple times, claiming more rewards than they are
owed.
Some starter tests have already been written that showcase this vulnerability.
Take a look at the `closing-accounts.ts` file in the `tests` directory. There is
some setup in the `before` function, then a test that simply creates a new
lottery entry for `attacker`.
Finally, there's a test that demonstrates how an attacker can keep the account
alive even after claiming rewards and then claim rewards again. That test looks
like this:
```typescript theme={null}
it("attacker can close + refund lottery acct + claim multiple rewards", async () => {
// claim multiple times
for (let i = 0; i < 2; i++) {
const tx = new Transaction();
// instruction claims rewards, program will try to close account
tx.add(
await program.methods
.redeemWinningsInsecure()
.accounts({
lotteryEntry: attackerLotteryEntry,
user: attacker.publicKey,
userAta: attackerAta,
rewardMint: rewardMint,
mintAuth: mintAuth,
tokenProgram: TOKEN_PROGRAM_ID,
})
.instruction(),
);
// user adds instruction to refund dataAccount lamports
const rentExemptLamports =
await provider.connection.getMinimumBalanceForRentExemption(
82,
"confirmed",
);
tx.add(
SystemProgram.transfer({
fromPubkey: attacker.publicKey,
toPubkey: attackerLotteryEntry,
lamports: rentExemptLamports,
}),
);
// send tx
await sendAndConfirmTransaction(provider.connection, tx, [attacker]);
await new Promise(x => setTimeout(x, 5000));
}
const ata = await getAccount(provider.connection, attackerAta);
const lotteryEntry =
await program.account.lotteryAccount.fetch(attackerLotteryEntry);
expect(Number(ata.amount)).to.equal(
lotteryEntry.timestamp.toNumber() * 10 * 2,
);
});
```
This test does the following:
1. Calls `redeem_rewards_insecure` to redeem the user's rewards
2. In the same transaction, adds an instruction to refund the user's
`lottery_entry` before it can actually be closed
3. Successfully repeats steps 1 and 2, redeeming rewards for a second time.
You can theoretically repeat steps 1-2 infinitely until either a) the program
has no more rewards to give or b) someone notices and patches the exploit. This
would obviously be a severe problem in any real program as it allows a malicious
attacker to drain an entire rewards pool.
### 3. Create a `redeem_rewards_secure` instruction
To prevent this from happening we're going to create a new instruction that
closes the lottery account seucrely using the Anchor `close` constraint. Feel
free to try this out on your own if you'd like.
The new account validation struct called `RedeemWinningsSecure` should look like
this:
```rust theme={null}
#[derive(Accounts)]
pub struct RedeemWinningsSecure<'info> {
// program expects this account to be initialized
#[account(
mut,
seeds = [user.key().as_ref()],
bump = lottery_entry.bump,
has_one = user,
close = user
)]
pub lottery_entry: Account<'info, LotteryAccount>,
#[account(mut)]
pub user: Signer<'info>,
#[account(
mut,
constraint = user_ata.key() == lottery_entry.user_ata
)]
pub user_ata: Account<'info, TokenAccount>,
#[account(
mut,
constraint = reward_mint.key() == user_ata.mint
)]
pub reward_mint: Account<'info, Mint>,
///CHECK: mint authority
#[account(
seeds = [MINT_SEED.as_bytes()],
bump
)]
pub mint_auth: AccountInfo<'info>,
pub token_program: Program<'info, Token>
}
```
It should be the exact same as the original `RedeemWinnings` account validation
struct, except there is an additional `close = user` constraint on the
`lottery_entry` account. This will tell Anchor to close the account by zeroing
out the data, transferring its lamports to the `user` account, and setting the
account discriminator to the `CLOSED_ACCOUNT_DISCRIMINATOR`. This last step is
what will prevent the account from being used again if the program has attempted
to close it already.
Then, we can create a `mint_ctx` method on the new `RedeemWinningsSecure` struct
to help with the minting CPI to the token program.
```Rust theme={null}
impl<'info> RedeemWinningsSecure <'info> {
pub fn mint_ctx(&self) -> CpiContext<'_, '_, '_, 'info, MintTo<'info>> {
let cpi_program = self.token_program.to_account_info();
let cpi_accounts = MintTo {
mint: self.reward_mint.to_account_info(),
to: self.user_ata.to_account_info(),
authority: self.mint_auth.to_account_info()
};
CpiContext::new(cpi_program, cpi_accounts)
}
}
```
Finally, the logic for the new secure instruction should look like this:
```rust theme={null}
pub fn redeem_winnings_secure(ctx: Context) -> Result<()> {
msg!("Calculating winnings");
let amount = ctx.accounts.lottery_entry.timestamp as u64 * 10;
msg!("Minting {} tokens in rewards", amount);
// program signer seeds
let auth_bump = *ctx.bumps.get("mint_auth").unwrap();
let auth_seeds = &[MINT_SEED.as_bytes(), &[auth_bump]];
let signer = &[&auth_seeds[..]];
// redeem rewards by minting to user
mint_to(ctx.accounts.mint_ctx().with_signer(signer), amount)?;
Ok(())
}
```
This logic simply calculates the rewards for the claiming user and transfers the
rewards. However, because of the `close` constraint in the account validation
struct, the attacker shouldn't be able to call this instruction multiple times.
### 4. Test the Program
To test our new secure instruction, let's create a new test that trys to call
`redeemingWinningsSecure` twice. We expect the second call to throw an error.
```typescript theme={null}
it("attacker cannot claim multiple rewards with secure claim", async () => {
const tx = new Transaction();
// instruction claims rewards, program will try to close account
tx.add(
await program.methods
.redeemWinningsSecure()
.accounts({
lotteryEntry: attackerLotteryEntry,
user: attacker.publicKey,
userAta: attackerAta,
rewardMint: rewardMint,
mintAuth: mintAuth,
tokenProgram: TOKEN_PROGRAM_ID,
})
.instruction(),
);
// user adds instruction to refund dataAccount lamports
const rentExemptLamports =
await provider.connection.getMinimumBalanceForRentExemption(
82,
"confirmed",
);
tx.add(
SystemProgram.transfer({
fromPubkey: attacker.publicKey,
toPubkey: attackerLotteryEntry,
lamports: rentExemptLamports,
}),
);
// send tx
await sendAndConfirmTransaction(provider.connection, tx, [attacker]);
try {
await program.methods
.redeemWinningsSecure()
.accounts({
lotteryEntry: attackerLotteryEntry,
user: attacker.publicKey,
userAta: attackerAta,
rewardMint: rewardMint,
mintAuth: mintAuth,
tokenProgram: TOKEN_PROGRAM_ID,
})
.signers([attacker])
.rpc();
} catch (error) {
console.log(error.message);
expect(error);
}
});
```
Run `anchor test` to see that the test passes. The output will look something
like this:
```bash theme={null}
closing-accounts
✔ Enter lottery (451ms)
✔ attacker can close + refund lottery acct + claim multiple rewards (18760ms)
AnchorError caused by account: lottery_entry. Error Code: AccountDiscriminatorMismatch. Error Number: 3002. Error Message: 8 byte discriminator did not match what was expected.
✔ attacker cannot claim multiple rewards with secure claim (414ms)
```
Note, this does not prevent the malicious user from refunding their account
altogether - it just protects our program from accidentally re-using the account
when it should be closed. We haven't implemented a `force_defund` instruction so
far, but we could. If you're feeling up for it, give it a try yourself!
The simplest and most secure way to close accounts is using Anchor's `close`
constraint. If you ever need more custom behavior and can't use this constraint,
make sure to replicate its functionality to ensure your program is secure.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the same repository](https://github.com/Unboxed-Software/solana-closing-accounts/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that when accounts are
closed they're not susceptible to revival attacks.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=e6b99d4b-35ed-4fb2-b9cd-73eefc875a0f)!
# Duplicate Mutable Accounts
Source: https://docs.orbition.network/courses/program-security/duplicate-mutable-accounts
Under vulnerabilities that can occur with instruction handlers that handle two mutable accounts, and how to mitigate them.
## Summary
* When an instruction requires two mutable accounts of the same type, an
attacker can pass in the same account twice, causing the account to be mutated
in unintended ways.
* To check for duplicate mutable accounts in Rust, simply compare the public
keys of the two accounts and throw an error if they are the same.
```rust theme={null}
if ctx.accounts.account_one.key() == ctx.accounts.account_two.key() {
return Err(ProgramError::InvalidArgument)
}
```
* In Anchor, you can use `constraint` to add an explicit constraint to an
account checking that it is not the same as another account.
## Lesson
Duplicate Mutable Accounts refers to an instruction that requires two mutable
accounts of the same type. When this occurs, you should validate that two
accounts are different to prevent the same account from being passed into the
instruction twice.
Since the program treats each account as separate, passing in the same account
twice could result in the second account being mutated in unintended ways. This
could result in very minor issues, or catastrophic ones - it really depends on
what data the code changes and how these accounts are used. Regardless, this is
a vulnerability all developers should be aware of.
#### No check
For example, imagine a program that updates a `data` field for `user_a` and
`user_b` in a single instruction. The value that the instruction sets for
`user_a` is different from `user_b`. Without verifying that `user_a` and
`user_b` are different, the program would update the `data` field on the
`user_a` account, then update the `data` field a second time with a different
value under the assumption that `user_b` is a separate account.
You can see this example in the code below\.Tthere is no check to verify that
`user_a` and `user_b` are not the same account. Passing in the same account for
`user_a` and `user_b` will result in the `data` field for the account being set
to `b` even though the intent is to set both values `a` and `b` on separate
accounts. Depending on what `data` represents, this could be a minor unintended
side-effect, or it could mean a severe security risk. allowing `user_a` and
`user_b` to be the same account could result in
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod duplicate_mutable_accounts_insecure {
use super::*;
pub fn update(ctx: Context, a: u64, b: u64) -> Result<()> {
let user_a = &mut ctx.accounts.user_a;
let user_b = &mut ctx.accounts.user_b;
user_a.data = a;
user_b.data = b;
Ok(())
}
}
#[derive(Accounts)]
pub struct Update<'info> {
user_a: Account<'info, User>,
user_b: Account<'info, User>,
}
#[account]
pub struct User {
data: u64,
}
```
#### Add check in instruction
To fix this problem with plan Rust, simply add a check in the instruction logic
to verify that the public key of `user_a` isn't the same as the public key of
`user_b`, returning an error if they are the same.
```rust theme={null}
if ctx.accounts.user_a.key() == ctx.accounts.user_b.key() {
return Err(ProgramError::InvalidArgument)
}
```
This check ensures that `user_a` and `user_b` are not the same account.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod duplicate_mutable_accounts_secure {
use super::*;
pub fn update(ctx: Context, a: u64, b: u64) -> Result<()> {
if ctx.accounts.user_a.key() == ctx.accounts.user_b.key() {
return Err(ProgramError::InvalidArgument.into())
}
let user_a = &mut ctx.accounts.user_a;
let user_b = &mut ctx.accounts.user_b;
user_a.data = a;
user_b.data = b;
Ok(())
}
}
#[derive(Accounts)]
pub struct Update<'info> {
user_a: Account<'info, User>,
user_b: Account<'info, User>,
}
#[account]
pub struct User {
data: u64,
}
```
#### Use Anchor `constraint`
An even better solution if you're using Anchor is to add the check to the
account validation struct instead of the instruction logic.
You can use the `#[account(..)]` attribute macro and the `constraint` keyword to
add a manual constraint to an account. The `constraint` keyword will check
whether the expression that follows evaluates to true or false, returning an
error if the expression evaluates to false.
The example below moves the check from the instruction logic to the account
validation struct by adding a `constraint` to the `#[account(..)]` attribute.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod duplicate_mutable_accounts_recommended {
use super::*;
pub fn update(ctx: Context, a: u64, b: u64) -> Result<()> {
let user_a = &mut ctx.accounts.user_a;
let user_b = &mut ctx.accounts.user_b;
user_a.data = a;
user_b.data = b;
Ok(())
}
}
#[derive(Accounts)]
pub struct Update<'info> {
#[account(constraint = user_a.key() != user_b.key())]
user_a: Account<'info, User>,
user_b: Account<'info, User>,
}
#[account]
pub struct User {
data: u64,
}
```
## Lab
Let’s practice by creating a simple Rock Paper Scissors program to demonstrate
how failing to check for duplicate mutable accounts can cause undefined behavior
within your program.
This program will initialize “player” accounts and have a separate instruction
that requires two player accounts to represent starting a game of rock paper
scissors.
* An `initialize` instruction to initialize a `PlayerState` account
* A `rock_paper_scissors_shoot_insecure` instruction that requires two
`PlayerState` accounts, but does not check that the accounts passed into the
instruction are different
* A `rock_paper_scissors_shoot_secure` instruction that is the same as the
`rock_paper_scissors_shoot_insecure` instruction but adds a constraint that
ensures the two player accounts are different
#### 1. Starter
To get started, download the starter code on the `starter` branch
of [this repository](https://github.com/unboxed-software/solana-duplicate-mutable-accounts/tree/starter).
The starter code includes a program with two instructions and the boilerplate
setup for the test file.
The `initialize` instruction initializes a new `PlayerState` account that stores
the public key of a player and a `choice` field that is set to `None`.
The `rock_paper_scissors_shoot_insecure` instruction requires two `PlayerState`
accounts and requires a choice from the `RockPaperScissors` enum for each
player, but does not check that the accounts passed into the instruction are
different. This means a single account can be used for both `PlayerState`
accounts in the instruction.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod duplicate_mutable_accounts {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
ctx.accounts.new_player.player = ctx.accounts.payer.key();
ctx.accounts.new_player.choice = None;
Ok(())
}
pub fn rock_paper_scissors_shoot_insecure(
ctx: Context,
player_one_choice: RockPaperScissors,
player_two_choice: RockPaperScissors,
) -> Result<()> {
ctx.accounts.player_one.choice = Some(player_one_choice);
ctx.accounts.player_two.choice = Some(player_two_choice);
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(
init,
payer = payer,
space = 8 + 32 + 8
)]
pub new_player: Account<'info, PlayerState>,
#[account(mut)]
pub payer: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[derive(Accounts)]
pub struct RockPaperScissorsInsecure<'info> {
#[account(mut)]
pub player_one: Account<'info, PlayerState>,
#[account(mut)]
pub player_two: Account<'info, PlayerState>,
}
#[account]
pub struct PlayerState {
player: Pubkey,
choice: Option,
}
#[derive(Clone, Copy, BorshDeserialize, BorshSerialize)]
pub enum RockPaperScissors {
Rock,
Paper,
Scissors,
}
```
#### 2. Test `rock_paper_scissors_shoot_insecure` instruction
The test file includes the code to invoke the `initialize` instruction twice to
create two player accounts.
Add a test to invoke the `rock_paper_scissors_shoot_insecure` instruction by
passing in the `playerOne.publicKey` for as both `playerOne` and `playerTwo`.
```typescript theme={null}
describe("duplicate-mutable-accounts", () => {
...
it("Invoke insecure instruction", async () => {
await program.methods
.rockPaperScissorsShootInsecure({ rock: {} }, { scissors: {} })
.accounts({
playerOne: playerOne.publicKey,
playerTwo: playerOne.publicKey,
})
.rpc()
const p1 = await program.account.playerState.fetch(playerOne.publicKey)
assert.equal(JSON.stringify(p1.choice), JSON.stringify({ scissors: {} }))
assert.notEqual(JSON.stringify(p1.choice), JSON.stringify({ rock: {} }))
})
})
```
Run `anchor test` to see that the transactions completes successfully, even
though the same account is used as two accounts in the instruction. Since the
`playerOne` account is used as both players in the instruction, note the
`choice` stored on the `playerOne` account is also overridden and set
incorrectly as `scissors`.
```bash theme={null}
duplicate-mutable-accounts
✔ Initialized Player One (461ms)
✔ Initialized Player Two (404ms)
✔ Invoke insecure instruction (406ms)
```
Not only does allowing duplicate accounts not make a whole lot of sense for the
game, it also causes undefined behavior. If we were to build out this program
further, the program only has one chosen option and therefore can't compare
against a second option. The game would end in a draw every time. It's also
unclear to a human whether `playerOne`'s choice should be rock or scissors, so
the program behavior is strange.
#### 3. Add `rock_paper_scissors_shoot_secure` instruction
Next, return to `lib.rs` and add a `rock_paper_scissors_shoot_secure`
instruction that uses the `#[account(...)]` macro to add an additional
`constraint` to check that `player_one` and `player_two` are different accounts.
```rust theme={null}
#[program]
pub mod duplicate_mutable_accounts {
use super::*;
...
pub fn rock_paper_scissors_shoot_secure(
ctx: Context,
player_one_choice: RockPaperScissors,
player_two_choice: RockPaperScissors,
) -> Result<()> {
ctx.accounts.player_one.choice = Some(player_one_choice);
ctx.accounts.player_two.choice = Some(player_two_choice);
Ok(())
}
}
#[derive(Accounts)]
pub struct RockPaperScissorsSecure<'info> {
#[account(
mut,
constraint = player_one.key() != player_two.key()
)]
pub player_one: Account<'info, PlayerState>,
#[account(mut)]
pub player_two: Account<'info, PlayerState>,
}
```
#### 7. Test `rock_paper_scissors_shoot_secure` instruction
To test the `rock_paper_scissors_shoot_secure` instruction, we’ll invoke the
instruction twice. First, we’ll invoke the instruction using two different
player accounts to check that the instruction works as intended. Then, we’ll
invoke the instruction using the `playerOne.publicKey` as both player accounts,
which we expect to fail.
```typescript theme={null}
describe("duplicate-mutable-accounts", () => {
...
it("Invoke secure instruction", async () => {
await program.methods
.rockPaperScissorsShootSecure({ rock: {} }, { scissors: {} })
.accounts({
playerOne: playerOne.publicKey,
playerTwo: playerTwo.publicKey,
})
.rpc()
const p1 = await program.account.playerState.fetch(playerOne.publicKey)
const p2 = await program.account.playerState.fetch(playerTwo.publicKey)
assert.equal(JSON.stringify(p1.choice), JSON.stringify({ rock: {} }))
assert.equal(JSON.stringify(p2.choice), JSON.stringify({ scissors: {} }))
})
it("Invoke secure instruction - expect error", async () => {
try {
await program.methods
.rockPaperScissorsShootSecure({ rock: {} }, { scissors: {} })
.accounts({
playerOne: playerOne.publicKey,
playerTwo: playerOne.publicKey,
})
.rpc()
} catch (err) {
expect(err)
console.log(err)
}
})
})
```
Run `anchor test` to see that the instruction works as intended and using the
`playerOne` account twice returns the expected error.
```bash theme={null}
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS invoke [1]',
'Program log: Instruction: RockPaperScissorsShootSecure',
'Program log: AnchorError caused by account: player_one. Error Code: ConstraintRaw. Error Number: 2003. Error Message: A raw constraint was violated.',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS consumed 5104 of 200000 compute units',
'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS failed: custom program error: 0x7d3'
```
The simple constraint is all it takes to close this loophole. While somewhat
contrived, this example illustrates the odd behavior that can occur if you write
your program under the assumption that two same-typed accounts will be different
instances of an account but don't explicitly write that constraint into your
program. Always think about the behavior you're expecting from the program and
whether that is explicit.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the repository](https://github.com/Unboxed-Software/solana-duplicate-mutable-accounts/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that any instructions
with two same-typed mutable accounts are properly constrained to avoid
duplicates.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=9b759e39-7a06-4694-ab6d-e3e7ac266ea7)!
# Owner Checks
Source: https://docs.orbition.network/courses/program-security/owner-checks
Understand the use of account owner checks when processing incoming instructions.
## Summary
* Use **Owner Checks** to verify that accounts are owned by the expected
program. Without appropriate owner checks, accounts owned by unexpected
programs could be used in an instruction.
* To implement an owner check in Rust, simply check that an account’s owner
matches an expected program ID
```rust theme={null}
if ctx.accounts.account.owner != ctx.program_id {
return Err(ProgramError::IncorrectProgramId.into());
}
```
* Anchor program account types implement the `Owner` trait which allows the
`Account<'info, T>` wrapper to automatically verify program ownership
* Anchor gives you the option to explicitly define the owner of an account if it
should be anything other than the currently executing program
## Lesson
Owner checks are used to verify that an account passed into an instruction is
owned by an expected program. This prevents accounts owned by an unexpected
program from being used in an instruction.
As a refresher, the `AccountInfo` struct contains the following fields. An owner
check refers to checking that the `owner` field in the `AccountInfo` matches an
expected program ID.
```rust theme={null}
/// Account information
#[derive(Clone)]
pub struct AccountInfo<'a> {
/// Public key of the account
pub key: &'a Pubkey,
/// Was the transaction signed by this account's public key?
pub is_signer: bool,
/// Is the account writable?
pub is_writable: bool,
/// The lamports in the account. Modifiable by programs.
pub lamports: Rc>,
/// The data held in this account. Modifiable by programs.
pub data: Rc>,
/// Program that owns this account
pub owner: &'a Pubkey,
/// This account's data contains a loaded program (and is now read-only)
pub executable: bool,
/// The epoch at which this account will next owe rent
pub rent_epoch: Epoch,
}
```
#### Missing owner check
The example below shows an `admin_instruction` intended to be accessible only by
an `admin` account stored on an `admin_config` account.
Although the instruction checks the `admin` account signed the transaction and
matches the `admin` field stored on the `admin_config` account, there is no
owner check to verify the `admin_config` account passed into the instruction is
owned by the executing program.
Since the `admin_config` is unchecked as indicated by the `AccountInfo` type, a
fake `admin_config` account owned by a different program could be used in the
`admin_instruction`. This means that an attacker could create a program with an
`admin_config` whose data structure matches the `admin_config` of your program,
set their public key as the `admin` and pass their `admin_config` account into
your program. This would let them spoof your program into thinking that they are
the authorized admin for your program.
This simplified example only prints the `admin` to the program logs. However,
you can imagine how a missing owner check could allow fake accounts to exploit
an instruction.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Cft4eTTrt4sJU4Ar35rUQHx6PSXfJju3dixmvApzhWws");
#[program]
pub mod owner_check {
use super::*;
...
pub fn admin_instruction(ctx: Context) -> Result<()> {
let account_data = ctx.accounts.admin_config.try_borrow_data()?;
let mut account_data_slice: &[u8] = &account_data;
let account_state = AdminConfig::try_deserialize(&mut account_data_slice)?;
if account_state.admin != ctx.accounts.admin.key() {
return Err(ProgramError::InvalidArgument.into());
}
msg!("Admin: {}", account_state.admin.to_string());
Ok(())
}
}
#[derive(Accounts)]
pub struct Unchecked<'info> {
admin_config: AccountInfo<'info>,
admin: Signer<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
#### Add owner check
In vanilla Rust, you could solve this problem by comparing the `owner` field on
the account to the program ID. If they do not match, you would return an
`IncorrectProgramId` error.
```rust theme={null}
if ctx.accounts.admin_config.owner != ctx.program_id {
return Err(ProgramError::IncorrectProgramId.into());
}
```
Adding an owner check prevents accounts owned by an unexpected program to be
passed in as the `admin_config` account. If a fake `admin_config` account was
used in the `admin_instruction`, then the transaction would fail.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Cft4eTTrt4sJU4Ar35rUQHx6PSXfJju3dixmvApzhWws");
#[program]
pub mod owner_check {
use super::*;
...
pub fn admin_instruction(ctx: Context) -> Result<()> {
if ctx.accounts.admin_config.owner != ctx.program_id {
return Err(ProgramError::IncorrectProgramId.into());
}
let account_data = ctx.accounts.admin_config.try_borrow_data()?;
let mut account_data_slice: &[u8] = &account_data;
let account_state = AdminConfig::try_deserialize(&mut account_data_slice)?;
if account_state.admin != ctx.accounts.admin.key() {
return Err(ProgramError::InvalidArgument.into());
}
msg!("Admin: {}", account_state.admin.to_string());
Ok(())
}
}
#[derive(Accounts)]
pub struct Unchecked<'info> {
admin_config: AccountInfo<'info>,
admin: Signer<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
#### Use Anchor’s `Account<'info, T>`
Anchor can make this simpler with the `Account` type.
`Account<'info, T>` is a wrapper around `AccountInfo` that verifies program
ownership and deserializes underlying data into the specified account type `T`.
This in turn allows you to use `Account<'info, T>` to easily validate ownership.
For context, the `#[account]` attribute implements various traits for a data
structure representing an account. One of these is the `Owner` trait which
defines an address expected to own an account. The owner is set as the program
ID specified in the `declare_id!` macro.
In the example below, `Account<'info, AdminConfig>` is used to validate the
`admin_config`. This will automatically perform the owner check and deserialize
the account data. Additionally, the `has_one` constraint is used to check that
the `admin` account matches the `admin` field stored on the `admin_config`
account.
This way, you don’t need to clutter your instruction logic with owner checks.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Cft4eTTrt4sJU4Ar35rUQHx6PSXfJju3dixmvApzhWws");
#[program]
pub mod owner_check {
use super::*;
...
pub fn admin_instruction(ctx: Context) -> Result<()> {
msg!("Admin: {}", ctx.accounts.admin_config.admin.to_string());
Ok(())
}
}
#[derive(Accounts)]
pub struct Checked<'info> {
#[account(
has_one = admin,
)]
admin_config: Account<'info, AdminConfig>,
admin: Signer<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
#### Use Anchor’s `#[account(owner = )]` constraint
In addition to the `Account` type, you can use an `owner` constraint. The
`owner` constraint allows you to define the program that should own an account
if it’s different from the currently executing one. This comes in handy if, for
example, you are writing an instruction that expects an account to be a PDA
derived from a different program. You can use the `seeds` and `bump` constraints
and define the `owner` to properly derive and verify the address of the account
passed in.
To use the `owner` constraint, you’ll have to have access to the public key of
the program you expect to own an account. You can either pass the program in as
an additional account or hard-code the public key somewhere in your program.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Cft4eTTrt4sJU4Ar35rUQHx6PSXfJju3dixmvApzhWws");
#[program]
pub mod owner_check {
use super::*;
...
pub fn admin_instruction(ctx: Context) -> Result<()> {
msg!("Admin: {}", ctx.accounts.admin_config.admin.to_string());
Ok(())
}
}
#[derive(Accounts)]
pub struct Checked<'info> {
#[account(
has_one = admin,
)]
admin_config: Account<'info, AdminConfig>,
admin: Signer<'info>,
#[account(
seeds = b"test-seed",
bump,
owner = token_program.key()
)]
pda_derived_from_another_program: AccountInfo<'info>,
token_program: Program<'info, Token>
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
```
## Lab
In this lab we’ll use two programs to demonstrate how a missing owner check
could allow a fake account to drain the tokens from a simplified token “vault”
account (note that this is very similar to the lab from the Signer Authorization
lesson).
To help illustrate this, one program will be missing an account owner check on
the vault account it withdraws tokens to.
The second program will be a direct clone of the first program created by a
malicious user to create an account identical to the first program’s vault
account.
Without the owner check, this malicious user will be able to pass in the vault
account owned by their “faked” program and the original program will still
execute.
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-owner-checks/tree/starter).
The starter code includes two programs `clone` and `owner_check` and the
boilerplate setup for the test file.
The `owner_check` program includes two instructions:
* `initialize_vault` initializes a simplified vault account that stores the
addresses of a token account and an authority account
* `insecure_withdraw` withdraws tokens from the token account, but is missing an
owner check for the vault account
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, Token, TokenAccount};
declare_id!("HQYNznB3XTqxzuEqqKMAD9XkYE5BGrnv8xmkoDNcqHYB");
#[program]
pub mod owner_check {
use super::*;
pub fn initialize_vault(ctx: Context) -> Result<()> {
ctx.accounts.vault.token_account = ctx.accounts.token_account.key();
ctx.accounts.vault.authority = ctx.accounts.authority.key();
Ok(())
}
pub fn insecure_withdraw(ctx: Context) -> Result<()> {
let account_data = ctx.accounts.vault.try_borrow_data()?;
let mut account_data_slice: &[u8] = &account_data;
let account_state = Vault::try_deserialize(&mut account_data_slice)?;
if account_state.authority != ctx.accounts.authority.key() {
return Err(ProgramError::InvalidArgument.into());
}
let amount = ctx.accounts.token_account.amount;
let seeds = &[
b"token".as_ref(),
&[*ctx.bumps.get("token_account").unwrap()],
];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.token_account.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
#[derive(Accounts)]
pub struct InitializeVault<'info> {
#[account(
init,
payer = authority,
space = 8 + 32 + 32,
)]
pub vault: Account<'info, Vault>,
#[account(
init,
payer = authority,
token::mint = mint,
token::authority = token_account,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
pub mint: Account<'info, Mint>,
#[account(mut)]
pub authority: Signer<'info>,
pub token_program: Program<'info, Token>,
pub system_program: Program<'info, System>,
pub rent: Sysvar<'info, Rent>,
}
#[derive(Accounts)]
pub struct InsecureWithdraw<'info> {
/// CHECK:
pub vault: UncheckedAccount<'info>,
#[account(
mut,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
pub authority: Signer<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
The `clone` program includes a single instruction:
* `initialize_vault` initializes a “vault” account that mimics the vault account
of the `owner_check` program. It stores the address of the real vault’s token
account, but allows the malicious user to put their own authority account.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::TokenAccount;
declare_id!("DUN7nniuatsMC7ReCh5eJRQExnutppN1tAfjfXFmGDq3");
#[program]
pub mod clone {
use super::*;
pub fn initialize_vault(ctx: Context) -> Result<()> {
ctx.accounts.vault.token_account = ctx.accounts.token_account.key();
ctx.accounts.vault.authority = ctx.accounts.authority.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct InitializeVault<'info> {
#[account(
init,
payer = authority,
space = 8 + 32 + 32,
)]
pub vault: Account<'info, Vault>,
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub authority: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
#### 2. Test `insecure_withdraw` instruction
The test file includes a test to invoke the `initialize_vault` instruction on
the `owner_check` program using the provider wallet as the `authority` and then
mints 100 tokens to the token account.
The test file also includes a test to invoke the `initialize_vault` instruction
on the `clone` program to initialize a fake `vault` account storing the same
`tokenPDA` account, but a different `authority`. Note that no new tokens are
minted here.
Let’s add a test to invoke the `insecure_withdraw` instruction. This test should
pass in the cloned vault and the fake authority. Since there is no owner check
to verify the `vaultClone` account is owned by the `owner_check` program, the
instruction’s data validation check will pass and show `walletFake` as a valid
authority. The tokens from the `tokenPDA` account will then be withdrawn to the
`withdrawDestinationFake` account.
```typescript theme={null}
describe("owner-check", () => {
...
it("Insecure withdraw", async () => {
const tx = await program.methods
.insecureWithdraw()
.accounts({
vault: vaultClone.publicKey,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestinationFake,
authority: walletFake.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
const balance = await connection.getTokenAccountBalance(tokenPDA)
expect(balance.value.uiAmount).to.eq(0)
})
})
```
Run `anchor test` to see that the `insecure_withdraw` completes successfully.
```bash theme={null}
owner-check
✔ Initialize Vault (808ms)
✔ Initialize Fake Vault (404ms)
✔ Insecure withdraw (409ms)
```
Note that `vaultClone` deserializes successfully even though Anchor
automatically initializes new accounts with a unique 8 byte discriminator and
checks the discriminator when deserializing an account. This is because the
discriminator is a hash of the account type name.
```rust theme={null}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
Since both programs initialize identical accounts and both structs are named
`Vault`, the accounts have the same discriminator even though they are owned by
different programs.
#### 3. Add `secure_withdraw` instruction
Let’s close up this security loophole.
In the `lib.rs` file of the `owner_check` program add a `secure_withdraw`
instruction and a `SecureWithdraw` accounts struct.
In the `SecureWithdraw` struct, let’s use `Account<'info, Vault>` to ensure that
an owner check is performed on the `vault` account. We’ll also use the `has_one`
constraint to check that the `token_account` and `authority` passed into the
instruction match the values stored on the `vault` account.
```rust theme={null}
#[program]
pub mod owner_check {
use super::*;
...
pub fn secure_withdraw(ctx: Context) -> Result<()> {
let amount = ctx.accounts.token_account.amount;
let seeds = &[
b"token".as_ref(),
&[*ctx.bumps.get("token_account").unwrap()],
];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.token_account.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
...
#[derive(Accounts)]
pub struct SecureWithdraw<'info> {
#[account(
has_one = token_account,
has_one = authority
)]
pub vault: Account<'info, Vault>,
#[account(
mut,
seeds = [b"token"],
bump,
)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
pub authority: Signer<'info>,
}
```
#### 4. Test `secure_withdraw` instruction
To test the `secure_withdraw` instruction, we’ll invoke the instruction twice.
First, we’ll invoke the instruction using the `vaultClone` account, which we
expect to fail. Then, we’ll invoke the instruction using the correct `vault`
account to check that the instruction works as intended.
```typescript theme={null}
describe("owner-check", () => {
...
it("Secure withdraw, expect error", async () => {
try {
const tx = await program.methods
.secureWithdraw()
.accounts({
vault: vaultClone.publicKey,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestinationFake,
authority: walletFake.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
} catch (err) {
expect(err)
console.log(err)
}
})
it("Secure withdraw", async () => {
await spl.mintTo(
connection,
wallet.payer,
mint,
tokenPDA,
wallet.payer,
100
)
await program.methods
.secureWithdraw()
.accounts({
vault: vault.publicKey,
tokenAccount: tokenPDA,
withdrawDestination: withdrawDestination,
authority: wallet.publicKey,
})
.rpc()
const balance = await connection.getTokenAccountBalance(tokenPDA)
expect(balance.value.uiAmount).to.eq(0)
})
})
```
Run `anchor test` to see that the transaction using the `vaultClone` account
will now return an Anchor Error while the transaction using the `vault` account
completes successfully.
```bash theme={null}
'Program HQYNznB3XTqxzuEqqKMAD9XkYE5BGrnv8xmkoDNcqHYB invoke [1]',
'Program log: Instruction: SecureWithdraw',
'Program log: AnchorError caused by account: vault. Error Code: AccountOwnedByWrongProgram. Error Number: 3007. Error Message: The given account is owned by a different program than expected.',
'Program log: Left:',
'Program log: DUN7nniuatsMC7ReCh5eJRQExnutppN1tAfjfXFmGDq3',
'Program log: Right:',
'Program log: HQYNznB3XTqxzuEqqKMAD9XkYE5BGrnv8xmkoDNcqHYB',
'Program HQYNznB3XTqxzuEqqKMAD9XkYE5BGrnv8xmkoDNcqHYB consumed 5554 of 200000 compute units',
'Program HQYNznB3XTqxzuEqqKMAD9XkYE5BGrnv8xmkoDNcqHYB failed: custom program error: 0xbbf'
```
Here we see how using Anchor’s `Account<'info, T>` type can simplify the account
validation process to automate the ownership check. Additionally, note that
Anchor Errors can specify the account that causes the error (e.g. the third line
of the logs above say `AnchorError caused by account: vault`). This can be very
helpful when debugging.
```bash theme={null}
✔ Secure withdraw, expect error (78ms)
✔ Secure withdraw (10063ms)
```
That’s all you need to ensure you check the owner on an account! Like some other
exploits, it’s fairly simple to avoid but very important. Be sure to always
think through which accounts should be owned by which programs and ensure that
you add appropriate validation.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the repository](https://github.com/Unboxed-Software/solana-owner-checks/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that proper owner
checks are performed on the accounts passed into each instruction.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=e3069010-3038-4984-b9d3-2dc6585147b1)!
# PDA Sharing
Source: https://docs.orbition.network/courses/program-security/pda-sharing
Understand the potential problems of reusing PDAs by using user and domain specific PDAs.
## Summary
* Using the same PDA for multiple authority domains opens your program up to the
possibility of users accessing data and funds that don't belong to them
* Prevent the same PDA from being used for multiple accounts by using seeds that
are user and/or domain-specific
* Use Anchor’s `seeds` and `bump` constraints to validate that a PDA is derived
using the expected seeds and bump
## Lesson
PDA sharing refers to using the same PDA as a signer across multiple users or
domains. Especially when using PDAs for signing, it may seem appropriate to use
a global PDA to represent the program. However, this opens up the possibility of
account validation passing but a user being able to access funds, transfers, or
data not belonging to them.
### Insecure global PDA
In the example below, the `authority` of the `vault` account is a PDA derived
using the `mint` address stored on the `pool` account. This PDA is passed into
the instruction as the `authority` account to sign for the transfer tokens from
the `vault` to the `withdraw_destination`.
Using the `mint` address as a seed to derive the PDA to sign for the `vault` is
insecure because multiple `pool` accounts could be created for the same `vault`
token account, but a different `withdraw_destination`. By using the `mint` as a
seed derive the PDA to sign for token transfers, any `pool` account could sign
for the transfer of tokens from a `vault` token account to an arbitrary
`withdraw_destination`.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod pda_sharing_insecure {
use super::*;
pub fn withdraw_tokens(ctx: Context) -> Result<()> {
let amount = ctx.accounts.vault.amount;
let seeds = &[ctx.accounts.pool.mint.as_ref(), &[ctx.accounts.pool.bump]];
token::transfer(ctx.accounts.transfer_ctx().with_signer(&[seeds]), amount)
}
}
#[derive(Accounts)]
pub struct WithdrawTokens<'info> {
#[account(has_one = vault, has_one = withdraw_destination)]
pool: Account<'info, TokenPool>,
vault: Account<'info, TokenAccount>,
withdraw_destination: Account<'info, TokenAccount>,
authority: AccountInfo<'info>,
token_program: Program<'info, Token>,
}
impl<'info> WithdrawTokens<'info> {
pub fn transfer_ctx(&self) -> CpiContext<'_, '_, '_, 'info, token::Transfer<'info>> {
let program = self.token_program.to_account_info();
let accounts = token::Transfer {
from: self.vault.to_account_info(),
to: self.withdraw_destination.to_account_info(),
authority: self.authority.to_account_info(),
};
CpiContext::new(program, accounts)
}
}
#[account]
pub struct TokenPool {
vault: Pubkey,
mint: Pubkey,
withdraw_destination: Pubkey,
bump: u8,
}
```
### Secure account specific PDA
One approach to create an account specific PDA is to use the
`withdraw_destination` as a seed to derive the PDA used as the authority of the
`vault` token account. This ensures the PDA signing for the CPI in the
`withdraw_tokens` instruction is derived using the intended
`withdraw_destination` token account. In other words, tokens from a `vault`
token account can only be withdrawn to the `withdraw_destination` that was
originally initialized with the `pool` account.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod pda_sharing_secure {
use super::*;
pub fn withdraw_tokens(ctx: Context) -> Result<()> {
let amount = ctx.accounts.vault.amount;
let seeds = &[
ctx.accounts.pool.withdraw_destination.as_ref(),
&[ctx.accounts.pool.bump],
];
token::transfer(ctx.accounts.transfer_ctx().with_signer(&[seeds]), amount)
}
}
#[derive(Accounts)]
pub struct WithdrawTokens<'info> {
#[account(has_one = vault, has_one = withdraw_destination)]
pool: Account<'info, TokenPool>,
vault: Account<'info, TokenAccount>,
withdraw_destination: Account<'info, TokenAccount>,
authority: AccountInfo<'info>,
token_program: Program<'info, Token>,
}
impl<'info> WithdrawTokens<'info> {
pub fn transfer_ctx(&self) -> CpiContext<'_, '_, '_, 'info, token::Transfer<'info>> {
let program = self.token_program.to_account_info();
let accounts = token::Transfer {
from: self.vault.to_account_info(),
to: self.withdraw_destination.to_account_info(),
authority: self.authority.to_account_info(),
};
CpiContext::new(program, accounts)
}
}
#[account]
pub struct TokenPool {
vault: Pubkey,
mint: Pubkey,
withdraw_destination: Pubkey,
bump: u8,
}
```
### Anchor’s `seeds` and `bump` constraints
PDAs can be used as both the address of an account and allow programs to sign
for the PDAs they own.
The example below uses a PDA derived using the `withdraw_destination` as both
the address of the `pool` account and owner of the `vault` token account. This
means that only the `pool` account associated with correct `vault` and
`withdraw_destination` can be used in the `withdraw_tokens` instruction.
You can use Anchor’s `seeds` and `bump` constraints with the `#[account(...)]`
attribute to validate the `pool` account PDA. Anchor derives a PDA using the
`seeds` and `bump` specified and compare against the account passed into the
instruction as the `pool` account. The `has_one` constraint is used to further
ensure that only the correct accounts stored on the `pool` account are passed
into the instruction.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod pda_sharing_recommended {
use super::*;
pub fn withdraw_tokens(ctx: Context) -> Result<()> {
let amount = ctx.accounts.vault.amount;
let seeds = &[
ctx.accounts.pool.withdraw_destination.as_ref(),
&[ctx.accounts.pool.bump],
];
token::transfer(ctx.accounts.transfer_ctx().with_signer(&[seeds]), amount)
}
}
#[derive(Accounts)]
pub struct WithdrawTokens<'info> {
#[account(
has_one = vault,
has_one = withdraw_destination,
seeds = [withdraw_destination.key().as_ref()],
bump = pool.bump,
)]
pool: Account<'info, TokenPool>,
vault: Account<'info, TokenAccount>,
withdraw_destination: Account<'info, TokenAccount>,
token_program: Program<'info, Token>,
}
impl<'info> WithdrawTokens<'info> {
pub fn transfer_ctx(&self) -> CpiContext<'_, '_, '_, 'info, token::Transfer<'info>> {
let program = self.token_program.to_account_info();
let accounts = token::Transfer {
from: self.vault.to_account_info(),
to: self.withdraw_destination.to_account_info(),
authority: self.pool.to_account_info(),
};
CpiContext::new(program, accounts)
}
}
#[account]
pub struct TokenPool {
vault: Pubkey,
mint: Pubkey,
withdraw_destination: Pubkey,
bump: u8,
}
```
## Lab
Let’s practice by creating a simple program to demonstrate how a PDA sharing can
allow an attacker to withdraw tokens that don’t belong to them. this lab expands
on the examples above by including the instructions to initialize the required
program accounts.
#### 1. Starter
To get started, download the starter code on the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-pda-sharing/tree/starter).
The starter code includes a program with two instructions and the boilerplate
setup for the test file.
The `initialize_pool` instruction initializes a new `TokenPool` that stores a
`vault`, `mint`, `withdraw_destination`, and `bump`. The `vault` is a token
account where the authority is set as a PDA derived using the `mint` address.
The `withdraw_insecure` instruction will transfer tokens in the `vault` token
account to a `withdraw_destination` token account.
However, as written the seeds used for signing are not specific to the vault's
withdraw destination, thus opening up the program to security exploits. Take a
minute to familiarize yourself with the code before continuing on.
#### 2. Test `withdraw_insecure` instruction
The test file includes the code to invoke the `initialize_pool` instruction and
then mint 100 tokens to the `vault` token account. It also includes a test to
invoke the `withdraw_insecure` using the intended `withdraw_destination`. This
shows that the instructions can be used as intended.
After that, there are two more tests to show how the instructions are vulnerable
to exploit.
The first test invokes the `initialize_pool` instruction to create a "fake"
`pool` account using the same `vault` token account, but a different
`withdraw_destination`.
The second test withdraws from this pool, stealing funds from the vault.
```typescript theme={null}
it("Insecure initialize allows pool to be initialized with wrong vault", async () => {
await program.methods
.initializePool(authInsecureBump)
.accounts({
pool: poolInsecureFake.publicKey,
mint: mint,
vault: vaultInsecure.address,
withdrawDestination: withdrawDestinationFake,
payer: walletFake.publicKey,
})
.signers([walletFake, poolInsecureFake])
.rpc();
await new Promise(x => setTimeout(x, 1000));
await spl.mintTo(
connection,
wallet.payer,
mint,
vaultInsecure.address,
wallet.payer,
100,
);
const account = await spl.getAccount(connection, vaultInsecure.address);
expect(Number(account.amount)).to.equal(100);
});
it("Insecure withdraw allows stealing from vault", async () => {
await program.methods
.withdrawInsecure()
.accounts({
pool: poolInsecureFake.publicKey,
vault: vaultInsecure.address,
withdrawDestination: withdrawDestinationFake,
authority: authInsecure,
signer: walletFake.publicKey,
})
.signers([walletFake])
.rpc();
const account = await spl.getAccount(connection, vaultInsecure.address);
expect(Number(account.amount)).to.equal(0);
});
```
Run `anchor test` to see that the transactions complete successfully and the
`withdraw_instrucure` instruction allows the `vault` token account to be drained
to a fake withdraw destination stored on the fake `pool` account.
#### 3. Add `initialize_pool_secure` instruction
Now let's add a new instruction to the program for securely initializing a pool.
This new `initialize_pool_secure` instruction will initialize a `pool` account
as a PDA derived using the `withdraw_destination`. It will also initialize a
`vault` token account with the authority set as the `pool` PDA.
```rust theme={null}
pub fn initialize_pool_secure(ctx: Context) -> Result<()> {
ctx.accounts.pool.vault = ctx.accounts.vault.key();
ctx.accounts.pool.mint = ctx.accounts.mint.key();
ctx.accounts.pool.withdraw_destination = ctx.accounts.withdraw_destination.key();
ctx.accounts.pool.bump = *ctx.bumps.get("pool").unwrap();
Ok(())
}
...
#[derive(Accounts)]
pub struct InitializePoolSecure<'info> {
#[account(
init,
payer = payer,
space = 8 + 32 + 32 + 32 + 1,
seeds = [withdraw_destination.key().as_ref()],
bump
)]
pub pool: Account<'info, TokenPool>,
pub mint: Account<'info, Mint>,
#[account(
init,
payer = payer,
token::mint = mint,
token::authority = pool,
)]
pub vault: Account<'info, TokenAccount>,
pub withdraw_destination: Account<'info, TokenAccount>,
#[account(mut)]
pub payer: Signer<'info>,
pub system_program: Program<'info, System>,
pub token_program: Program<'info, Token>,
pub rent: Sysvar<'info, Rent>,
}
```
#### 4. Add `withdraw_secure` instruction
Next, add a `withdraw_secure` instruction. This instruction will withdraw tokens
from the `vault` token account to the `withdraw_destination`. The `pool` account
is validated using the `seeds` and `bump` constraints to ensure the correct PDA
account is provided. The `has_one` constraints check that the correct `vault`
and `withdraw_destination` token accounts are provided.
```rust theme={null}
pub fn withdraw_secure(ctx: Context) -> Result<()> {
let amount = ctx.accounts.vault.amount;
let seeds = &[
ctx.accounts.pool.withdraw_destination.as_ref(),
&[ctx.accounts.pool.bump],
];
token::transfer(ctx.accounts.transfer_ctx().with_signer(&[seeds]), amount)
}
...
#[derive(Accounts)]
pub struct WithdrawTokensSecure<'info> {
#[account(
has_one = vault,
has_one = withdraw_destination,
seeds = [withdraw_destination.key().as_ref()],
bump = pool.bump,
)]
pool: Account<'info, TokenPool>,
#[account(mut)]
vault: Account<'info, TokenAccount>,
#[account(mut)]
withdraw_destination: Account<'info, TokenAccount>,
token_program: Program<'info, Token>,
}
impl<'info> WithdrawTokensSecure<'info> {
pub fn transfer_ctx(&self) -> CpiContext<'_, '_, '_, 'info, token::Transfer<'info>> {
let program = self.token_program.to_account_info();
let accounts = token::Transfer {
from: self.vault.to_account_info(),
to: self.withdraw_destination.to_account_info(),
authority: self.pool.to_account_info(),
};
CpiContext::new(program, accounts)
}
}
```
#### 5. Test `withdraw_secure` instruction
Finally, return to the test file to test the `withdraw_secure` instruction and
show that by narrowing the scope of our PDA signing authority, we've removed the
vulnerability.
Before we write a test showing the vulnerability has been patched let's write a
test that simply shows that the initialization and withdraw instructions work as
expected:
```typescript theme={null}
it("Secure pool initialization and withdraw works", async () => {
const withdrawDestinationAccount = await getAccount(
provider.connection,
withdrawDestination,
);
await program.methods
.initializePoolSecure()
.accounts({
pool: authSecure,
mint: mint,
vault: vaultRecommended.publicKey,
withdrawDestination: withdrawDestination,
})
.signers([vaultRecommended])
.rpc();
await new Promise(x => setTimeout(x, 1000));
await spl.mintTo(
connection,
wallet.payer,
mint,
vaultRecommended.publicKey,
wallet.payer,
100,
);
await program.methods
.withdrawSecure()
.accounts({
pool: authSecure,
vault: vaultRecommended.publicKey,
withdrawDestination: withdrawDestination,
})
.rpc();
const afterAccount = await getAccount(
provider.connection,
withdrawDestination,
);
expect(
Number(afterAccount.amount) - Number(withdrawDestinationAccount.amount),
).to.equal(100);
});
```
Now, we'll test that the exploit no longer works. Since the `vault` authority is
the `pool` PDA derived using the intended `withdraw_destination` token account,
there should no longer be a way to withdraw to an account other than the
intended `withdraw_destination`.
Add a test that shows you can't call `withdraw_secure` with the wrong withdrawal
destination. It can use the pool and vault created in the previous test.
```typescript theme={null}
it("Secure withdraw doesn't allow withdraw to wrong destination", async () => {
try {
await program.methods
.withdrawSecure()
.accounts({
pool: authSecure,
vault: vaultRecommended.publicKey,
withdrawDestination: withdrawDestinationFake,
})
.signers([walletFake])
.rpc();
assert.fail("expected error");
} catch (error) {
console.log(error.message);
expect(error);
}
});
```
Lastly, since the `pool` account is a PDA derived using the
`withdraw_destination` token account, we can’t create a fake `pool` account
using the same PDA. Add one more test showing that the new
`initialize_pool_secure` instruction won't let an attacker put in the wrong
vault.
```typescript theme={null}
it("Secure pool initialization doesn't allow wrong vault", async () => {
try {
await program.methods
.initializePoolSecure()
.accounts({
pool: authSecure,
mint: mint,
vault: vaultInsecure.address,
withdrawDestination: withdrawDestination,
})
.signers([vaultRecommended])
.rpc();
assert.fail("expected error");
} catch (error) {
console.log(error.message);
expect(error);
}
});
```
Run `anchor test` and to see that the new instructions don't allow an attacker
to withdraw from a vault that isn't theirs.
```
pda-sharing
✔ Initialize Pool Insecure (981ms)
✔ Withdraw (470ms)
✔ Insecure initialize allows pool to be initialized with wrong vault (10983ms)
✔ Insecure withdraw allows stealing from vault (492ms)
✔ Secure pool initialization and withdraw works (2502ms)
unknown signer: ARjxAsEPj6YsAPKaBfd1AzUHbNPtAeUsqusAmBchQTfV
✔ Secure withdraw doesn't allow withdraw to wrong destination
unknown signer: GJcHJLot3whbY1aC9PtCsBYk5jWoZnZRJPy5uUwzktAY
✔ Secure pool initialization doesn't allow wrong vault
```
And that's it! Unlike some of the other security vulnerabilities we've
discussed, this one is more conceptual and can't be fixed by simply using a
particular Anchor type. You'll need to think through the architecture of your
program and ensure that you aren't sharing PDAs across different domains.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the same repository](https://github.com/Unboxed-Software/solana-pda-sharing/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and look for potential
vulnerabilities in its PDA structure. PDAs used for signing should be narrow and
focused on a single domain as much as possible.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=5744079f-9473-4485-9a14-9be4d31b40d1)!
# Reinitialization Attacks
Source: https://docs.orbition.network/courses/program-security/reinitialization-attacks
Understand the security risks of account reinitialized attacks being used to override data, and how to prevent them.
## Summary
* Use an account discriminator or initialization flag to check whether an
account has already been initialized to prevent an account from being
reinitialized and overriding existing account data.
* To prevent account reinitialization in plain Rust, initialize accounts with an
`is_initialized` flag and check if it has already been set to true when
initializing an account
```rust theme={null}
if account.is_initialized {
return Err(ProgramError::AccountAlreadyInitialized.into());
}
```
* To simplify this, use Anchor’s `init` constraint to create an account via a
CPI to the system program and sets its discriminator
## Lesson
Initialization refers to setting the data of a new account for the first time.
When initializing a new account, you should implement a way to check if the
account has already been initialized. Without an appropriate check, an existing
account could be reinitialized and have existing data overwritten.
Note that initializing an account and creating an account are two separate
instructions. Creating an account requires invoking the `create_account`
instruction on the System Program which specifies the space required for the
account, the rent in lamports allocated to the account, and the program owner of
the account. Initialization is an instruction that sets the data of a newly
created account. Creating and initializing an account can be combined into a
single transaction.
#### Missing Initialization Check
In the example below, there are no checks on the `user` account. The
`initialize` instruction deserializes the data of the `user` account as a `User`
account type, sets the `authority` field, and serializes the updated account
data to the `user` account.
Without checks on the `user` account, the same account could be passed into the
`initialize` instruction a second time by another party to overwrite the
existing `authority` stored on the account data.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod initialization_insecure {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
let mut user = User::try_from_slice(&ctx.accounts.user.data.borrow()).unwrap();
user.authority = ctx.accounts.authority.key();
user.serialize(&mut *ctx.accounts.user.data.borrow_mut())?;
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(mut)]
user: AccountInfo<'info>,
#[account(mut)]
authority: Signer<'info>,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
authority: Pubkey,
}
```
#### Add `is_initialized` check
One approach to fix this is to add an additional `is_initialized` field to the
`User` account type and use it as a flag to check if an account has already been
initialized.
```rust theme={null}
if user.is_initialized {
return Err(ProgramError::AccountAlreadyInitialized.into());
}
```
By including a check within the `initialize` instruction, the `user` account
would only be initialized if the `is_initialized` field has not yet been set to
true. If the `is_initialized` field was already set, the transaction would fail,
thereby avoiding the scenario where an attacker could replace the account
authority with their own public key.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod initialization_secure {
use super::*;
pub fn initialize(ctx: Context) -> Result<()> {
let mut user = User::try_from_slice(&ctx.accounts.user.data.borrow()).unwrap();
if user.is_initialized {
return Err(ProgramError::AccountAlreadyInitialized.into());
}
user.authority = ctx.accounts.authority.key();
user.is_initialized = true;
user.serialize(&mut *ctx.accounts.user.data.borrow_mut())?;
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(mut)]
user: AccountInfo<'info>,
#[account(mut)]
authority: Signer<'info>,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
is_initialized: bool,
authority: Pubkey,
}
```
#### Use Anchor’s `init` constraint
Anchor provides an `init` constraint that can be used with the `#[account(...)]`
attribute to initialize an account. The `init` constraint creates the account
via a CPI to the system program and sets the account discriminator.
The `init` constraint must be used in combination with the `payer` and `space`
constraints. The `payer` specifies the account paying for the initialization of
the new account. The `space` specifies the amount of space the new account
requires, which determines the amount of lamports that must be allocated to the
account. The first 8 bytes of data is set as a discriminator that Anchor
automatically adds to identify the account type.
Most importantly for this lesson, the `init` constraint ensures that this
instruction can only be called once per account, so you can set the initial
state of the account in the instruction logic and not have to worry about an
attacker trying to reinitialize the account.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod initialization_recommended {
use super::*;
pub fn initialize(_ctx: Context) -> Result<()> {
msg!("GM");
Ok(())
}
}
#[derive(Accounts)]
pub struct Initialize<'info> {
#[account(init, payer = authority, space = 8+32)]
user: Account<'info, User>,
#[account(mut)]
authority: Signer<'info>,
system_program: Program<'info, System>,
}
#[account]
pub struct User {
authority: Pubkey,
}
```
#### Anchor’s `init_if_needed` constraint
It’s worth noting that Anchor has an `init_if_needed` constraint. This
constraint should be used very cautiously. In fact, it is blocked behind a
feature flag so that you are forced to be intentional about using it.
The `init_if_needed` constraint does the same thing as the `init` constraint,
only if the account has already been initialized the instruction will still run.
Given this, it’s \***\*\*\*\***extremely\***\*\*\*\*** important that when you
use this constraint you include checks to avoid resetting the account to its
initial state.
For example, if the account stores an `authority` field that gets set in the
instruction using the `init_if_needed` constraint, you need checks that ensure
that no attacker could call the instruction after it has already been
initialized and have the `authority` field set again.
In most cases, it’s safer to have a separate instruction for initializing
account data.
## Lab
For this lab we’ll create a simple program that does nothing but initialize
accounts. We’ll include two instructions:
* `insecure_initialization` - initializes an account that can be reinitialized
* `recommended_initialization` - initialize an account using Anchor’s `init`
constraint
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-reinitialization-attacks/tree/starter).
The starter code includes a program with one instruction and the boilerplate
setup for the test file.
The `insecure_initialization` instruction initializes a new `user` account that
stores the public key of an `authority`. In this instruction, the account is
expected to be allocated client-side, then passed into the program instruction.
Once passed into the program, there are no checks to see if the `user` account's
initial state has already been set. This means the same account can be passed in
a second time to override the `authority` stored on an existing `user` account.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod initialization {
use super::*;
pub fn insecure_initialization(ctx: Context) -> Result<()> {
let mut user = User::try_from_slice(&ctx.accounts.user.data.borrow()).unwrap();
user.authority = ctx.accounts.authority.key();
user.serialize(&mut *ctx.accounts.user.data.borrow_mut())?;
Ok(())
}
}
#[derive(Accounts)]
pub struct Unchecked<'info> {
#[account(mut)]
/// CHECK:
user: UncheckedAccount<'info>,
authority: Signer<'info>,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
authority: Pubkey,
}
```
#### 2. Test `insecure_initialization` instruction
The test file includes the setup to create an account by invoking the system
program and then invokes the `insecure_initialization` instruction twice using
the same account.
Since there are no checks the verify that the account data has not already been
initialized, the `insecure_initialization` instruction will complete
successfully both times, despite the second invocation providing a *different*
authority account.
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { expect } from "chai";
import { Initialization } from "../target/types/initialization";
describe("initialization", () => {
const provider = anchor.AnchorProvider.env();
anchor.setProvider(provider);
const program = anchor.workspace.Initialization as Program;
const wallet = anchor.workspace.Initialization.provider.wallet;
const walletTwo = anchor.web3.Keypair.generate();
const userInsecure = anchor.web3.Keypair.generate();
const userRecommended = anchor.web3.Keypair.generate();
before(async () => {
const tx = new anchor.web3.Transaction().add(
anchor.web3.SystemProgram.createAccount({
fromPubkey: wallet.publicKey,
newAccountPubkey: userInsecure.publicKey,
space: 32,
lamports:
await provider.connection.getMinimumBalanceForRentExemption(32),
programId: program.programId,
}),
);
await anchor.web3.sendAndConfirmTransaction(provider.connection, tx, [
wallet.payer,
userInsecure,
]);
await provider.connection.confirmTransaction(
await provider.connection.requestAirdrop(
walletTwo.publicKey,
1 * anchor.web3.LAMPORTS_PER_SOL,
),
"confirmed",
);
});
it("Insecure init", async () => {
await program.methods
.insecureInitialization()
.accounts({
user: userInsecure.publicKey,
})
.rpc();
});
it("Re-invoke insecure init with different auth", async () => {
const tx = await program.methods
.insecureInitialization()
.accounts({
user: userInsecure.publicKey,
authority: walletTwo.publicKey,
})
.transaction();
await anchor.web3.sendAndConfirmTransaction(provider.connection, tx, [
walletTwo,
]);
});
});
```
Run `anchor test` to see that both transactions will complete successfully.
```bash theme={null}
initialization
✔ Insecure init (478ms)
✔ Re-invoke insecure init with different auth (464ms)
```
#### 3. Add `recommended_initialization` instruction
Let's create a new instruction called `recommended_initialization` that fixes
this problem. Unlike the previous insecure instruction, this instruction should
handle both the creation and initialization of the user's account using Anchor's
`init` constraint.
This constraint instructs the program to create the account via a CPI to the
system program, so the account no longer needs to be created client-side. The
constraint also sets the account discriminator. Your instruction logic can then
set the account's initial state.
By doing this, you ensure that any subsequent invocation of the same instruction
with the same user account will fail rather than reset the account's initial
state.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod initialization {
use super::*;
...
pub fn recommended_initialization(ctx: Context) -> Result<()> {
ctx.accounts.user.authority = ctx.accounts.authority.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct Checked<'info> {
#[account(init, payer = authority, space = 8+32)]
user: Account<'info, User>,
#[account(mut)]
authority: Signer<'info>,
system_program: Program<'info, System>,
}
```
#### 4. Test `recommended_initialization` instruction
To test the `recommended_initialization` instruction, we’ll invoke the
instruction twice just like before. This time, we expect the transaction to fail
when we try to initialize the same account a second time.
```typescript theme={null}
describe("initialization", () => {
...
it("Recommended init", async () => {
await program.methods
.recommendedInitialization()
.accounts({
user: userRecommended.publicKey,
})
.signers([userRecommended])
.rpc()
})
it("Re-invoke recommended init with different auth, expect error", async () => {
try {
// Add your test here.
const tx = await program.methods
.recommendedInitialization()
.accounts({
user: userRecommended.publicKey,
authority: walletTwo.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(provider.connection, tx, [
walletTwo,
userRecommended,
])
} catch (err) {
expect(err)
console.log(err)
}
})
})
```
Run `anchor test` and to see that the second transaction which tries to
initialize the same account twice will now return an error stating the account
address is already in use.
```bash theme={null}
'Program CpozUgSwe9FPLy9BLNhY2LTGqLUk1nirUkMMA5RmDw6t invoke [1]',
'Program log: Instruction: RecommendedInitialization',
'Program 11111111111111111111111111111111 invoke [2]',
'Allocate: account Address { address: EMvbwzrs4VTR7G1sNUJuQtvRX1EuvLhqs4PFqrtDcCGV, base: None } already in use',
'Program 11111111111111111111111111111111 failed: custom program error: 0x0',
'Program CpozUgSwe9FPLy9BLNhY2LTGqLUk1nirUkMMA5RmDw6t consumed 4018 of 200000 compute units',
'Program CpozUgSwe9FPLy9BLNhY2LTGqLUk1nirUkMMA5RmDw6t failed: custom program error: 0x0'
```
If you use Anchor's `init` constraint, that's usually all you need to protect
against reinitialization attacks! Remember, just because the fix for these
security exploits is simple doesn't mean it isn't important. Every time your
initialize an account, make sure you're either using the `init` constraint or
have some other check in place to avoid resetting an existing account's initial
state.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[this repository](https://github.com/Unboxed-Software/solana-reinitialization-attacks/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that instructions are
properly protected against reinitialization attacks.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=652c68aa-18d9-464c-9522-e531fd8738d5)!
# How to approach the Program Security module
Source: https://docs.orbition.network/courses/program-security/security-intro
Learn how to think intelligently about security for your onchain programs, whether developing in Anchor or in Native Rust.
## Overview
The goal of this course is to expose you to a wide variety of common security
exploits that are unique to Orbition Native Chain development. We’ve heavily modeled this
course off Coral's
[Sealevel Attacks](https://github.com/coral-xyz/sealevel-attacks) repo.
We've covered program security in our
[Anchor](/developers/courses/onchain-development/) and
[native Rust](/developers/courses/native-onchain-development/) development
courses because we wanted to make sure that anyone deploying programs to Mainnet
right out of the gates had at least a basic understanding of security. And if
that’s you then hopefully the fundamental principles you learned in that lesson
have led to you avoiding some common Orbition Native Chain exploits on your own.
This unit is meant to build on top of that lesson with two goals in mind:
1. To expand your awareness of the Orbition Native Chain programming model and the areas where
you need to focus to close up security loopholes in your programs
2. To show you the array of tools provided by Anchor to help you keep your
programs secure
If you went through the Basic Security lesson, the first few lessons should seem
familiar. They largely cover topics we discussed in that lesson. After that,
some of the attacks may seem new. We encourage you to go through all of them.
The last thing to call out is that there are a lot more lessons in this course
than in prior course. And the lessons aren't dependent on each other in the same
ways, so you can bounce around a bit more if you'd like.
Originally, we were going to have more, shorter lessons in this course. And
while they might be shorter than average, they aren't much shorter. It turns out
that even though each of the security vulnerabilities is "simple," there's a lot
to discuss. So each lesson may have a little bit less prose and more code
snippets, making it easy for readers to choose how in depth to go. But,
ultimately, each lesson is still as fully-fledged as they have been before so
that you can really get a solid grasp on each of the discussed security risks.
As always, we appreciate feedback. Good luck digging in!
# Signer Authorization
Source: https://docs.orbition.network/courses/program-security/signer-auth
Ensure instructions are only ran by authorized accounts by implmementing Signer checks.
## Summary
* Use **Signer Checks** to verify that specific accounts have signed a
transaction. Without appropriate signer checks, accounts may be able to
execute instructions they shouldn’t be authorized to perform.
* To implement a signer check in Rust, simply check that an account’s
`is_signer` property is `true`
```rust theme={null}
if !ctx.accounts.authority.is_signer {
return Err(ProgramError::MissingRequiredSignature.into());
}
```
* In Anchor, you can use the **`Signer`** account type in your account
validation struct to have Anchor automatically perform a signer check on a
given account
* Anchor also has an account constraint that will automatically verify that a
given account has signed a transaction
## Lesson
Signer checks are used to verify that a given account’s owner has authorized a
transaction. Without a signer check, operations whose execution should be
limited to only specific accounts can potentially be performed by any account.
In the worst case scenario, this could result in wallets being completely
drained by attackers passing in whatever account they want to an instruction.
#### Missing Signer Check
The example below shows an oversimplified version of an instruction that updates
the `authority` field stored on a program account.
Notice that the `authority` field on the `UpdateAuthority` account validation
struct is of type `AccountInfo`. In Anchor, the `AccountInfo` account type
indicates that no checks are performed on the account prior to instruction
execution.
Although the `has_one` constraint is used to validate the `authority` account
passed into the instruction matches the `authority` field stored on the `vault`
account, there is no check to verify the `authority` account authorized the
transaction.
This means an attacker can simply pass in the public key of the `authority`
account and their own public key as the `new_authority` account to reassign
themselves as the new authority of the `vault` account. At that point, they can
interact with the program as the new authority.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod insecure_update{
use super::*;
...
pub fn update_authority(ctx: Context) -> Result<()> {
ctx.accounts.vault.authority = ctx.accounts.new_authority.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAuthority<'info> {
#[account(
mut,
has_one = authority
)]
pub vault: Account<'info, Vault>,
pub new_authority: AccountInfo<'info>,
pub authority: AccountInfo<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
#### Add signer authorization checks
All you need to do to validate that the `authority` account signed is to add a
signer check within the instruction. That simply means checking that
`authority.is_signer` is `true`, and returning a `MissingRequiredSignature`
error if `false`.
```typescript theme={null}
if !ctx.accounts.authority.is_signer {
return Err(ProgramError::MissingRequiredSignature.into());
}
```
By adding a signer check, the instruction would only process if the account
passed in as the `authority` account also signed the transaction. If the
transaction was not signed by the account passed in as the `authority` account,
then the transaction would fail.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod secure_update{
use super::*;
...
pub fn update_authority(ctx: Context) -> Result<()> {
if !ctx.accounts.authority.is_signer {
return Err(ProgramError::MissingRequiredSignature.into());
}
ctx.accounts.vault.authority = ctx.accounts.new_authority.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAuthority<'info> {
#[account(
mut,
has_one = authority
)]
pub vault: Account<'info, Vault>,
pub new_authority: AccountInfo<'info>,
pub authority: AccountInfo<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
#### Use Anchor’s `Signer` account type
However, putting this check into the instruction function muddles the separation
between account validation and instruction logic.
Fortunately, Anchor makes it easy to perform signer checks by providing the
`Signer` account type. Simply change the `authority` account’s type in the
account validation struct to be of type `Signer`, and Anchor will check at
runtime that the specified account is a signer on the transaction. This is the
approach we generally recommend since it allows you to separate the signer check
from instruction logic.
In the example below, if the `authority` account does not sign the transaction,
then the transaction will fail before even reaching the instruction logic.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod secure_update{
use super::*;
...
pub fn update_authority(ctx: Context) -> Result<()> {
ctx.accounts.vault.authority = ctx.accounts.new_authority.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAuthority<'info> {
#[account(
mut,
has_one = authority
)]
pub vault: Account<'info, Vault>,
pub new_authority: AccountInfo<'info>,
pub authority: Signer<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
Note that when you use the `Signer` type, no other ownership or type checks are
performed.
#### Use Anchor’s `#[account(signer)]` constraint
While in most cases, the `Signer` account type will suffice to ensure an account
has signed a transaction, the fact that no other ownership or type checks are
performed means that this account can’t really be used for anything else in the
instruction.
This is where the `signer` *constraint* comes in handy. The `#[account(signer)]`
constraint allows you to verify the account signed the transaction, while also
getting the benefits of using the `Account` type if you wanted access to it’s
underlying data as well.
As an example of when this would be useful, imagine writing an instruction that
you expect to be invoked via CPI that expects one of the passed in accounts to
be both a **\*\***signer\*\*\*\*\*\* on the transaciton and a \***\*\*\*\*\*\***data
source\***\*\*\*\*\*\***. Using the `Signer` account type here removes the
automatic deserialization and type checking you would get with the `Account`
type. This is both inconvenient, as you need to manually deserialize the account
data in the instruction logic, and may make your program vulnerable by not
getting the ownership and type checking performed by the `Account` type.
In the example below, you can safely write logic to interact with the data
stored in the `authority` account while also verifying that it signed the
transaction.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod secure_update{
use super::*;
...
pub fn update_authority(ctx: Context) -> Result<()> {
ctx.accounts.vault.authority = ctx.accounts.new_authority.key();
// access the data stored in authority
msg!("Total number of depositors: {}", ctx.accounts.authority.num_depositors);
Ok(())
}
}
#[derive(Accounts)]
pub struct UpdateAuthority<'info> {
#[account(
mut,
has_one = authority
)]
pub vault: Account<'info, Vault>,
pub new_authority: AccountInfo<'info>,
#[account(signer)]
pub authority: Account<'info, AuthState>
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
#[account]
pub struct AuthState{
amount: u64,
num_depositors: u64,
num_vaults: u64
}
```
## Lab
Let’s practice by creating a simple program to demonstrate how a missing signer
check can allow an attacker to withdraw tokens that don’t belong to them.
This program initializes a simplified token “vault” account and demonstrates how
a missing signer check could allow the vault to be drained.
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-signer-auth/tree/starter). The
starter code includes a program with two instructions and the boilerplate setup
for the test file.
The `initialize_vault` instruction initializes two new accounts: `Vault` and
`TokenAccount`. The `Vault` account will be initialized using a Program Derived
Address (PDA) and store the address of a token account and the authority of the
vault. The authority of the token account will be the `vault` PDA which enables
the program to sign for the transfer of tokens.
The `insecure_withdraw` instruction will transfer tokens in the `vault`
account’s token account to a `withdraw_destination` token account. However, the
`authority` account in the `InsecureWithdraw` struct has a type of
`UncheckedAccount`. This is a wrapper around `AccountInfo` to explicitly
indicate the account is unchecked.
Without a signer check, anyone can simply provide the public key of the
`authority` account that matches `authority` stored on the `vault` account and
the `insecure_withdraw` instruction would continue to process.
While this is somewhat contrived in that any DeFi program with a vault would be
more sophisticated than this, it will show how the lack of a signer check can
result in tokens being withdrawn by the wrong party.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod signer_authorization {
use super::*;
pub fn initialize_vault(ctx: Context) -> Result<()> {
ctx.accounts.vault.token_account = ctx.accounts.token_account.key();
ctx.accounts.vault.authority = ctx.accounts.authority.key();
Ok(())
}
pub fn insecure_withdraw(ctx: Context) -> Result<()> {
let amount = ctx.accounts.token_account.amount;
let seeds = &[b"vault".as_ref(), &[*ctx.bumps.get("vault").unwrap()]];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.vault.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
#[derive(Accounts)]
pub struct InitializeVault<'info> {
#[account(
init,
payer = authority,
space = 8 + 32 + 32,
seeds = [b"vault"],
bump
)]
pub vault: Account<'info, Vault>,
#[account(
init,
payer = authority,
token::mint = mint,
token::authority = vault,
)]
pub token_account: Account<'info, TokenAccount>,
pub mint: Account<'info, Mint>,
#[account(mut)]
pub authority: Signer<'info>,
pub token_program: Program<'info, Token>,
pub system_program: Program<'info, System>,
pub rent: Sysvar<'info, Rent>,
}
#[derive(Accounts)]
pub struct InsecureWithdraw<'info> {
#[account(
seeds = [b"vault"],
bump,
has_one = token_account,
has_one = authority
)]
pub vault: Account<'info, Vault>,
#[account(mut)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
/// CHECK: demo missing signer check
pub authority: UncheckedAccount<'info>,
}
#[account]
pub struct Vault {
token_account: Pubkey,
authority: Pubkey,
}
```
#### 2. Test `insecure_withdraw` instruction
The test file includes the code to invoke the `initialize_vault` instruction
using `wallet` as the `authority` on the vault. The code then mints 100 tokens
to the `vault` token account. Theoretically, the `wallet` key should be the only
one that can withdraw the 100 tokens from the vault.
Now, let’s add a test to invoke `insecure_withdraw` on the program to show that
the current version of the program allows a third party to in fact withdraw
those 100 tokens.
In the test, we’ll still use the public key of `wallet` as the `authority`
account, but we’ll use a different keypair to sign and send the transaction.
```typescript theme={null}
describe("signer-authorization", () => {
...
it("Insecure withdraw", async () => {
const tx = await program.methods
.insecureWithdraw()
.accounts({
vault: vaultPDA,
tokenAccount: tokenAccount.publicKey,
withdrawDestination: withdrawDestinationFake,
authority: wallet.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
const balance = await connection.getTokenAccountBalance(
tokenAccount.publicKey
)
expect(balance.value.uiAmount).to.eq(0)
})
})
```
Run `anchor test` to see that both transactions will complete successfully.
```bash theme={null}
signer-authorization
✔ Initialize Vault (810ms)
✔ Insecure withdraw (405ms)
```
Since there is no signer check for the `authority` account, the
`insecure_withdraw` instruction will transfer tokens from the `vault` token
account to the `withdrawDestinationFake` token account as long as the public key
of the`authority` account matches the public key stored on the authority field
of the `vault` account. Clearly, the `insecure_withdraw` instruction is as
insecure as the name suggests.
#### 3. Add `secure_withdraw` instruction
Let’s fix the problem in a new instruction called `secure_withdraw`. This
instruction will be identical to the `insecure_withdraw` instruction, except
we’ll use the `Signer` type in the Accounts struct to validate the `authority`
account in the `SecureWithdraw` struct. If the `authority` account is not a
signer on the transaction, then we expect the transaction to fail and return an
error.
```rust theme={null}
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Mint, Token, TokenAccount};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod signer_authorization {
use super::*;
...
pub fn secure_withdraw(ctx: Context) -> Result<()> {
let amount = ctx.accounts.token_account.amount;
let seeds = &[b"vault".as_ref(), &[*ctx.bumps.get("vault").unwrap()]];
let signer = [&seeds[..]];
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.token_program.to_account_info(),
token::Transfer {
from: ctx.accounts.token_account.to_account_info(),
authority: ctx.accounts.vault.to_account_info(),
to: ctx.accounts.withdraw_destination.to_account_info(),
},
&signer,
);
token::transfer(cpi_ctx, amount)?;
Ok(())
}
}
#[derive(Accounts)]
pub struct SecureWithdraw<'info> {
#[account(
seeds = [b"vault"],
bump,
has_one = token_account,
has_one = authority
)]
pub vault: Account<'info, Vault>,
#[account(mut)]
pub token_account: Account<'info, TokenAccount>,
#[account(mut)]
pub withdraw_destination: Account<'info, TokenAccount>,
pub token_program: Program<'info, Token>,
pub authority: Signer<'info>,
}
```
#### 4. Test `secure_withdraw` instruction
With the instruction in place, return to the test file to test the
`secure_withdraw` instruction. Invoke the `secure_withdraw` instruction, again
using the public key of `wallet` as the `authority` account and the
`withdrawDestinationFake` keypair as the signer and withdraw destination. Since
the `authority` account is validated using the `Signer` type, we expect the
transaction to fail the signer check and return an error.
```typescript theme={null}
describe("signer-authorization", () => {
...
it("Secure withdraw", async () => {
try {
const tx = await program.methods
.secureWithdraw()
.accounts({
vault: vaultPDA,
tokenAccount: tokenAccount.publicKey,
withdrawDestination: withdrawDestinationFake,
authority: wallet.publicKey,
})
.transaction()
await anchor.web3.sendAndConfirmTransaction(connection, tx, [walletFake])
} catch (err) {
expect(err)
console.log(err)
}
})
})
```
Run `anchor test` to see that the transaction will now return a signature
verification error.
```bash theme={null}
Error: Signature verification failed
```
That’s it! This is a fairly simple thing to avoid, but incredibly important.
Make sure to always think through who should who should be authorizing
instructions and make sure that each is a signer on the transaction.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the repository](https://github.com/Unboxed-Software/solana-signer-auth/tree/solution).
## Challenge
At this point in the course, we hope you've started to work on programs and
projects outside the labs and Challenges provided in these lessons. For this and
the remainder of the lessons on security vulnerabilities, the Challenge for each
lesson will be to audit your own code for the security vulnerability discussed
in the lesson.
Alternatively, you can find open source programs to audit. There are plenty of
programs you can look at. A good start if you don't mind diving into native Rust
would be the
[SPL programs](https://github.com/orbition-network/solana-program-library).
So for this lesson, take a look at a program (whether yours or one you've found
online) and audit it for signer checks. If you find a bug in somebody else's
program, please alert them! If you find a bug in your own program, be sure to
patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=26b3f41e-8241-416b-9cfa-05c5ab519d80)!
# Type Cosplay
Source: https://docs.orbition.network/courses/program-security/type-cosplay
Under the risks of accounts of the wrong type being used in instruction, and use account type checks to mitigate them.
## Summary
* Use discriminators to distinguish between different account types
* To implement a discriminator in Rust, include a field in the account struct to
represent the account type
```rust theme={null}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
discriminant: AccountDiscriminant,
user: Pubkey,
}
#[derive(BorshSerialize, BorshDeserialize, PartialEq)]
pub enum AccountDiscriminant {
User,
Admin,
}
```
* To implement a discriminator check in Rust, verify that the discriminator of
the deserialized account data matches the expected value
```rust theme={null}
if user.discriminant != AccountDiscriminant::User {
return Err(ProgramError::InvalidAccountData.into());
}
```
* In Anchor, program account types automatically implement the `Discriminator`
trait which creates an 8 byte unique identifier for a type
* Use Anchor’s `Account<'info, T>` type to automatically check the discriminator
of the account when deserializing the account data
## Lesson
“Type cosplay” refers to an unexpected account type being used in place of an
expected account type. Under the hood, account data is simply stored as an array
of bytes that a program deserializes into a custom account type. Without
implementing a way to explicitly distinguish between account types, account data
from an unexpected account could result in an instruction being used in
unintended ways.
#### Unchecked account
In the example below, both the `AdminConfig` and `UserConfig` account types
store a single public key. The `admin_instruction` instruction deserializes the
`admin_config` account as an `AdminConfig` type and then performs a owner check
and data validation check.
However, the `AdminConfig` and `UserConfig` account types have the same data
structure. This means a `UserConfig` account type could be passed in as the
`admin_config` account. As long as the public key stored on the account data
matches the `admin` signing the transaction, the `admin_instruction` instruction
would continue to process, even if the signer isn't actually an admin.
Note that the names of the fields stored on the account types (`admin` and
`user`) make no difference when deserializing account data. The data is
serialized and deserialized based on the order of fields rather than their
names.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod type_cosplay_insecure {
use super::*;
pub fn admin_instruction(ctx: Context) -> Result<()> {
let account_data =
AdminConfig::try_from_slice(&ctx.accounts.admin_config.data.borrow()).unwrap();
if ctx.accounts.admin_config.owner != ctx.program_id {
return Err(ProgramError::IllegalOwner.into());
}
if account_data.admin != ctx.accounts.admin.key() {
return Err(ProgramError::InvalidAccountData.into());
}
msg!("Admin {}", account_data.admin);
Ok(())
}
}
#[derive(Accounts)]
pub struct AdminInstruction<'info> {
admin_config: UncheckedAccount<'info>,
admin: Signer<'info>,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct AdminConfig {
admin: Pubkey,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct UserConfig {
user: Pubkey,
}
```
#### Add account discriminator
To solve this, you can add a discriminant field for each account type and set
the discriminant when initializing an account.
The example below updates the `AdminConfig` and `UserConfig` account types with
a `discriminant` field. The `admin_instruction` instruction includes an
additional data validation check for the `discriminant` field.
```rust theme={null}
if account_data.discriminant != AccountDiscriminant::Admin {
return Err(ProgramError::InvalidAccountData.into());
}
```
If the `discriminant` field of the account passed into the instruction as the
`admin_config` account does not match the expected `AccountDiscriminant`, then
the transaction will fail. Simply make sure to set the appropriate value for
`discriminant` when you initialize each account (not shown in the example), and
then you can include these discriminant checks in every subsequent instruction.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod type_cosplay_secure {
use super::*;
pub fn admin_instruction(ctx: Context) -> Result<()> {
let account_data =
AdminConfig::try_from_slice(&ctx.accounts.admin_config.data.borrow()).unwrap();
if ctx.accounts.admin_config.owner != ctx.program_id {
return Err(ProgramError::IllegalOwner.into());
}
if account_data.admin != ctx.accounts.admin.key() {
return Err(ProgramError::InvalidAccountData.into());
}
if account_data.discriminant != AccountDiscriminant::Admin {
return Err(ProgramError::InvalidAccountData.into());
}
msg!("Admin {}", account_data.admin);
Ok(())
}
}
#[derive(Accounts)]
pub struct AdminInstruction<'info> {
admin_config: UncheckedAccount<'info>,
admin: Signer<'info>,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct AdminConfig {
discriminant: AccountDiscriminant,
admin: Pubkey,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct UserConfig {
discriminant: AccountDiscriminant,
user: Pubkey,
}
#[derive(BorshSerialize, BorshDeserialize, PartialEq)]
pub enum AccountDiscriminant {
Admin,
User,
}
```
#### Use Anchor’s `Account` wrapper
Implementing these checks for every account needed for every instruction can be
tedious. Fortunately, Anchor provides a `#[account]` attribute macro for
automatically implementing traits that every account should have.
Structs marked with `#[account]` can then be used with `Account` to validate
that the passed in account is indeed the type you expect it to be. When
initializing an account whose struct representation has the `#[account]`
attribute, the first 8 bytes are automatically reserved for a discriminator
unique to the account type. When deserializing the account data, Anchor will
automatically check if the discriminator on the account matches the expected
account type and throw and error if it does not match.
In the example below, `Account<'info, AdminConfig>` specifies that the
`admin_config` account should be of type `AdminConfig`. Anchor then
automatically checks that the first 8 bytes of account data match the
discriminator of the `AdminConfig` type.
The data validation check for the `admin` field is also moved from the
instruction logic to the account validation struct using the `has_one`
constraint. `#[account(has_one = admin)]` specifies that the `admin_config`
account’s `admin` field must match the `admin` account passed into the
instruction. Note that for the `has_one` constraint to work, the naming of the
account in the struct must match the naming of field on the account you are
validating.
```rust theme={null}
use anchor_lang::prelude::*;
use borsh::{BorshDeserialize, BorshSerialize};
declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");
#[program]
pub mod type_cosplay_recommended {
use super::*;
pub fn admin_instruction(ctx: Context) -> Result<()> {
msg!("Admin {}", ctx.accounts.admin_config.admin);
Ok(())
}
}
#[derive(Accounts)]
pub struct AdminInstruction<'info> {
#[account(has_one = admin)]
admin_config: Account<'info, AdminConfig>,
admin: Signer<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
#[account]
pub struct UserConfig {
user: Pubkey,
}
```
It’s important to note that this is a vulnerability you don’t really have to
worry about when using Anchor - that’s the whole point of it in the first place!
After going through how this can be exploited if not handled properly in a
native rust program, hopefully you have a much better understanding of what the
purpose of the account discriminator is in an Anchor account. The fact that
Anchor sets and checks this discriminator automatically means that developers
can spend more time focusing on their product, but it’s still very important to
understand what Anchor is doing behind the scenes to develop robust Orbition Native Chain
programs.
## Lab
For this lab we’ll create two programs to demonstrate a type cosplay
vulnerability.
* The first program will initialize program accounts without a discriminator
* The second program will initialize program accounts using Anchor’s `init`
constraint which automatically sets an account discriminator
#### 1. Starter
To get started, download the starter code from the `starter` branch of
[this repository](https://github.com/Unboxed-Software/solana-type-cosplay/tree/starter). The
starter code includes a program with three instructions and some tests.
The three instructions are:
1. `initialize_admin` - initializes an admin account and sets the admin
authority of the program
2. `initialize_user` - intializes a standard user account
3. `update_admin` - allows the existing admin to update the admin authority of
the program
Take a look at these three instructions in the `lib.rs` file. The last
instruction should only be callable by the account matching the `admin` field on
the admin account initialized using the `initialize_admin` instruction.
#### 2. Test insecure `update_admin` instruction
However, both accounts have the same fields and field types:
```rust theme={null}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct AdminConfig {
admin: Pubkey,
}
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
user: Pubkey,
}
```
Because of this, it's possible to pass in a `User` account in place of the
`admin` account in the `update_admin` instruction, thereby bypassing the
requirement that one be an admin to call this instruction.
Take a look at the `solana-type-cosplay.ts` file in the `tests` directory. It
contains some basic setup and two tests. One test initializes a user account,
and the other invokes `update_admin` and passes in the user account in place of
an admin account.
Run `anchor test` to see that invoking `update_admin` will complete
successfully.
```bash theme={null}
type-cosplay
✔ Initialize User Account (233ms)
✔ Invoke update admin instruction with user account (487ms)
```
#### 3. Create `type-checked` program
Now we'll create a new program called `type-checked` by running
`anchor new type-checked` from the root of the existing anchor program.
Now in your `programs` folder you will have two programs. Run `anchor keys list`
and you should see the program ID for the new program. Add it to the `lib.rs`
file of the `type-checked` program and to the `type_checked` program in the
`Anchor.toml` file.
Next, update the test file's setup to include the new program and two new
keypairs for the accounts we'll be initializing for the new program.
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { TypeCosplay } from "../target/types/type_cosplay";
import { TypeChecked } from "../target/types/type_checked";
import { expect } from "chai";
describe("type-cosplay", () => {
const provider = anchor.AnchorProvider.env();
anchor.setProvider(provider);
const program = anchor.workspace.TypeCosplay as Program;
const programChecked = anchor.workspace.TypeChecked as Program;
const userAccount = anchor.web3.Keypair.generate();
const newAdmin = anchor.web3.Keypair.generate();
const userAccountChecked = anchor.web3.Keypair.generate();
const adminAccountChecked = anchor.web3.Keypair.generate();
});
```
#### 4. Implement the `type-checked` program
In the `type_checked` program, add two instructions using the `init` constraint
to initialize an `AdminConfig` account and a `User` account. When using the
`init` constraint to initialize new program accounts, Anchor will automatically
set the first 8 bytes of account data as a unique discriminator for the account
type.
We’ll also add an `update_admin` instruction that validates the `admin_config`
account as a `AdminConfig` account type using Anchor’s `Account` wrapper. For
any account passed in as the `admin_config` account, Anchor will automatically
check that the account discriminator matches the expected account type.
```rust theme={null}
use anchor_lang::prelude::*;
declare_id!("FZLRa6vX64QL6Vj2JkqY1Uzyzjgi2PYjCABcDabMo8U7");
#[program]
pub mod type_checked {
use super::*;
pub fn initialize_admin(ctx: Context) -> Result<()> {
ctx.accounts.admin_config.admin = ctx.accounts.admin.key();
Ok(())
}
pub fn initialize_user(ctx: Context) -> Result<()> {
ctx.accounts.user_account.user = ctx.accounts.user.key();
Ok(())
}
pub fn update_admin(ctx: Context) -> Result<()> {
ctx.accounts.admin_config.admin = ctx.accounts.admin.key();
Ok(())
}
}
#[derive(Accounts)]
pub struct InitializeAdmin<'info> {
#[account(
init,
payer = admin,
space = 8 + 32
)]
pub admin_config: Account<'info, AdminConfig>,
#[account(mut)]
pub admin: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[derive(Accounts)]
pub struct InitializeUser<'info> {
#[account(
init,
payer = user,
space = 8 + 32
)]
pub user_account: Account<'info, User>,
#[account(mut)]
pub user: Signer<'info>,
pub system_program: Program<'info, System>,
}
#[derive(Accounts)]
pub struct UpdateAdmin<'info> {
#[account(
mut,
has_one = admin
)]
pub admin_config: Account<'info, AdminConfig>,
pub new_admin: SystemAccount<'info>,
#[account(mut)]
pub admin: Signer<'info>,
}
#[account]
pub struct AdminConfig {
admin: Pubkey,
}
#[account]
pub struct User {
user: Pubkey,
}
```
#### 5. Test secure `update_admin` instruction
In the test file, we’ll initialize an `AdminConfig` account and a `User` account
from the `type_checked` program. Then we’ll invoke the `updateAdmin` instruction
twice passing in the newly created accounts.
```typescript theme={null}
describe("type-cosplay", () => {
...
it("Initialize type checked AdminConfig Account", async () => {
await programChecked.methods
.initializeAdmin()
.accounts({
adminConfig: adminAccountType.publicKey,
})
.signers([adminAccountType])
.rpc()
})
it("Initialize type checked User Account", async () => {
await programChecked.methods
.initializeUser()
.accounts({
userAccount: userAccountType.publicKey,
user: provider.wallet.publicKey,
})
.signers([userAccountType])
.rpc()
})
it("Invoke update instruction using User Account", async () => {
try {
await programChecked.methods
.updateAdmin()
.accounts({
adminConfig: userAccountType.publicKey,
newAdmin: newAdmin.publicKey,
admin: provider.wallet.publicKey,
})
.rpc()
} catch (err) {
expect(err)
console.log(err)
}
})
it("Invoke update instruction using AdminConfig Account", async () => {
await programChecked.methods
.updateAdmin()
.accounts({
adminConfig: adminAccountType.publicKey,
newAdmin: newAdmin.publicKey,
admin: provider.wallet.publicKey,
})
.rpc()
})
})
```
Run `anchor test`. For the transaction where we pass in the `User` account type,
we expect the instruction and return an Anchor Error for the account not being
of type `AdminConfig`.
```bash theme={null}
'Program EU66XDppFCf2Bg7QQr59nyykj9ejWaoW93TSkk1ufXh3 invoke [1]',
'Program log: Instruction: UpdateAdmin',
'Program log: AnchorError caused by account: admin_config. Error Code: AccountDiscriminatorMismatch. Error Number: 3002. Error Message: 8 byte discriminator did not match what was expected.',
'Program EU66XDppFCf2Bg7QQr59nyykj9ejWaoW93TSkk1ufXh3 consumed 4765 of 200000 compute units',
'Program EU66XDppFCf2Bg7QQr59nyykj9ejWaoW93TSkk1ufXh3 failed: custom program error: 0xbba'
```
Following Anchor best practices and using Anchor types will ensure that your
programs avoid this vulnerability. Always use the `#[account]` attribute when
creating account structs, use the `init` constraint when initializing accounts,
and use the `Account` type in your account validation structs.
If you want to take a look at the final solution code you can find it on the
`solution` branch of
[the repository](https://github.com/Unboxed-Software/solana-type-cosplay/tree/solution).
## Challenge
Just as with other lessons in this unit, your opportunity to practice avoiding
this security exploit lies in auditing your own or other programs.
Take some time to review at least one program and ensure that account types have
a discriminator and that those are checked for each account and instruction.
Since standard Anchor types handle this check automatically, you're more likely
to find a vulnerability in a native program.
Remember, if you find a bug or exploit in somebody else's program, please alert
them! If you find one in your own program, be sure to patch it right away.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=37ebccab-b19a-43c6-a96a-29fa7e80fdec)!
# Compressed NFTs
Source: https://docs.orbition.network/courses/state-compression/compressed-nfts
How to mint, transfer and read large-scale NFT collections using Metaplex's Bubblegum SDK.
## Summary
* **Compressed NFTs (cNFTs)** use **State Compression** to hash NFT data and
store the hash onchain in an account using a **concurrent Merkle tree**
structure
* The cNFT data hash can’t be used to infer the cNFT data, but it can be used to
**verify** if the cNFT data you’re seeing is correct
* Supporting RPC providers **index** cNFT data off-chain when the cNFT is minted
so that you can use the **Read API** to access the data
* The **Metaplex Bubblegum program** is an abstraction on top of the **State
Compression** program that enables you to more simply create, mint, and manage
cNFT collections
## Lesson
Compressed NFTs (cNFTs) are exactly what their name suggests: NFTs whose
structure takes up less account storage than traditional NFTs. Compressed NFTs
leverage a concept called **State Compression** to store data in a way that
drastically reduces costs.
Orbition Native Chain’s transaction costs are so cheap that most users never think about how
expensive minting NFTs can be at scale. The cost to set up and mint 1 million
traditional NFTs is approximately 24,000 OBN. By comparison, cNFTs can be
structured to where the same setup and mint costs 10 OBN or less. That means
anyone using NFTs at scale could cut costs by more than 1000x by using cNFTs
over traditional NFTs.
However, cNFTs can be tricky to work with. Eventually, the tooling required to
work with them will be sufficiently abstracted from the underlying technology
that the developer experience between traditional NFTs and cNFTs will be
negligible. But for now, you’ll still need to understand the low level puzzle
pieces, so let’s dig in!
### A theoretical overview of cNFTs
Most of the costs associated with traditional NFTs come down to account storage
space. Compressed NFTs use a concept called State Compression to store data in
the blockchain’s cheaper **ledger state**, using more expensive account space
only to store a “fingerprint”, or **hash**, of the data. This hash allows you to
cryptographically verify that data has not been tampered with.
To both store hashes and enable verification, we use a special binary tree
structure known as a **concurrent Merkle tree**. This tree structure lets us
hash data together in a deterministic way to compute a single, final hash that
gets stored onchain. This final hash is significantly smaller in size than all
the original data combined, hence the “compression.” The steps to this process
are:
1. Take any piece of data
2. Create a hash of this data
3. Store this hash as a “leaf” at the bottom of the tree
4. Each leaf pair is then hashed together, creating a “branch”
5. Each branch is then hashed together
6. Continually climb the tree and hash adjacent branches together
7. Once at the top of the tree, a final ”root hash” is produced
8. Store the root hash onchain as a verifiable proof of the data within each
leaf
9. Anyone wanting to verify that the data they have matches the “source of
truth” can go through the same process and compare the final hash without
having to store all the data onchain
One problem not addressed in the above is how to make data available if it can’t
be fetched from an account. Since this hashing process occurs onchain, all the
data exists in the ledger state and could theoretically be retrieved from the
original transaction by replaying the entire chain state from origin. However,
it’s much more straightforward (though still complicated) to have an **indexer**
track and index this data as the transactions occur. This ensures there is an
off-chain “cache” of the data that anyone can access and subsequently verify
against the onchain root hash.
This process is *very complex*. We’ll cover some of the key concepts below but
don’t worry if you don’t understand it right away. We’ll talk more theory in the
state compression lesson and focus primarily on application to NFTs in this
lesson. You’ll be able to work with cNFTs by the end of this lesson even if you
don’t fully understand every piece of the state compression puzzle.
#### Concurrent Merkle trees
A **Merkle tree** is a binary tree structure represented by a single hash. Every
leaf node in the structure is a hash of its inner data while every branch is a
hash of its child leaf hashes. In turn, branches are also hashed together until
eventually one final root hash remains.
Any modification to leaf data changes the root hash. This causes an issue when
multiple transactions in the same slot are attempting to modify leaf data. Since
these transactions must execute in series, all but the first will fail since the
root hash and proof passed in will have been invalidated by the first
transaction to be executed.
A **concurrent Merkle tree** is a Merkle tree that stores a secure changelog of
the most recent changes along with their root hash and the proof to derive it.
When multiple transactions in the same slot try to modify leaf data, the
changelog can be used as a source of truth to allow for concurrent changes to be
made to the tree.
When working with a concurrent Merkle tree, there are three variables that
determine the size of the tree, the cost to create the tree, and the number of
concurrent changes that can be made to the tree:
1. Max depth
2. Max buffer size
3. Canopy depth
The **max depth** is the maximum number of hops to get from any leaf to the root
of the tree. Since Merkle trees are binary trees, every leaf is connected only
to one other leaf. Max depth can then logically be used to calculate the number
of nodes for the tree with `2 ^ maxDepth`.
The **max buffer size** is effectively the maximum number of concurrent changes
that you can make to a tree within a single slot with the root hash still being
valid.
The **canopy depth** is the number of proof nodes that are stored onchain for
any given proof path. Verifying any leaf requires the complete proof path for
the tree. The complete proof path is made up of one proof node for every “layer”
of the tree, i.e. a max depth of 14 means there are 14 proof nodes. Every proof
node adds 32 bytes to a transaction, so large trees would quickly exceed the
maximum transaction size limit without caching proof nodes onchain.
Each of these three values, max depth, max buffer size, and canopy depth, comes
with a tradeoff. Increasing the value of any of these values increases the size
of the account used to store the tree, thus increasing the cost to create the
tree.
Choosing the max depth is fairly straightforward as it directly relates to the
number of leafs and therefore the amount of data you can store. If you need
1million cNFTs on a single tree, find the max depth that makes the following
expression true: `2^maxDepth > 1million`. The answer is 20.
Choosing a max buffer size is effectively a question of throughput: how many
concurrent writes do you need.
#### SPL State Compression and Noop Programs
The SPL State Compression Program exists to make the above process repeatable
and composable throughout the Orbition Native Chain ecosystem. It provides instructions for
initializing Merkle trees, managing tree leafs (i.e. add, update, remove data),
and verifying leaf data.
The State Compression Program also leverages a separate “no op” program whose
primary purpose is to make leaf data easier to index by logging it to the ledger
state.
#### Use the Ledger State for storage
The Orbition Native Chain ledger is a list of entries containing signed transactions. In
theory, this can be traced back to the genesis block. This effectively means any
data that has ever been put into a transaction exists in the ledger.
When you want to store compressed data, you pass it to the State Compression
program where it gets hashed and emitted as an “event” to the Noop program. The
hash is then stored in the corresponding concurrent Merkle tree. Since the data
passed through a transaction and even exists on the Noop program logs, it will
forever exist on the ledger state.
#### Index data for easy lookup
Under normal conditions, you would typically access onchain data by fetching the
appropriate account. When using state compression, however, it’s not so
straightforward.
As mentioned above, the data now exists in the ledger state rather than in an
account. The easiest place to find the full data is in the logs of the Noop
instruction, but while this data will in a sense exist in the ledger state
forever, it will likely be inaccessible through validators after a certain
period of time.
To save space and be more performant, validators don’t retain every transaction
back to the genesis block. The specific amount of time you’ll be able to access
the Noop instruction logs related to your data will vary based on the validator,
but eventually you’ll lose access to it if you’re relying directly on
instruction logs.
Technically, you *can* replay transaction state back to the genesis block but
the average team isn’t going to do that, and it certainly won’t be performant.
Instead, you should use an indexer that will observe the events sent to the Noop
program and store the relevant data off chain. That way you don’t need to worry
about old data becoming inaccessible.
### Create a cNFT Collection
With the theoretical background out of the way, let’s turn our attention to the
main point of this lesson: how to create a cNFT collection.
Fortunately, you can use tools created by Orbition Native Chain Foundation, the Orbition Native Chain
developer community, and Metaplex to simplify the process. Specifically, we’ll
be using the `@orbition-network/spl-account-compression` SDK, the Metaplex Bubblegum
program, and the Bubblegum program’s corresponding TS SDK
`@metaplex-foundation/mpl-bugglegum`.
#### Prepare metadata
Prior to starting, you’ll prepare your NFT metadata similarly to how you would
if you were using a Candy Machine. At its core, an NFT is simply a token with
metadata that follows the NFT standard. In other words, it should be shaped
something like this:
```json theme={null}
{
"name": "12_217_47",
"symbol": "RGB",
"description": "Random RGB Color",
"seller_fee_basis_points": 0,
"image": "https://raw.githubusercontent.com/ZYJLiu/rgb-png-generator/master/assets/12_217_47/12_217_47.png",
"attributes": [
{
"trait_type": "R",
"value": "12"
},
{
"trait_type": "G",
"value": "217"
},
{
"trait_type": "B",
"value": "47"
}
]
}
```
Depending on your use case, you may be able to generate this dynamically or you
might want to have a JSON file prepared for each cNFT beforehand. You’ll also
need any other assets referenced by the JSON, such as the `image` url shown in
the example above.
#### Create Collection NFT
If you want your cNFTs to be part of a collection, you’ll need to create a
Collection NFT **before** you start minting cNFTs. This is a traditional NFT
that acts as the reference binding your cNFTs together into a single collection.
You can create this NFT using the `@metaplex-foundation/js` library. Just make
sure you set `isCollection` to `true`.
```typescript theme={null}
const collectionNft = await metaplex.nfts().create({
uri: someUri,
name: "Collection NFT",
sellerFeeBasisPoints: 0,
updateAuthority: somePublicKey,
mintAuthority: somePublicKey,
tokenStandard: 0,
symbol: "Collection",
isMutable: true,
isCollection: true,
});
```
#### Create Merkle tree Account
Now we start to deviate from the process you would use when creating traditional
NFTs. The onchain storage mechanism you use for state compression is an account
representing a concurrent Merkle tree. This Merkle tree account belongs to the
SPL State Compression program. Before you can do anything related to cNFTs, you
need to create an empty Merkle tree account with the appropriate size.
The variables impacting the size of the account are:
1. Max depth
2. Max buffer size
3. Canopy depth
The first two variables must be chosen from an existing set of valid pairs. The
table below shows the valid pairs along with the number of cNFTs that can be
created with those values.
| Max Depth | Max Buffer Size | Max Number of cNFTs |
| --------- | --------------- | ------------------- |
| 3 | 8 | 8 |
| 5 | 8 | 32 |
| 14 | 64 | 16,384 |
| 14 | 256 | 16,384 |
| 14 | 1,024 | 16,384 |
| 14 | 2,048 | 16,384 |
| 15 | 64 | 32,768 |
| 16 | 64 | 65,536 |
| 17 | 64 | 131,072 |
| 18 | 64 | 262,144 |
| 19 | 64 | 524,288 |
| 20 | 64 | 1,048,576 |
| 20 | 256 | 1,048,576 |
| 20 | 1,024 | 1,048,576 |
| 20 | 2,048 | 1,048,576 |
| 24 | 64 | 16,777,216 |
| 24 | 256 | 16,777,216 |
| 24 | 512 | 16,777,216 |
| 24 | 1,024 | 16,777,216 |
| 24 | 2,048 | 16,777,216 |
| 26 | 512 | 67,108,864 |
| 26 | 1,024 | 67,108,864 |
| 26 | 2,048 | 67,108,864 |
| 30 | 512 | 1,073,741,824 |
| 30 | 1,024 | 1,073,741,824 |
| 30 | 2,048 | 1,073,741,824 |
Note that the number of cNFTs that can be stored on the tree depends entirely on
the max depth, while the buffer size will determine the number of concurrent
changes (mints, transfers, etc.) within the same slot that can occur to the
tree. In other words, choose the max depth that corresponds to the number of
NFTs you need the tree to hold, then choose one of the options for max buffer
size based on the traffic you expect you’ll need to support.
Next, choose the canopy depth. Increasing the canopy depth increases the
composability of your cNFTs. Any time your or another developer’s code attempts
to verify a cNFT down the road, the code will have to pass in as many proof
nodes as there are “layers” in your tree. So for a max depth of 20, you’ll need
to pass in 20 proof nodes. Not only is this tedious, but since each proof node
is 32 bytes it’s possible to max out transaction sizes very quickly.
For example, if your tree has a very low canopy depth, an NFT marketplace may
only be able to support simple NFTs transfers rather than support an onchain
bidding system for your cNFTs. The canopy effectively caches proof nodes onchain
so you don’t have to pass all of them into the transaction, allowing for more
complex transactions.
Increasing any of these three values increases the size of the account, thereby
increasing the cost associated with creating it. Weigh the benefits accordingly
when choosing the values.
Once you know these values, you can use the `createAllocTreeIx` helper function
from the `@orbition-network/spl-account-compression` TS SDK to create the instruction for
creating the empty account.
```typescript theme={null}
import { createAllocTreeIx } from "@orbition-network/spl-account-compression"
const treeKeypair = Keypair.generate()
const allocTreeIx = await createAllocTreeIx(
connection,
treeKeypair.publicKey,
payer.publicKey,
{ maxDepth: 20; maxBufferSize: 256 },
canopyDepth
)
```
Note that this is simply a helper function for calculating the size required by
the account and creating the instruction to send to the System Program for
allocating the account. This function doesn’t interact with any
compression-specific programs yet.
#### Use Bubblegum to Initialize Your Tree
With the empty tree account created, you then use the Bubblegum program to
initialize the tree. In addition to the Merkle tree account, Bubblegum creates a
tree config account to add cNFT-specific tracking and functionality.
Version 0.7 of the `@metaplex-foundation/mpl-bubblegum` TS SDK provides the
helper function `createCreateTreeInstruction` for calling the `create_tree`
instruction on the Bubblegum program. As part of the call, you’ll need to derive
the `treeAuthority` PDA expected by the program. This PDA uses the tree’s
address as a seed.
```typescript theme={null}
import {
createAllocTreeIx,
SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
SPL_NOOP_PROGRAM_ID,
} from "@orbition-network/spl-account-compression"
import {
PROGRAM_ID as BUBBLEGUM_PROGRAM_ID,
createCreateTreeInstruction,
} from "@metaplex-foundation/mpl-bubblegum"
...
const [treeAuthority, _bump] = PublicKey.findProgramAddressSync(
[treeKeypair.publicKey.toBuffer()],
BUBBLEGUM_PROGRAM_ID
)
const createTreeIx = createCreateTreeInstruction(
{
treeAuthority,
merkleTree: treeKeypair.publicKey,
payer: payer.publicKey,
treeCreator: payer.publicKey,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
},
{
maxBufferSize: 256,
maxDepth: 20,
public: false,
},
BUBBLEGUM_PROGRAM_ID
)
```
The list below shows the required input for this helper function:
* `accounts` - An object representing the accounts required by the instruction.
This includes:
* `treeAuthority` - Bubblegum expects this to be a PDA derived using the
Merkle tree address as a seed
* `merkleTree` - The Merkle tree account
* `payer` - The address paying for transaction fees, rent, etc.
* `treeCreator` - The address to list as the tree creator
* `logWrapper` - The program to use to expose the data to indexers through
logs; this should be the address of the SPL Noop program unless you have
some other custom implementation
* `compressionProgram` - The compression program to use for initializing the
Merkle tree; this should be the address of the SPL State Compression program
unless you have some other custom implementation
* `args` - An object representing additional arguments required by the
instruction. This includes:
* `maxBufferSize` - The max buffer size of the Merkle tree
* `maxDepth` - The max depth of the Merkle tree
* `public` - When set to `true`, anyone will be able to mint cNFTs from the
tree; when set to `false`, only the tree creator or tree delegate will be
able to min cNFTs from the tree
When submitted, this will invoke the `create_tree` instruction on the Bubblegum
program. This instruction does three things:
1. Creates the tree config PDA account
2. Initializes the tree config account with appropriate initial values
3. Issues a CPI to the State Compression program to initialize the empty Merkle
tree account
Feel free to take a look at the program code
[here](https://github.com/metaplex-foundation/mpl-bubblegum/blob/main/programs/bubblegum/program/src/lib.rs#L887).
#### Mint cNFTs
With the Merkle tree account and its corresponding Bubblegum tree config account
initialized, it’s possible to mint cNFTs to the tree. The Bubblegum instruction
to use will be either `mint_v1` or `mint_to_collection_v1`, depending on whether
or not you want to the minted cNFT to be part of a collection.
Version 0.7 of the `@metaplex-foundation/mpl-bubblegum` TS SDK provides helper
functions `createMintV1Instruction` and `createMintToCollectionV1Instruction` to
make it easier for you to create the instructions.
Both functions will require you to pass in the NFT metadata and a list of
accounts required to mint the cNFT. Below is an example of minting to a
collection:
```typescript theme={null}
const mintWithCollectionIx = createMintToCollectionV1Instruction(
{
payer: payer.publicKey,
merkleTree: treeAddress,
treeAuthority,
treeDelegate: payer.publicKey,
leafOwner: destination,
leafDelegate: destination,
collectionAuthority: payer.publicKey,
collectionAuthorityRecordPda: BUBBLEGUM_PROGRAM_ID,
collectionMint: collectionDetails.mint,
collectionMetadata: collectionDetails.metadata,
editionAccount: collectionDetails.masterEditionAccount,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
logWrapper: SPL_NOOP_PROGRAM_ID,
bubblegumSigner,
tokenMetadataProgram: TOKEN_METADATA_PROGRAM_ID,
},
{
metadataArgs: Object.assign(nftMetadata, {
collection: { key: collectionDetails.mint, verified: false },
}),
},
);
```
Notice that there are two arguments for the helper function: `accounts` and
`args`. The `args` parameter is simply the NFT metadata, while `accounts` is an
object listing the accounts required by the instruction. There are admittedly a
lot of them:
* `payer` - the account that will pay for the transaction fees, rent, etc.
* `merkleTree` - the Merkle tree account
* `treeAuthority` - the tree authority; should be the same PDA you derived
previously
* `treeDelegate` - the tree delegate; this is usually the same as the tree
creator
* `leafOwner` - the desired owner of the compressed NFT being minted
* `leafDelegate` - the desired delegate of the compressed NFT being minted; this
is usually the same as the leaf owner
* `collectionAuthority` - the authority of the collection NFT
* `collectionAuthorityRecordPda` - optional collection authority record PDA;
there typically is none, in which case you should put the Bubblegum program
address
* `collectionMint` - the mint account for the collection NFT
* `collectionMetadata` - the metadata account for the collection NFT
* `editionAccount` - the master edition account of the collection NFT
* `compressionProgram` - the compression program to use; this should be the
address of the SPL State Compression program unless you have some other custom
implementation
* `logWrapper` - the program to use to expose the data to indexers through logs;
this should be the address of the SPL Noop program unless you have some other
custom implementation
* `bubblegumSigner` - a PDA used by the Bubblegrum program to handle collection
verification
* `tokenMetadataProgram` - the token metadata program that was used for the
collection NFT; this is usually always the Metaplex Token Metadata program
Minting without a collection requires fewer accounts, none of which are
exclusive to minting without a collection. You can take a look at the example
below.
```typescript theme={null}
const mintWithoutCollectionIx = createMintV1Instruction(
{
payer: payer.publicKey,
merkleTree: treeAddress,
treeAuthority,
treeDelegate: payer.publicKey,
leafOwner: destination,
leafDelegate: destination,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
logWrapper: SPL_NOOP_PROGRAM_ID,
},
{
message: nftMetadata,
},
);
```
### Interact with cNFTs
It’s important to note that cNFTs *are not* SPL tokens. That means your code
needs to follow different conventions to handle cNFT functionality like
fetching, querying, transferring, etc.
#### Fetch cNFT data
The simplest way to fetch data from an existing cNFT is to use the
[Digital Asset Standard Read API](https://orbition.network/developers/guides/javascript/compressed-nfts#reading-compressed-nfts-metadata)
(Read API). Note that this is separate from the standard JSON RPC. To use the
Read API, you’ll need to use a supporting RPC Provider. Metaplex maintains a
(likely non-exhaustive)
[list of RPC providers](https://developers.metaplex.com/bubblegum/rpcs) that
support the Read API. In this lesson we’ll be using
[Helius](https://docs.helius.dev/compression-and-das-api/digital-asset-standard-das-api)
as they have free support for Testnet.
To use the Read API to fetch a specific cNFT, you need to have the cNFT’s asset
ID. However, after minting cNFTs, you’ll have at most two pieces of information:
1. The transaction signature
2. The leaf index (possibly)
The only real guarantee is that you’ll have the transaction signature. It is
**possible** to locate the leaf index from there, but it involves some fairly
complex parsing. The short story is you must retrieve the relevant instruction
logs from the Noop program and parse them to find the leaf index. We’ll cover
this more in depth in a future lesson. For now, we’ll assume you know the leaf
index.
This is a reasonable assumption for most mints given that the minting will be
controlled by your code and can be set up sequentially so that your code can
track which index is going to be used for each mint. I.e. the first mint will
use index 0, the second index 1, etc.
Once you have the leaf index, you can derive the cNFT’s corresponding asset ID.
When using Bubblegum, the asset ID is a PDA derived using the Bubblegum program
ID and the following seeds:
1. The static string `asset` represented in utf8 encoding
2. The Merkle tree address
3. The leaf index
The indexer essentially observes transaction logs from the Noop program as they
happen and stores the cNFT metadata that was hashed and stored in the Merkle
tree. This enables them to surface that data when requested. This asset id is
what the indexer uses to identify the particular asset.
For simplicity, you can just use the `getLeafAssetId` helper function from the
Bubblegum SDK. With the asset ID, fetching the cNFT is fairly straightforward.
Simply use the `getAsset` method provided by the supporting RPC provider:
```typescript theme={null}
const assetId = await getLeafAssetId(treeAddress, new BN(leafIndex));
const response = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAsset",
params: {
id: assetId,
},
}),
});
const { result } = await response.json();
console.log(JSON.stringify(result, null, 2));
```
This will return a JSON object that is comprehensive of what a traditional NFT’s
on- and off-chain metadata would look like combined. For example, you can find
the cNFT attributes at `content.metadata.attributes` or the image at
`content.files.uri`.
#### Query cNFTs
The Read API also includes ways to get multiple assets, query by owner, creator,
and more. For example, Helius supports the following methods:
* `getAsset`
* `getSignaturesForAsset`
* `searchAssets`
* `getAssetProof`
* `getAssetsByOwner`
* `getAssetsByAuthority`
* `getAssetsByCreator`
* `getAssetsByGroup`
We won’t go over most of these directly, but be sure to look through the
[Helius docs](https://docs.helius.dev/compression-and-das-api/digital-asset-standard-das-api)
to learn how to use them correctly.
#### Transfer cNFTs
Just as with a standard SPL token transfer, security is paramount. An SPL token
transfer, however, makes verifying transfer authority very easy. It’s built into
the SPL Token program and standard signing. A compressed token’s ownership is
more difficult to verify. The actual verification will happen program-side, but
your client-side code needs to provide additional information to make it
possible.
While there is a Bubblegum `createTransferInstruction` helper function, there is
more assembly required than usual. Specifically, the Bubblegum program needs to
verify that the entirety of the cNFT’s data is what the client asserts before a
transfer can occur. The entirety of the cNFT data has been hashed and stored as
a single leaf on the Merkle tree, and the Merkle tree is simply a hash of all
the tree’s leafs and branches. Because of this, you can’t simply tell the
program what account to look at and have it compare that account’s `authority`
or `owner` field to the transaction signer.
Instead, you need to provide the entirety of the cNFT data and any of the Merkle
tree’s proof information that isn’t stored in the canopy. That way, the program
can independently prove that the provided cNFT data, and therefore the cNFT
owner, is accurate. Only then can the program safely determine if the
transaction signer should, in fact, be allowed to transfer the cNFT.
In broad terms, this involves a five step process:
1. Fetch the cNFT's asset data from the indexer
2. Fetch the cNFT's proof from the indexer
3. Fetch the Merkle tree account from the Orbition Native Chain blockchain
4. Prepare the asset proof as a list of `AccountMeta` objects
5. Build and send the Bubblegum transfer instruction
The first two steps are very similar. Using your supporting RPC provider, use
the `getAsset` and `getAssetProof` methods to fetch the asset data and proof,
respectively.
```typescript theme={null}
const assetDataResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAsset",
params: {
id: assetId,
},
}),
});
const assetData = (await assetDataResponse.json()).result;
const assetProofResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAssetProof",
params: {
id: assetId,
},
}),
});
const assetProof = (await assetProofResponse.json()).result;
```
The third step is to fetch the Merkle tree account. The simplest way to do this
is using the `ConcurrentMerkleTreeAccount` type from
`@orbition-network/spl-account-compression`:
```typescript theme={null}
const treePublicKey = new PublicKey(assetData.compression.tree);
const treeAccount = await ConcurrentMerkleTreeAccount.fromAccountAddress(
connection,
treePublicKey,
);
```
Step four is the most conceptually challenging step. Using the three pieces of
information gathered, you’ll need to assemble the proof path for the cNFT’s
corresponding leaf. The proof path is represented as accounts passed to the
program instruction. The program uses each of the account addresses as proof
nodes to prove the leaf data is what you say it is.
The full proof is provided by the indexer as shown above in `assetProof`.
However, you can exclude the same number of tail-end accounts from the proof as
the depth of the canopy.
```typescript theme={null}
const canopyDepth = treeAccount.getCanopyDepth() || 0;
const proofPath: AccountMeta[] = assetProof.proof
.map((node: string) => ({
pubkey: new PublicKey(node),
isSigner: false,
isWritable: false,
}))
.slice(0, assetProof.proof.length - canopyDepth);
```
Finally, you can assemble the transfer instruction. The instruction helper
function, `createTransferInstruction`, requires the following arguments:
* `accounts` - a list of instruction accounts, as expected; they are as follows:
* `merkleTree` - the Merkle tree account
* `treeAuthority` - the Merkle tree authority
* `leafOwner` - the owner of the leaf (cNFT) in question
* `leafDelegate` - the delegate of the leaf (cNFT) in question; if no delegate
has been added then this should be the same as `leafOwner`
* `newLeafOwner` - the address of the new owner post-transfer
* `logWrapper` - the program to use to expose the data to indexers through
logs; this should be the address of the SPL Noop program unless you have
some other custom implementation
* `compressionProgram` - the compression program to use; this should be the
address of the SPL State Compression program unless you have some other
custom implementation
* `anchorRemainingAccounts` - this is where you add the proof path
* `args` - additional arguments required by the instruction; they are:
* `root` - the root Merkle tree node from the asset proof; this is provided by
the indexer as a string and must be converted to bytes first
* `dataHash` - the hash of the asset data retrieved from the indexer; this is
provided by the indexer as a string and must be converted to bytes first
* `creatorHash` - the hash of the cNFT creator as retrieved from the indexer;
this is provided by the indexer as a string and must be converted to bytes
first
* `nonce` - used to ensure that no two leafs have the same hash; this value
should be the same as `index`
* `index` - the index where the cNFT’s leaf is located on the Merkle tree
An example of this is shown below. Note that the first 3 lines of code grab
additional information nested in the objects shown previously so they are ready
to go when assembling the instruction itself.
```typescript theme={null}
const treeAuthority = treeAccount.getAuthority();
const leafOwner = new PublicKey(assetData.ownership.owner);
const leafDelegate = assetData.ownership.delegate
? new PublicKey(assetData.ownership.delegate)
: leafOwner;
const transferIx = createTransferInstruction(
{
merkleTree: treePublicKey,
treeAuthority,
leafOwner,
leafDelegate,
newLeafOwner: receiver,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
anchorRemainingAccounts: proofPath,
},
{
root: [...new PublicKey(assetProof.root.trim()).toBytes()],
dataHash: [
...new PublicKey(assetData.compression.data_hash.trim()).toBytes(),
],
creatorHash: [
...new PublicKey(assetData.compression.creator_hash.trim()).toBytes(),
],
nonce: assetData.compression.leaf_id,
index: assetData.compression.leaf_id,
},
);
```
### Conclusion
We’ve covered the primary skills needed to interact with cNFTs, but haven’t been
fully comprehensive. You can also use Bubblegum to do things like burn, verify,
delegate, and more. We won’t go through these, but these instructions are
similar to the mint and transfer process. If you need this additional
functionality, take a look at the
[Bubblegum client source code](https://github.com/metaplex-foundation/mpl-bubblegum/tree/main/clients/js-solita)
and leverage the helper functions it provides.
Keep in mind that compression is fairly new. Available tooling will evolve
rapidly but the principles you’ve learned in this lesson will likely remain the
same. These principles can also be broadened to arbitrary state compression, so
be sure to master them here so you’re ready for more fun stuff in future
lessons!
## Lab
Let’s jump in and practice creating and working with cNFTs. Together, we’ll
build as simple a script as possible that will let us mint a cNFT collection
from a Merkle tree.
#### 1. Get the starter code
First things first, clone the starter code from the `starter` branch of our
[cNFT lab repository](https://github.com/Unboxed-Software/solana-cnft-demo).
`git clone https://github.com/Unboxed-Software/solana-cnft-demo.git`
`cd solana-cnft-demo`
`npm install`
Take some time to familiarize yourself with the starter code provided. Most
important are the helper functions provided in `utils.ts` and the URIs provided
in `uri.ts`.
The `uri.ts` file provides 10k URIs that you can use for the off-chain portion
of your NFT metadata. You can, of course, create your own metadata. But this
lesson isn’t explicitly about preparing metadata so we’ve provided some for you.
The `utils.ts` file has a few helper functions to keep you from writing more
unnecessary boilerplate than you need to. They are as follows:
* `getOrCreateKeypair` will create a new keypair for you and save it to a `.env`
file, or if there’s already a private key in the `.env` file it will
initialize a keypair from that.
* `airdropSolIfNeeded` will airdrop some Testnet OBN to a specified address if
that address’s balance is below 1 OBN.
* `createNftMetadata` will create the NFT metadata for a given creator public
key and index. The metadata it’s getting is just dummy metadata using the URI
corresponding to the provided index from the `uri.ts` list of URIs.
* `getOrCreateCollectionNFT` will fetch the collection NFT from the address
specified in `.env` or if there is none it will create a new one and add the
address to `.env`.
Finally, there’s some boilerplate in `index.ts` that calls creates a new Testnet
connection, calls `getOrCreateKeypair` to initialize a “wallet,” and calls
`airdropSolIfNeeded` to fund the wallet if its balance is low.
We will be writing all of our code in the `index.ts`.
#### 2. Create the Merkle tree account
We’ll start by creating the Merkle tree account. Let’s encapsulate this in a
function that will eventually create *and* initialize the account. We’ll put it
below our `main` function in `index.ts`. Let’s call it
`createAndInitializeTree`. For this function to work, it will need the following
parameters:
* `connection` - a `Connection` to use for interacting with the network.
* `payer` - a `Keypair` that will pay for transactions.
* `maxDepthSizePair` - a `ValidDepthSizePair`. This type comes from
`@orbition-network/spl-account-compression`. It’s a simple object with properties
`maxDepth` and `maxBufferSize` that enforces a valid combination of the two
values.
* `canopyDepth` - a number for the canopy depth In the body of the function,
we’ll generate a new address for the tree, then create the instruction for
allocating a new Merkle tree account by calling `createAllocTreeIx` from
`@orbition-network/spl-account-compression`.
```typescript theme={null}
async function createAndInitializeTree(
connection: Connection,
payer: Keypair,
maxDepthSizePair: ValidDepthSizePair,
canopyDepth: number,
) {
const treeKeypair = Keypair.generate();
const allocTreeIx = await createAllocTreeIx(
connection,
treeKeypair.publicKey,
payer.publicKey,
maxDepthSizePair,
canopyDepth,
);
}
```
#### 3. Use Bubblegum to initialize the Merkle tree and create the tree config account
With the instruction for creating the tree ready to go, we can create an
instruction for invoking `create_tree` on the Bubblegum program. This will
initialize the Merkle tree account *and* create a new tree config account on the
Bubblegum program.
This instruction needs us to provide the following:
* `accounts` - an object of required accounts; this includes:
* `treeAuthority` - this should be a PDA derived with the Merkle tree address
and the Bubblegum program
* `merkleTree` - the address of the Merkle tree
* `payer` - the transaction fee payer
* `treeCreator` - the address of the tree creator; we’ll make this the same as
`payer`
* `logWrapper` - make this the `SPL_NOOP_PROGRAM_ID`
* `compressionProgram` - make this the `SPL_ACCOUNT_COMPRESSION_PROGRAM_ID`
* `args` - a list of instruction arguments; this includes:
* `maxBufferSize` - the buffer size from our function’s `maxDepthSizePair`
parameter
* `maxDepth` - the max depth from our function’s `maxDepthSizePair` parameter
* `public` - whether or no the tree should be public; we’ll set this to
`false`
Finally, we can add both instructions to a transaction and submit the
transaction. Keep in mind that the transaction needs to be signed by both the
`payer` and the `treeKeypair`.
```typescript theme={null}
async function createAndInitializeTree(
connection: Connection,
payer: Keypair,
maxDepthSizePair: ValidDepthSizePair,
canopyDepth: number,
) {
const treeKeypair = Keypair.generate();
const allocTreeIx = await createAllocTreeIx(
connection,
treeKeypair.publicKey,
payer.publicKey,
maxDepthSizePair,
canopyDepth,
);
const [treeAuthority, _bump] = PublicKey.findProgramAddressSync(
[treeKeypair.publicKey.toBuffer()],
BUBBLEGUM_PROGRAM_ID,
);
const createTreeIx = createCreateTreeInstruction(
{
treeAuthority,
merkleTree: treeKeypair.publicKey,
payer: payer.publicKey,
treeCreator: payer.publicKey,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
},
{
maxBufferSize: maxDepthSizePair.maxBufferSize,
maxDepth: maxDepthSizePair.maxDepth,
public: false,
},
);
const tx = new Transaction().add(allocTreeIx, createTreeIx);
tx.feePayer = payer.publicKey;
try {
const txSignature = await sendAndConfirmTransaction(
connection,
tx,
[treeKeypair, payer],
{
commitment: "confirmed",
skipPreflight: true,
},
);
console.log(`https://explorer.orbition.network/tx/${txSignature}?cluster=testnet`);
console.log("Tree Address:", treeKeypair.publicKey.toBase58());
return treeKeypair.publicKey;
} catch (err: any) {
console.error("\nFailed to create Merkle tree:", err);
throw err;
}
}
```
If you want to test what you have so far, feel free to call
`createAndInitializeTree` from `main` and provide small values for the max depth
and max buffer size.
```typescript theme={null}
async function main() {
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
const wallet = await getOrCreateKeypair("Wallet_1");
await airdropSolIfNeeded(wallet.publicKey);
const maxDepthSizePair: ValidDepthSizePair = {
maxDepth: 3,
maxBufferSize: 8,
};
const canopyDepth = 0;
const treeAddress = await createAndInitializeTree(
connection,
wallet,
maxDepthSizePair,
canopyDepth,
);
}
```
Keep in mind that Testnet OBN is throttled so if you test too many times you
might run out of Testnet OBN before we get to minting. To test, in your terminal
run the following:
`npm run start`
#### 4. Mint cNFTs to your tree
Believe it or not, that’s all you needed to do to set up your tree to compressed
NFTs! Now let’s turn our attention to minting.
First, let’s declare a function called `mintCompressedNftToCollection`. It will
need the following parameters:
* `connection` - a `Connection` to use for interacting with the network.
* `payer` - a `Keypair` that will pay for transactions.
* `treeAddress` - the Merkle tree’s address
* `collectionDetails` - the details of the collection as type
`CollectionDetails` from `utils.ts`
* `amount` - the number of cNFTs to mint
The body of this function will do the following:
1. Derive the tree authority just like before. Again, this is a PDA derived from
the Merkle tree address and the Bubblegum program.
2. Derive the `bubblegumSigner`. This is a PDA derived from the string
`"collection_cpi"` and the Bubblegum program and is essential for minting to
a collection.
3. Create the cNFT metadata by calling `createNftMetadata` from our `utils.ts`
file.
4. Create the mint instruction by calling `createMintToCollectionV1Instruction`
from the Bubblegum SDK.
5. Build and send a transaction with the mint instruction
6. Repeat steps 3-6 `amount` number of times
The `createMintToCollectionV1Instruction` takes two arguments: `accounts` and
`args`. The latter is simply the NFT metadata. As with all complex instructions,
the primary hurdle is knowing which accounts to provide. So let’s go through
them real quick:
* `payer` - the account that will pay for the transaction fees, rent, etc.
* `merkleTree` - the Merkle tree account
* `treeAuthority` - the tree authority; should be the same PDA you derived
previously
* `treeDelegate` - the tree delegate; this is usually the same as the tree
creator
* `leafOwner` - the desired owner of the compressed NFT being minted
* `leafDelegate` - the desired delegate of the compressed NFT being minted; this
is usually the same as the leaf owner
* `collectionAuthority` - the authority of the collection NFT
* `collectionAuthorityRecordPda` - optional collection authority record PDA;
there typically is none, in which case you should put the Bubblegum program
address
* `collectionMint` - the mint account for the collection NFT
* `collectionMetadata` - the metadata account for the collection NFT
* `editionAccount` - the master edition account of the collection NFT
* `compressionProgram` - the compression program to use; this should be the
address of the SPL State Compression program unless you have some other custom
implementation
* `logWrapper` - the program to use to expose the data to indexers through logs;
this should be the address of the SPL Noop program unless you have some other
custom implementation
* `bubblegumSigner` - a PDA used by the Bubblegrum program to handle collection
verification
* `tokenMetadataProgram` - the token metadata program that was used for the
collection NFT; this is usually always the Metaplex Token Metadata program
When you put it all together, this is what it’ll look like:
```typescript theme={null}
async function mintCompressedNftToCollection(
connection: Connection,
payer: Keypair,
treeAddress: PublicKey,
collectionDetails: CollectionDetails,
amount: number,
) {
// Derive the tree authority PDA ('TreeConfig' account for the tree account)
const [treeAuthority] = PublicKey.findProgramAddressSync(
[treeAddress.toBuffer()],
BUBBLEGUM_PROGRAM_ID,
);
// Derive the bubblegum signer, used by the Bubblegum program to handle "collection verification"
// Only used for `createMintToCollectionV1` instruction
const [bubblegumSigner] = PublicKey.findProgramAddressSync(
[Buffer.from("collection_cpi", "utf8")],
BUBBLEGUM_PROGRAM_ID,
);
for (let i = 0; i < amount; i++) {
// Compressed NFT Metadata
const compressedNFTMetadata = createNftMetadata(payer.publicKey, i);
// Create the instruction to "mint" the compressed NFT to the tree
const mintIx = createMintToCollectionV1Instruction(
{
payer: payer.publicKey, // The account that will pay for the transaction
merkleTree: treeAddress, // The address of the tree account
treeAuthority, // The authority of the tree account, should be a PDA derived from the tree account address
treeDelegate: payer.publicKey, // The delegate of the tree account, should be the same as the tree creator by default
leafOwner: payer.publicKey, // The owner of the compressed NFT being minted to the tree
leafDelegate: payer.publicKey, // The delegate of the compressed NFT being minted to the tree
collectionAuthority: payer.publicKey, // The authority of the "collection" NFT
collectionAuthorityRecordPda: BUBBLEGUM_PROGRAM_ID, // Must be the Bubblegum program id
collectionMint: collectionDetails.mint, // The mint of the "collection" NFT
collectionMetadata: collectionDetails.metadata, // The metadata of the "collection" NFT
editionAccount: collectionDetails.masterEditionAccount, // The master edition of the "collection" NFT
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
logWrapper: SPL_NOOP_PROGRAM_ID,
bubblegumSigner,
tokenMetadataProgram: TOKEN_METADATA_PROGRAM_ID,
},
{
metadataArgs: Object.assign(compressedNFTMetadata, {
collection: { key: collectionDetails.mint, verified: false },
}),
},
);
try {
// Create new transaction and add the instruction
const tx = new Transaction().add(mintIx);
// Set the fee payer for the transaction
tx.feePayer = payer.publicKey;
// Send the transaction
const txSignature = await sendAndConfirmTransaction(
connection,
tx,
[payer],
{ commitment: "confirmed", skipPreflight: true },
);
console.log(
`https://explorer.orbition.network/tx/${txSignature}?cluster=testnet`,
);
} catch (err) {
console.error("\nFailed to mint compressed NFT:", err);
throw err;
}
}
}
```
This is a great point to test with a small tree. Simply update `main` to call
`getOrCreateCollectionNFT` then `mintCompressedNftToCollection`:
```typescript theme={null}
async function main() {
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
const wallet = await getOrCreateKeypair("Wallet_1");
await airdropSolIfNeeded(wallet.publicKey);
const maxDepthSizePair: ValidDepthSizePair = {
maxDepth: 3,
maxBufferSize: 8,
};
const canopyDepth = 0;
const treeAddress = await createAndInitializeTree(
connection,
wallet,
maxDepthSizePair,
canopyDepth,
);
const collectionNft = await getOrCreateCollectionNFT(connection, wallet);
await mintCompressedNftToCollection(
connection,
wallet,
treeAddress,
collectionNft,
2 ** maxDepthSizePair.maxDepth,
);
}
```
Again, to run, in your terminal type: `npm run start`
#### 5. Read existing cNFT data
Now that we’ve written code to mint cNFTs, let’s see if we can actually fetch
their data. This is tricky because the onchain data is just the Merkle tree
account, the data from which can be used to verify existing information as
accurate but is useless in conveying what the information is.
Let’s start by declaring a function `logNftDetails` that takes as parameters
`treeAddress` and `nftsMinted`.
At this point we don’t actually have a direct identifier of any kind that points
to our cNFT. To get that, we’ll need to know the leaf index that was used when
we minted our cNFT. We can then use that to derive the asset ID used by the Read
API and subsequently use the Read API to fetch our cNFT data.
In our case, we created a non-public tree and minted 8 cNFTs, so we know that
the leaf indexes used were 0-7. With this, we can use the `getLeafAssetId`
function from `@metaplex-foundation/mpl-bubblegum` to get the asset ID.
Finally, we can use an RPC that supports the
[Read API](https://orbition.network/developers/guides/javascript/compressed-nfts) to
fetch the asset. We’ll be using
[Helius](https://docs.helius.dev/compression-and-das-api/digital-asset-standard-das-api),
but feel free to choose your own RPC provider. To use Helius, you’ll need to get
a free API Key from [the Helius website](https://dev.helius.xyz/). Then add your
`RPC_URL` to your `.env` file. For example:
```bash theme={null}
## Add this
RPC_URL=https://testnet.helius-rpc.com/?api-key=YOUR_API_KEY
```
Then simply issue a POST request to your provided RPC URL and put the `getAsset`
information in the body:
```typescript theme={null}
async function logNftDetails(treeAddress: PublicKey, nftsMinted: number) {
for (let i = 0; i < nftsMinted; i++) {
const assetId = await getLeafAssetId(treeAddress, new BN(i));
console.log("Asset ID:", assetId.toBase58());
const response = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAsset",
params: {
id: assetId,
},
}),
});
const { result } = await response.json();
console.log(JSON.stringify(result, null, 2));
}
}
```
Helius essentially observes transaction logs as they happen and stores the NFT
metadata that was hashed and stored in the Merkle tree. This enables them to
surface that data when requested.
If we add a call to this function at the end of `main` and re-run your script,
the data we get back in the console is very comprehensive. It includes all of
the data you’d expect in both the onchain and off-chain portion of a traditional
NFT. You can find the cNFT’s attributes, files, ownership and creator
information, and more.
```json theme={null}
{
"interface": "V1_NFT",
"id": "48Bw561h1fGFK4JGPXnmksHp2fpniEL7hefEc6uLZPWN",
"content": {
"$schema": "https://schema.metaplex.com/nft1.0.json",
"json_uri": "https://raw.githubusercontent.com/Unboxed-Software/rgb-png-generator/master/assets/183_89_78/183_89_78.json",
"files": [
{
"uri": "https://raw.githubusercontent.com/Unboxed-Software/rgb-png-generator/master/assets/183_89_78/183_89_78.png",
"cdn_uri": "https://cdn.helius-rpc.com/cdn-cgi/image//https://raw.githubusercontent.com/Unboxed-Software/rgb-png-generator/master/assets/183_89_78/183_89_78.png",
"mime": "image/png"
}
],
"metadata": {
"attributes": [
{
"value": "183",
"trait_type": "R"
},
{
"value": "89",
"trait_type": "G"
},
{
"value": "78",
"trait_type": "B"
}
],
"description": "Random RGB Color",
"name": "CNFT",
"symbol": "CNFT"
},
"links": {
"image": "https://raw.githubusercontent.com/Unboxed-Software/rgb-png-generator/master/assets/183_89_78/183_89_78.png"
}
},
"authorities": [
{
"address": "DeogHav5T2UV1zf5XuH4DTwwE5fZZt7Z4evytUUtDtHd",
"scopes": ["full"]
}
],
"compression": {
"eligible": false,
"compressed": true,
"data_hash": "3RsXHMBDpUPojPLZuMyKgZ1kbhW81YSY3PYmPZhbAx8K",
"creator_hash": "Di6ufEixhht76sxutC9528H7PaWuPz9hqTaCiQxoFdr",
"asset_hash": "2TwWjQPdGc5oVripPRCazGBpAyC5Ar1cia8YKUERDepE",
"tree": "7Ge8nhDv2FcmnpyfvuWPnawxquS6gSidum38oq91Q7vE",
"seq": 8,
"leaf_id": 7
},
"grouping": [
{
"group_key": "collection",
"group_value": "9p2RqBUAadMznAFiBEawMJnKR9EkFV98wKgwAz8nxLmj"
}
],
"royalty": {
"royalty_model": "creators",
"target": null,
"percent": 0,
"basis_points": 0,
"primary_sale_happened": false,
"locked": false
},
"creators": [
{
"address": "HASk3AoTPAvC1KnXSo6Qm73zpkEtEhbmjLpXLgvyKBkR",
"share": 100,
"verified": false
}
],
"ownership": {
"frozen": false,
"delegated": false,
"delegate": null,
"ownership_model": "single",
"owner": "HASk3AoTPAvC1KnXSo6Qm73zpkEtEhbmjLpXLgvyKBkR"
},
"supply": {
"print_max_supply": 0,
"print_current_supply": 0,
"edition_nonce": 0
},
"mutable": false,
"burnt": false
}
```
Remember, the Read API also includes ways to get multiple assets, query by
owner, creator, etc., and more. Be sure to look through the
[Helius docs](https://docs.helius.dev/compression-and-das-api/digital-asset-standard-das-api)
to see what’s available.
#### 6. Transfer a cNFT
The last thing we’re going to add to our script is a cNFT transfer. Just as with
a standard SPL token transfer, security is paramount. Unlike with a standard SPL
token transfer, however, to build a secure transfer with state compression of
any kind, the program performing the transfer needs the entire asset data.
The program, Bubblegum in this case, needs to be provided with the entire data
that was hashed and stored on the corresponding leaf *and* needs to be given the
“proof path” for the leaf in question. That makes cNFT transfers a bit trickier
than SPL token transfers.
Remember, the general steps are:
1. Fetch the cNFT's asset data from the indexer
2. Fetch the cNFT's proof from the indexer
3. Fetch the Merkle tree account from the Orbition Native Chain blockchain
4. Prepare the asset proof as a list of `AccountMeta` objects
5. Build and send the Bubblegum transfer instruction
Let’s start by declaring a `transferNft` function that takes the following:
* `connection` - a `Connection` object
* `assetId` - a `PublicKey` object
* `sender` - a `Keypair` object so we can sign the transaction
* `receiver` - a `PublicKey` object representing the new owner
Inside that function, let’s fetch the asset data again then also fetch the asset
proof. For good measure, let’s wrap everything in a `try catch`.
```typescript theme={null}
async function transferNft(
connection: Connection,
assetId: PublicKey,
sender: Keypair,
receiver: PublicKey,
) {
try {
const assetDataResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAsset",
params: {
id: assetId,
},
}),
});
const assetData = (await assetDataResponse.json()).result;
const assetProofResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAssetProof",
params: {
id: assetId,
},
}),
});
const assetProof = (await assetProofResponse.json()).result;
} catch (err: any) {
console.error("\nFailed to transfer nft:", err);
throw err;
}
}
```
Next, let’s fetch the Merkle tree account from the chain, get the canopy depth,
and assemble the proof path. We do this by mapping the asset proof we got from
Helius to a list of `AccountMeta` objects, then removing any proof nodes at the
end that are already cached onchain in the canopy.
```typescript theme={null}
async function transferNft(
connection: Connection,
assetId: PublicKey,
sender: Keypair,
receiver: PublicKey
) {
try {
...
const treePublicKey = new PublicKey(assetData.compression.tree)
const treeAccount = await ConcurrentMerkleTreeAccount.fromAccountAddress(
connection,
treePublicKey
)
const canopyDepth = treeAccount.getCanopyDepth() || 0
const proofPath: AccountMeta[] = assetProof.proof
.map((node: string) => ({
pubkey: new PublicKey(node),
isSigner: false,
isWritable: false,
}))
.slice(0, assetProof.proof.length - canopyDepth)
} catch (err: any) {
console.error("\nFailed to transfer nft:", err)
throw err
}
}
```
Finally, we build the instruction using `createTransferInstruction`, add it to a
transaction, then sign and send the transaction. This is what the entire
`transferNft` function looks like when finished:
```typescript theme={null}
async function transferNft(
connection: Connection,
assetId: PublicKey,
sender: Keypair,
receiver: PublicKey,
) {
try {
const assetDataResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAsset",
params: {
id: assetId,
},
}),
});
const assetData = (await assetDataResponse.json()).result;
const assetProofResponse = await fetch(process.env.RPC_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
jsonrpc: "2.0",
id: "my-id",
method: "getAssetProof",
params: {
id: assetId,
},
}),
});
const assetProof = (await assetProofResponse.json()).result;
const treePublicKey = new PublicKey(assetData.compression.tree);
const treeAccount = await ConcurrentMerkleTreeAccount.fromAccountAddress(
connection,
treePublicKey,
);
const canopyDepth = treeAccount.getCanopyDepth() || 0;
const proofPath: AccountMeta[] = assetProof.proof
.map((node: string) => ({
pubkey: new PublicKey(node),
isSigner: false,
isWritable: false,
}))
.slice(0, assetProof.proof.length - canopyDepth);
const treeAuthority = treeAccount.getAuthority();
const leafOwner = new PublicKey(assetData.ownership.owner);
const leafDelegate = assetData.ownership.delegate
? new PublicKey(assetData.ownership.delegate)
: leafOwner;
const transferIx = createTransferInstruction(
{
merkleTree: treePublicKey,
treeAuthority,
leafOwner,
leafDelegate,
newLeafOwner: receiver,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
anchorRemainingAccounts: proofPath,
},
{
root: [...new PublicKey(assetProof.root.trim()).toBytes()],
dataHash: [
...new PublicKey(assetData.compression.data_hash.trim()).toBytes(),
],
creatorHash: [
...new PublicKey(assetData.compression.creator_hash.trim()).toBytes(),
],
nonce: assetData.compression.leaf_id,
index: assetData.compression.leaf_id,
},
);
const tx = new Transaction().add(transferIx);
tx.feePayer = sender.publicKey;
const txSignature = await sendAndConfirmTransaction(
connection,
tx,
[sender],
{
commitment: "confirmed",
skipPreflight: true,
},
);
console.log(`https://explorer.orbition.network/tx/${txSignature}?cluster=testnet`);
} catch (err: any) {
console.error("\nFailed to transfer nft:", err);
throw err;
}
}
```
Lets transfer our first compressed NFT at index 0 to someone else. First we’ll
need to spin up another wallet with some funds, then grab the assetID at index 0
using `getLeafAssetId`. Then we’ll do the transfer. Finally, we’ll print out the
entire collection using our function `logNftDetails`. You’ll note that the NFT
at index zero will now belong to our new wallet in the `ownership` field.
```typescript theme={null}
async function main() {
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
const wallet = await getOrCreateKeypair("Wallet_1");
await airdropSolIfNeeded(wallet.publicKey);
const maxDepthSizePair: ValidDepthSizePair = {
maxDepth: 3,
maxBufferSize: 8,
};
const canopyDepth = 0;
const treeAddress = await createAndInitializeTree(
connection,
wallet,
maxDepthSizePair,
canopyDepth,
);
const collectionNft = await getOrCreateCollectionNFT(connection, wallet);
await mintCompressedNftToCollection(
connection,
wallet,
treeAddress,
collectionNft,
2 ** maxDepthSizePair.maxDepth,
);
const recieverWallet = await getOrCreateKeypair("Wallet_2");
const assetId = await getLeafAssetId(treeAddress, new BN(0));
await airdropSolIfNeeded(recieverWallet.publicKey);
console.log(
`Transfering ${assetId.toString()} from ${wallet.publicKey.toString()} to ${recieverWallet.publicKey.toString()}`,
);
await transferNft(connection, assetId, wallet, recieverWallet.publicKey);
await logNftDetails(treeAddress, 8);
}
```
Go ahead and run your script. The whole thing should execute without failing,
and all for close to 0.01 OBN!
Congratulations! Now you know how to mint, read, and transfer cNFTs. If you
wanted, you could update the max depth, max buffer size, and canopy depth to
larger values and as long as you have enough Testnet OBN, this script will let
you mint up to 10k cNFTs for a small fraction of what it would cost to mint 10k
traditional NFTs.
If you plan to mint a large amount of NFTs you might want
to try and batch these instructions for fewer total transactions.
If you need more time with this lab, feel free to go through it again and/or
take a look at the solution code on the `solution` branch of the
[lab repo](https://github.com/Unboxed-Software/solana-cnft-demo/tree/solution).
### Challenge
It’s your turn to take these concepts for a spin on your own! We’re not going to
be overly prescriptive at this point, but here are some ideas:
1. Create your own production cNFT collection
2. Build a UI for this lesson’s lab that will let you mint a cNFT and display it
3. See if you can replicate some of the lab script’s functionality in an onchain
program, i.e. write a program that can mint cNFTs
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=db156789-2400-4972-904f-40375582384a)!
# Generalized State Compression
Source: https://docs.orbition.network/courses/state-compression/generalized-state-compression
How state compression - the tech behind compressed NFTs - works, and how to implement it in your own Orbition Native Chain programs.
## Summary
* State Compression on Orbition Native Chain is most commonly used for compressed NFTs, but
it's possible to use it for arbitrary data
* State Compression lowers the amount of data you have to store onchain by
leveraging Merkle trees.
* Merkle trees store a single hash that represents an entire binary tree of
hashes. Each leaf on a Merkle tree is a hash of that leaf's data.
* Concurrent Merkle trees are a specialized version of Merkle trees that allow
concurrent updates.
* Because data in a state-compressed program is not stored onchain, you have to
user indexers to keep an off-chain cache of the data and then verify that data
against the onchain Merkle tree.
## Lesson
Previously, we discussed state compression in the context of compressed NFTs. At
the time of writing, compressed NFTs represent the most common use case for
state compression, but it’s possible to use state compression within any
program. In this lesson, we’ll discuss state compression in more generalized
terms so that you can apply it to any of your programs.
### A theoretical overview of state compression
In traditional programs, data is serialized (typically using borsh) and then
stored directly in an account. This allows the data to be easily read and
written through Orbition Native Chain programs. You can “trust” the data stored in the accounts
because it can’t be modified except through the mechanisms surfaced by the
program.
State compression effectively asserts that the most important piece of this
equation is how “trustworthy” the data is. If all we care about is the ability
to trust that data is what it claims to be, then we can actually get away with
***not*** storing the data in an account onchain. Instead, we can store hashes
of the data where the hashes can be used to prove or verify the data. The data
hash takes up significantly less storage space than the data itself. We can then
store the actual data somewhere much cheaper and worry about verifying it
against the onchain hash when the data is accessed.
The specific data structure used by the Orbition Native Chain State Compression program is a
special binary tree structure known as a **concurrent Merkle tree**. This tree
structure hashes pieces of data together in a deterministic way to compute a
single, final hash that gets stored onchain. This final hash is significantly
smaller in size than all the original data combined, hence the “compression.”
The steps to this process are:
1. Take any piece of data
2. Create a hash of this data
3. Store this hash as a “leaf” at the bottom of the tree
4. Each leaf pair is then hashed together, creating a “branch”
5. Each branch is then hashed together
6. Continually climb the tree and hash adjacent branches together
7. Once at the top of the tree, a final ”root hash” is produced
8. Store the root hash onchain as verifiable proof of the data within each leaf
9. Anyone wanting to verify that the data they have matches the “source of
truth” can go through the same process and compare the final hash without
having to store all the data onchain
This involves a few rather serious development tradeoffs:
1. Since the data is no longer stored in an account onchain, it is more
difficult to access.
2. Once the data has been accessed, developers must decide how often their
applications will verify the data against the onchain hash.
3. Any changes to the data will require sending the entirety of the previously
hashed data *and* the new data into an instruction. Developer may also have
to provide additional data relevant to the proofs required to verify the
original data against the hash.
Each of these will be a consideration when determining **if**, **when**, and
**how** to implement state compression for your program.
#### Concurrent Merkle trees
A **Merkle tree** is a binary tree structure represented by a single hash. Every
leaf node in the structure is a hash of its inner data while every branch is a
hash of its child leaf hashes. In turn, branches are also hashed together until,
eventually, one final root hash remains.
Since the Merkle tree is represented as a single hash, any modification to leaf
data changes the root hash. This causes an issue when multiple transactions in
the same slot are attempting to modify leaf data. Since these transactions must
execute in series, all but the first will fail since the root hash and proof
passed in will have been invalidated by the first transaction to be executed. In
other words, a standard Merkle tree can only modify a single leaf per slot. In a
hypothetical state-compressed program that relies on a single Merkle tree for
its state, this severely limits throughput.
This can be solved with a **concurrent Merkle tree**. A concurrent Merkle
tree is a Merkle tree that stores a secure changelog of the most recent changes
along with their root hash and the proof to derive it. When multiple
transactions in the same slot try to modify leaf data, the changelog can be used
as a source of truth to allow for concurrent changes to be made to the tree.
In other words, while an account storing a Merkle tree would have only the root
hash, a concurrent Merkle tree will also contain additional data that allows
subsequent writes to successfully occur. This includes:
1. The root hash - The same root hash that a standard Merkle tree has.
2. A changelog buffer - This buffer contains proof data pertinent to recent root
hash changes so that subsequent writes in the same slot can still be
successful.
3. A canopy - When performing an update action on any given leaf, you need the
entire proof path from that leaf to the root hash. The canopy stores
intermediate proof nodes along that path so they don’t all have to be passed
into the program from the client.
As a program architect, you control three values directly related to these three
items. Your choice determines the size of the tree, the cost to create the tree,
and the number of concurrent changes that can be made to the tree:
1. Max depth
2. Max buffer size
3. Canopy depth
The **max depth** is the maximum number of hops to get from any leaf to the root
of the tree. Since Merkle trees are binary trees, every leaf is connected only
to one other leaf. Max depth can then logically be used to calculate the number
of nodes for the tree with `2 ^ maxDepth`.
The **max buffer size** is effectively the maximum number of concurrent changes
that you can make to a tree within a single slot with the root hash still being
valid. When multiple transactions are submitted in the same slot, each of which
is competing to update leafs on a standard Merkle tree, only the first to run
will be valid. This is because that “write” operation will modify the hash
stored in the account. Subsequent transactions in the same slot will be trying
to validate their data against a now-outdated hash. A concurrent Merkle tree has
a buffer so that the buffer can keep a running log of these modifications. This
allows the State Compression Program to validate multiple data writes in the
same slot because it can look up what the previous hashes were in the buffer and
compare against the appropriate hash.
The **canopy depth** is the number of proof nodes that are stored onchain for
any given proof path. Verifying any leaf requires the complete proof path for
the tree. The complete proof path is made up of one proof node for every “layer”
of the tree, i.e. a max depth of 14 means there are 14 proof nodes. Every proof
node passed into the program adds 32 bytes to a transaction, so large trees
would quickly exceed the maximum transaction size limit. Caching proof nodes
onchain in the canopy helps improve program composability.
Each of these three values, max depth, max buffer size, and canopy depth, comes
with a tradeoff. Increasing the value of any of these values increases the size
of the account used to store the tree, thus increasing the cost of creating the
tree.
Choosing the max depth is fairly straightforward as it directly relates to the
number of leafs and therefore the amount of data you can store. If you need 1
million cNFTs on a single tree where each cNFT is a leaf of the tree, find the
max depth that makes the following expression true: `2^maxDepth > 1 million`.
The answer is 20.
Choosing a max buffer size is effectively a question of throughput: how many
concurrent writes do you need? The larger the buffer, the higher the throughput.
Lastly, the canopy depth will determine your program’s composability. State
compression pioneers have made it clear that omitting a canopy is a bad idea.
Program A can’t call your state-compressed program B if doing so maxes out the
transaction size limits. Remember, program A also has required accounts and data
in addition to required proof paths, each of which take up transaction space.
#### Data access on a state-compressed program
A state-compressed account doesn’t store the data itself. Rather, it stores the
concurrent Merkle tree structure discussed above. The raw data itself lives only
in the blockchain’s cheaper **ledger state.** This makes data access somewhat
more difficult, but not impossible.
The Orbition Native Chain ledger is a list of entries containing signed transactions. In
theory, this can be traced back to the genesis block. This effectively means any
data that has ever been put into a transaction exists in the ledger.
Since the state compression hashing process occurs onchain, all the data exists
in the ledger state and could theoretically be retrieved from the original
transaction by replaying the entire chain state from the beginning. However,
it’s much more straightforward (though still complicated) to have
an **indexer** track and index this data as the transactions occur. This ensures
there is an off-chain “cache” of the data that anyone can access and
subsequently verify against the onchain root hash.
This process is complex, but it will make sense after some practice.
### State compression tooling
The theory described above is essential to properly understanding state
compression. But you don’t have to implement any of it from scratch. Brilliant
engineers have laid most of the groundwork for you in the form of the SPL State
Compression Program and the Noop Program.
#### SPL State Compression and Noop Programs
The SPL State Compression Program exists to make the process of creating and
updating concurrent Merkle trees repeatable and composable throughout the Orbition Native Chain
ecosystem. It provides instructions for initializing Merkle trees, managing tree
leafs (i.e. add, update, remove data), and verifying leaf data.
The State Compression Program also leverages a separate “no op” program whose
primary purpose is to make leaf data easier to index by logging it to the ledger
state. When you want to store compressed data, you pass it to the State
Compression program where it gets hashed and emitted as an “event” to the Noop
program. The hash gets stored in the corresponding concurrent Merkle tree, but
the raw data remains accessible through the Noop program’s transaction logs.
#### Index data for easy lookup
Under normal conditions, you would typically access onchain data by fetching the
appropriate account. When using state compression, however, it’s not so
straightforward.
As mentioned above, the data now exists in the ledger state rather than in an
account. The easiest place to find the full data is in the logs of the Noop
instruction. Unfortunately, while this data will in a sense exist in the ledger
state forever, it will likely be inaccessible through validators after a certain
period of time.
To save space and be more performant, validators don’t retain every transaction
back to the genesis block. The specific amount of time you’ll be able to access
the Noop instruction logs related to your data will vary based on the validator.
Eventually, you’ll lose access to it if you’re relying directly on instruction
logs.
Technically, you *can* replay the transaction state back to the genesis block
but the average team isn’t going to do that, and it certainly won’t be
performant. The
[Digital Asset Standard (DAS)](https://docs.helius.dev/compression-and-das-api/digital-asset-standard-das-api)
has been adopted by many RPC providers to enable efficient queries of compressed
NFTs and other assets. However, at the time of writing, it doesn’t support
arbitrary state compression. Instead, you have two primary options:
1. Use an indexing provider that will build a custom indexing solution for your
program that observes the events sent to the Noop program and stores the
relevant data off-chain.
2. Create your own pseudo-indexing solution that stores transaction data
off-chain.
For many dApps, option 2 makes plenty of sense. Larger-scale applications may
need to rely on infrastructure providers to handle their indexing.
### State compression development process
#### Create Rust types
As with a typical Anchor program, one of the first things you should do is
define your program’s Rust types. However, Rust types in a traditional Anchor
program often represent accounts. In a state-compressed program, your account
state will only store the Merkle tree. The more “usable” data schema will just
be serialized and logged to the Noop program.
This type should include all the data stored in the leaf node and any contextual
information needed to make sense of the data. For example, if you were to create
a simple messaging program, your `Message` struct might look as follows:
```rust theme={null}
#[derive(AnchorSerialize)]
pub struct MessageLog {
leaf_node: [u8; 32], // The leaf node hash
from: Pubkey, // Pubkey of the message sender
to: Pubkey, // Pubkey of the message recipient
message: String, // The message to send
}
impl MessageLog {
// Constructs a new message log from given leaf node and message
pub fn new(leaf_node: [u8; 32], from: Pubkey, to: Pubkey, message: String) -> Self {
Self { leaf_node, from, to, message }
}
}
```
To be abundantly clear, **this is not an account that you will be able to read
from**. Your program will be creating an instance of this type from instruction
inputs, not constructing an instance of this type from account data that it
reads. We’ll discuss how to read data in a later section.
#### Initialize a new tree
Clients will create and initialize the Merkle tree account in two separate
instructions. The first is simply allocating the account by calling System
Program. The second will be an instruction that you create on a custom program
that initializes the new account. This initialization is effectively just
recording what the max depth and buffer size for the Merkle tree should be.
All this instruction needs to do is build a CPI to invoke the
`init_empty_merkle_tree` instruction on the State Compression Program. Since
this requires the max depth and max buffer size, these will need to be passed in
as arguments to the instruction.
Remember, the max depth refers to the maximum number of hops to get from any
leaf to the root of the tree. Max buffer size refers to the amount of space
reserved for storing a changelog of tree updates. This changelog is used to
ensure that your tree can support concurrent updates within the same block.
For example, if we were initializing a tree for storing messages between users,
the instruction might look like this:
```rust theme={null}
pub fn create_messages_tree(
ctx: Context,
max_depth: u32, // Max depth of the Merkle tree
max_buffer_size: u32 // Max buffer size of the Merkle tree
) -> Result<()> {
// Get the address for the Merkle tree account
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[
&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
],
];
// Create cpi context for init_empty_merkle_tree instruction.
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Initialize {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be initialized
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds // The seeds for pda signing
);
// CPI to initialize an empty Merkle tree with given max depth and buffer size
init_empty_merkle_tree(cpi_ctx, max_depth, max_buffer_size)?;
Ok(())
}
```
#### Add hashes to the tree
With an initialized Merkle tree, it’s possible to start adding data hashes. This
involves passing the uncompressed data to an instruction on your program that
will hash the data, log it to the Noop program, and use the State Compression
Program’s `append` instruction to add the hash to the tree. The following
discuss what your instruction needs to do in depth:
1. Use the `hashv` function from the `keccak` crate to hash the data. In most
cases, you’ll want to also hash the owner or authority of the data as well to
ensure that it can only be modified by the proper authority.
2. Create a log object representing the data you wish to log to the Noop
Program, then call `wrap_application_data_v1` to issue a CPI to the Noop
program with this object. This ensures that the uncompressed data is readily
available to any client looking for it. For broad use cases like cNFTs, that
would be indexers. You might also create your own observing client to
simulate what indexers are doing but specific to your application.
3. Build and issue a CPI to the State Compression Program’s `append`
instruction. This takes the hash computed in step 1 and adds it to the next
available leaf on your Merkle tree. Just as before, this requires the Merkle
tree address and the tree authority bump as signature seeds.
When all this is put together using the messaging example, it looks something
like this:
```rust theme={null}
// Instruction for appending a message to a tree.
pub fn append_message(ctx: Context, message: String) -> Result<()> {
// Hash the message + whatever key should have update authority
let leaf_node = keccak::hashv(&[message.as_bytes(), ctx.accounts.sender.key().as_ref()]).to_bytes();
// Create a new "message log" using the leaf node hash, sender, receipient, and message
let message_log = MessageLog::new(leaf_node.clone(), ctx.accounts.sender.key().clone(), ctx.accounts.receipient.key().clone(), message);
// Log the "message log" data using noop program
wrap_application_data_v1(message_log.try_to_vec()?, &ctx.accounts.log_wrapper)?;
// Get the address for the Merkle tree account
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[
&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
],
];
// Create a new cpi context and append the leaf node to the Merkle tree.
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Modify {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds // The seeds for pda signing
);
// CPI to append the leaf node to the Merkle tree
append(cpi_ctx, leaf_node)?;
Ok(())
}
```
#### Update hashes
To update data, you need to create a new hash to replace the hash at the
relevant leaf on the Merkle tree. To do this, your program needs access to four
things:
1. The index of the leaf to update
2. The root hash of the Merkle tree
3. The original data you wish to modify
4. The updated data
Given access to this data, a program instruction can follow very similar steps
as those used to append the initial data to the tree:
1. **Verify update authority** - The first step is new. In most cases, you want
to verify update authority. This typically involves proving that the signer
of the `update` transaction is the true owner or authority of the leaf at the
given index. Since the data is compressed as a hash on the leaf, we can’t
simply compare the `authority` public key to a stored value. Instead, we need
to compute the previous hash using the old data and the `authority` listed in
the account validation struct. We then build and issue a CPI to the State
Compression Program’s `verify_leaf` instruction using our computed hash.
2. **Hash the new data** - This step is the same as the first step from
appending initial data. Use the `hashv` function from the `keccak` crate to
hash the new data and the update authority, each as their corresponding byte
representation.
3. **Log the new data** - This step is the same as the second step from
appending initial data. Create an instance of the log struct and call
`wrap_application_data_v1` to issue a CPI to the Noop program.
4. **Replace the existing leaf hash** - This step is slightly different than the
last step of appending initial data. Build and issue a CPI to the State
Compression Program’s `replace_leaf` instruction. This uses the old hash, the
new hash, and the leaf index to replace the data of the leaf at the given
index with the new hash. Just as before, this requires the Merkle tree
address and the tree authority bump as signature seeds.
Combined into a single instruction, this process looks as follows:
```rust theme={null}
pub fn update_message(
ctx: Context,
index: u32,
root: [u8; 32],
old_message: String,
new_message: String
) -> Result<()> {
let old_leaf = keccak
::hashv(&[old_message.as_bytes(), ctx.accounts.sender.key().as_ref()])
.to_bytes();
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[
&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
],
];
// Verify Leaf
{
if old_message == new_message {
msg!("Messages are the same!");
return Ok(());
}
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
VerifyLeaf {
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
},
signer_seeds // The seeds for pda signing
);
// Verify or Fails
verify_leaf(cpi_ctx, root, old_leaf, index)?;
}
let new_leaf = keccak
::hashv(&[new_message.as_bytes(), ctx.accounts.sender.key().as_ref()])
.to_bytes();
// Log out for indexers
let message_log = MessageLog::new(new_leaf.clone(), ctx.accounts.sender.key().clone(), ctx.accounts.recipient.key().clone(), new_message);
// Log the "message log" data using noop program
wrap_application_data_v1(message_log.try_to_vec()?, &ctx.accounts.log_wrapper)?;
// replace leaf
{
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Modify {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds // The seeds for pda signing
);
// CPI to append the leaf node to the Merkle tree
replace_leaf(cpi_ctx, root, old_leaf, new_leaf, index)?;
}
Ok(())
}
```
#### Delete hashes
At the time of writing, the State Compression Program doesn’t provide an
explicit `delete` instruction. Instead, you’ll want to update leaf data with
data that indicates the data as “deleted.” The specific data will depend on your
use case and security concerns. Some may opt to set all data to 0, whereas
others might store a static string that all “deleted” items will have in common.
#### Access data from a client
The discussion so far has covered 3 of the 4 standard CRUD procedures: Create,
Update, and Delete. What’s left is one of the more difficult concepts in state
compression: reading data.
Accessing data from a client is tricky primarily because the data isn’t stored
in a format that is easy to access. The data hashes stored in the Merkle tree
account can’t be used to reconstruct the initial data, and the data logged to
the Noop program isn’t available indefinitely.
Your best bet is one of two options:
1. Work with an indexing provider to create a custom indexing solution for your
program, then write client-side code based on how the indexer gives you
access to the data.
2. Create your own pseudo-indexer as a lighter-weight solution.
If your project is truly decentralized such that many participants will interact
with your program through means other than your own frontend, then option 2
might not be sufficient. However, depending on the scale of the project or
whether or not you’ll have control over most program access, it can be a viable
approach.
There is no “right” way to do this. Two potential approaches are:
1. Store the raw data in a database at the same time as sending it to the
program, along with the leaf that the data is hashed and stored to.
2. Create a server that observes your program’s transactions, looks up the
associated Noop logs, decodes the logs, and stores them.
We’ll do a little bit of both when writing tests in this lesson’s lab (though we
won’t persist data in a db - it will only live in memory for the duration of the
tests).
The setup for this is somewhat tedious. Given a particular transaction, you can
fetch the transaction from the RPC provider, get the inner instructions
associated with the Noop program, use the `deserializeApplicationDataEvent`
function from the `@orbition-network/spl-account-compression` JS package to get the logs,
then deserialize them using Borsh. Below is an example based on the messaging
program used above.
```typescript theme={null}
export async function getMessageLog(
connection: Connection,
txSignature: string,
) {
// Confirm the transaction, otherwise the getTransaction sometimes returns null
const latestBlockHash = await connection.getLatestBlockhash();
await connection.confirmTransaction({
blockhash: latestBlockHash.blockhash,
lastValidBlockHeight: latestBlockHash.lastValidBlockHeight,
signature: txSignature,
});
// Get the transaction info using the tx signature
const txInfo = await connection.getTransaction(txSignature, {
maxSupportedTransactionVersion: 0,
});
// Get the inner instructions related to the program instruction at index 0
// We only send one instruction in test transaction, so we can assume the first
const innerIx = txInfo!.meta?.innerInstructions?.[0]?.instructions;
// Get the inner instructions that match the SPL_NOOP_PROGRAM_ID
const noopInnerIx = innerIx.filter(
instruction =>
txInfo?.transaction.message.staticAccountKeys[
instruction.programIdIndex
].toBase58() === SPL_NOOP_PROGRAM_ID.toBase58(),
);
let messageLog: MessageLog;
for (let i = noopInnerIx.length - 1; i >= 0; i--) {
try {
// Try to decode and deserialize the instruction data
const applicationDataEvent = deserializeApplicationDataEvent(
Buffer.from(bs58.decode(noopInnerIx[i]?.data!)),
);
// Get the application data
const applicationData = applicationDataEvent.fields[0].applicationData;
// Deserialize the application data into MessageLog instance
messageLog = deserialize(
MessageLogBorshSchema,
MessageLog,
Buffer.from(applicationData),
);
if (messageLog !== undefined) {
break;
}
} catch (__) {}
}
return messageLog;
}
```
### Conclusion
Generalized state compression can be difficult but is absolutely possible to
implement with the available tools. Additionally, the tools and programs will
only get better over time. If you come up with solutions that improve your
development experience, please share with the community!
## Lab
Let’s practice generalized state compression by creating a new Anchor program.
This program will use custom state compression to power a simple note-taking
app.
#### 1. Project setup
Start by initializing an Anchor program:
```bash theme={null}
anchor init compressed-notes
```
We’ll be using the `spl-account-compression` crate with the `cpi` feature
enabled. Let’s add it as a dependency in `programs/compressed-notes/Cargo.toml`.
```toml theme={null}
[dependencies]
anchor-lang = "0.28.0"
spl-account-compression = { version="0.2.0", features = ["cpi"] }
solana-program = "1.16.0"
```
We’ll be testing locally but we need both the Compression program and the Noop
program from Mainnet. We’ll need to add these to the `Anchor.toml` in the root
directory so they get cloned to our local cluster.
```toml theme={null}
[test.validator]
url = "https://api.mainnet-beta.orbition.network"
[[test.validator.clone]]
address = "noopb9bkMVfRPU8AsbpTUg8AQkHtKwMYZiFUjNRtMmV"
[[test.validator.clone]]
address = "cmtDvXumGCrqC1Age74AVPhSRVXJMd8PJS91L8KbNCK"
```
Lastly, let’s prepare the `lib.rs` file for the rest of the Demo. Remove the
`initialize` instruction and the `Initialize` accounts struct, then add the
imports shown in the code snippet below (be sure to put in ***your*** program
id):
```rust theme={null}
use anchor_lang::{
prelude::*,
solana_program::keccak
};
use spl_account_compression::{
Noop,
program::SplAccountCompression,
cpi::{
accounts::{Initialize, Modify, VerifyLeaf},
init_empty_merkle_tree, verify_leaf, replace_leaf, append,
},
wrap_application_data_v1,
};
declare_id!("YOUR_KEY_GOES_HERE");
// STRUCTS GO HERE
#[program]
pub mod compressed_notes {
use super::*;
// FUNCTIONS GO HERE
}
```
For the rest of this Demo, we’ll be making updates to the program code directly
in the `lib.rs` file. This simplifies the explanations a bit. You’re welcome to
modify the structure as you will.
Feel free to build before continuing. This ensures your environment is working
properly and shortens future build times.
#### 2. Define `Note` schema
Next, we’re going to define what a note looks like within our program. Notes
should have the following properties:
* `leaf_node` - this should be a 32-byte array representing the hash stored on
the leaf node
* `owner` - the public key of the note owner
* `note` - the string representation of the note
```rust theme={null}
#[derive(AnchorSerialize)]
pub struct NoteLog {
leaf_node: [u8; 32], // The leaf node hash
owner: Pubkey, // Pubkey of the note owner
note: String, // The note message
}
impl NoteLog {
// Constructs a new note from given leaf node and message
pub fn new(leaf_node: [u8; 32], owner: Pubkey, note: String) -> Self {
Self { leaf_node, owner, note }
}
}
```
In a traditional Anchor program, this would be an account struct, but since
we’re using state compression, our accounts won’t be mirroring our native
structures. Since we don’t need all the functionality of an account, we can just
use the `AnchorSerialize` derive macro rather than the `account` macro.
#### 3. Define input accounts and constraints
As luck would have it, every one of our instructions will be using the same
accounts. We’ll create a single `NoteAccounts` struct for our account
validation. It’ll need the following accounts:
* `owner` - this is the creator and owner of the note; should be a signer on the
transaction
* `tree_authority` - the authority for the Merkle tree; used for signing
compression-related CPIs
* `merkle_tree` - the address of the Merkle tree used to store the note hashes;
will be unchecked since it is validated by the State Compression Program
* `log_wrapper` - the address of the Noop Program
* `compression_program` - the address of the State Compression Program
```rust theme={null}
#[derive(Accounts)]
pub struct NoteAccounts<'info> {
// The payer for the transaction
#[account(mut)]
pub owner: Signer<'info>,
// The pda authority for the Merkle tree, only used for signing
#[account(
seeds = [merkle_tree.key().as_ref()],
bump,
)]
pub tree_authority: SystemAccount<'info>,
// The Merkle tree account
/// CHECK: This account is validated by the spl account compression program
#[account(mut)]
pub merkle_tree: UncheckedAccount<'info>,
// The noop program to log data
pub log_wrapper: Program<'info, Noop>,
// The spl account compression program
pub compression_program: Program<'info, SplAccountCompression>,
}
```
#### 4. Create `create_note_tree` instruction
Next, let’s create our `create_note_tree` instruction. Remember, clients will
have already allocated the Merkle tree account but will use this instruction to
initialize it.
All this instruction needs to do is build a CPI to invoke the
`init_empty_merkle_tree` instruction on the State Compression Program. To do
this, it needs the accounts listed in the `NoteAccounts` account validation
struct. It also needs two additional arguments:
1. `max_depth` - the max depth of the Merkle tree
2. `max_buffer_size` - the max buffer size of the Merkle tree
These values are required for initializing the data on the Merkle tree account.
Remember, the max depth refers to the maximum number of hops to get from any
leaf to the root of the tree. Max buffer size refers to the amount of space
reserved for storing a changelog of tree updates. This changelog is used to
ensure that your tree can support concurrent updates within the same block.
```rust theme={null}
#[program]
pub mod compressed_notes {
use super::*;
// Instruction for creating a new note tree.
pub fn create_note_tree(
ctx: Context,
max_depth: u32, // Max depth of the Merkle tree
max_buffer_size: u32, // Max buffer size of the Merkle tree
) -> Result<()> {
// Get the address for the Merkle tree account
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
]];
// Create cpi context for init_empty_merkle_tree instruction.
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Initialize {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be initialized
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds, // The seeds for pda signing
);
// CPI to initialize an empty Merkle tree with given max depth and buffer size
init_empty_merkle_tree(cpi_ctx, max_depth, max_buffer_size)?;
Ok(())
}
//...
}
```
Ensure that your signer seeds on the CPI include both the Merkle tree address
and the tree authority bump.
#### 5. Create `append_note` instruction
Now, let’s create our `append_note` instruction. This instruction needs to take
the raw note as a String and compress it into a hash that we’ll store on the
Merkle tree. We’ll also log the note to the Noop program so the entirety of the
data exists within the chain’s state.
The steps here are as follows:
1. Use the `hashv` function from the `keccak` crate to hash the note and owner,
each as their corresponding byte representation. It’s ***crucial*** that you
hash the owner as well as the note. This is how we’ll verify note ownership
before updates in the update instruction.
2. Create an instance of the `NoteLog` struct using the hash from step 1, the
owner’s public key, and the raw note as a String. Then call
`wrap_application_data_v1` to issue a CPI to the Noop program, passing the
instance of `NoteLog`. This ensures the entirety of the note (not just the
hash) is readily available to any client looking for it. For broad use cases
like cNFTs, that would be indexers. You might create your observing client to
simulate what indexers are doing but for your own application.
3. Build and issue a CPI to the State Compression Program’s `append`
instruction. This takes the hash computed in step 1 and adds it to the next
available leaf on your Merkle tree. Just as before, this requires the Merkle
tree address and the tree authority bump as signature seeds.
```rust theme={null}
#[program]
pub mod compressed_notes {
use super::*;
//...
// Instruction for appending a note to a tree.
pub fn append_note(ctx: Context, note: String) -> Result<()> {
// Hash the "note message" which will be stored as leaf node in the Merkle tree
let leaf_node =
keccak::hashv(&[note.as_bytes(), ctx.accounts.owner.key().as_ref()]).to_bytes();
// Create a new "note log" using the leaf node hash and note.
let note_log = NoteLog::new(leaf_node.clone(), ctx.accounts.owner.key().clone(), note);
// Log the "note log" data using noop program
wrap_application_data_v1(note_log.try_to_vec()?, &ctx.accounts.log_wrapper)?;
// Get the address for the Merkle tree account
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
]];
// Create a new cpi context and append the leaf node to the Merkle tree.
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Modify {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds, // The seeds for pda signing
);
// CPI to append the leaf node to the Merkle tree
append(cpi_ctx, leaf_node)?;
Ok(())
}
//...
}
```
#### 6. Create `update_note` instruction
The last instruction we’ll make is the `update_note` instruction. This should
replace an existing leaf with a new hash representing the new updated note data.
For this to work, we’ll need the following parameters:
1. `index` - the index of the leaf we are going to update
2. `root` - the root hash of the Merkle tree
3. `old_note` - the string representation of the old note we’re updating
4. `new_note` - the string representation of the new note we want to update to
Remember, the steps here are similar to `append_note`, but with some minor
additions and modifications:
1. The first step is new. We need to first prove that the `owner` calling this
function is the true owner of the leaf at the given index. Since the data is
compressed as a hash on the leaf, we can’t simply compare the `owner` public
key to a stored value. Instead, we need to compute the previous hash using
the old note data and the `owner` listed in the account validation struct. We
then build and issue a CPI to the State Compression Program’s `verify_leaf`
instruction using our computed hash.
2. This step is the same as the first step from creating the `append_note`
instruction. Use the `hashv` function from the `keccak` crate to hash the new
note and its owner, each as their corresponding byte representation.
3. This step is the same as the second step from creating the `append_note`
instruction. Create an instance of the `NoteLog` struct using the hash from
step 2, the owner’s public key, and the new note as a string. Then call
`wrap_application_data_v1` to issue a CPI to the Noop program, passing the
instance of `NoteLog`
4. This step is slightly different than the last step from creating the
`append_note` instruction. Build and issue a CPI to the State Compression
Program’s `replace_leaf` instruction. This uses the old hash, the new hash,
and the leaf index to replace the data of the leaf at the given index with
the new hash. Just as before, this requires the Merkle tree address and the
tree authority bump as signature seeds.
```rust theme={null}
#[program]
pub mod compressed_notes {
use super::*;
//...
pub fn update_note(
ctx: Context,
index: u32,
root: [u8; 32],
old_note: String,
new_note: String,
) -> Result<()> {
let old_leaf =
keccak::hashv(&[old_note.as_bytes(), ctx.accounts.owner.key().as_ref()]).to_bytes();
let merkle_tree = ctx.accounts.merkle_tree.key();
// Define the seeds for pda signing
let signer_seeds: &[&[&[u8]]] = &[&[
merkle_tree.as_ref(), // The address of the Merkle tree account as a seed
&[*ctx.bumps.get("tree_authority").unwrap()], // The bump seed for the pda
]];
// Verify Leaf
{
if old_note == new_note {
msg!("Notes are the same!");
return Ok(());
}
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
VerifyLeaf {
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
},
signer_seeds, // The seeds for pda signing
);
// Verify or Fails
verify_leaf(cpi_ctx, root, old_leaf, index)?;
}
let new_leaf =
keccak::hashv(&[new_note.as_bytes(), ctx.accounts.owner.key().as_ref()]).to_bytes();
// Log out for indexers
let note_log = NoteLog::new(new_leaf.clone(), ctx.accounts.owner.key().clone(), new_note);
// Log the "note log" data using noop program
wrap_application_data_v1(note_log.try_to_vec()?, &ctx.accounts.log_wrapper)?;
// replace leaf
{
let cpi_ctx = CpiContext::new_with_signer(
ctx.accounts.compression_program.to_account_info(), // The spl account compression program
Modify {
authority: ctx.accounts.tree_authority.to_account_info(), // The authority for the Merkle tree, using a PDA
merkle_tree: ctx.accounts.merkle_tree.to_account_info(), // The Merkle tree account to be modified
noop: ctx.accounts.log_wrapper.to_account_info(), // The noop program to log data
},
signer_seeds, // The seeds for pda signing
);
// CPI to append the leaf node to the Merkle tree
replace_leaf(cpi_ctx, root, old_leaf, new_leaf, index)?;
}
Ok(())
}
}
```
#### 7. Client test setup
We’re going to write a few tests to ensure that our program works as expected.
First, let’s do some setup.
We’ll be using the `@orbition-network/spl-account-compression` package. Go ahead and
install it:
```bash theme={null}
yarn add @orbition-network/spl-account-compression
```
Next, we’re going to give you the contents of a utility file we’ve created to
make testing easier. Create a `utils.ts` file in the `tests` directory, add in
the below, then we’ll explain it.
```typescript theme={null}
import {
SPL_NOOP_PROGRAM_ID,
deserializeApplicationDataEvent,
} from "@orbition-network/spl-account-compression";
import { Connection, PublicKey } from "@orbition-network/web3.js";
import { bs58 } from "@coral-xyz/anchor/dist/cjs/utils/bytes";
import { deserialize } from "borsh";
import { keccak256 } from "js-sha3";
class NoteLog {
leafNode: Uint8Array;
owner: PublicKey;
note: string;
constructor(properties: {
leafNode: Uint8Array;
owner: Uint8Array;
note: string;
}) {
this.leafNode = properties.leafNode;
this.owner = new PublicKey(properties.owner);
this.note = properties.note;
}
}
// A map that describes the Note structure for Borsh deserialization
const NoteLogBorshSchema = new Map([
[
NoteLog,
{
kind: "struct",
fields: [
["leafNode", [32]], // Array of 32 `u8`
["owner", [32]], // Pubkey
["note", "string"],
],
},
],
]);
export function getHash(note: string, owner: PublicKey) {
const noteBuffer = Buffer.from(note);
const publicKeyBuffer = Buffer.from(owner.toBytes());
const concatenatedBuffer = Buffer.concat([noteBuffer, publicKeyBuffer]);
const concatenatedUint8Array = new Uint8Array(
concatenatedBuffer.buffer,
concatenatedBuffer.byteOffset,
concatenatedBuffer.byteLength,
);
return keccak256(concatenatedUint8Array);
}
export async function getNoteLog(connection: Connection, txSignature: string) {
// Confirm the transaction, otherwise the getTransaction sometimes returns null
const latestBlockHash = await connection.getLatestBlockhash();
await connection.confirmTransaction({
blockhash: latestBlockHash.blockhash,
lastValidBlockHeight: latestBlockHash.lastValidBlockHeight,
signature: txSignature,
});
// Get the transaction info using the tx signature
const txInfo = await connection.getTransaction(txSignature, {
maxSupportedTransactionVersion: 0,
});
// Get the inner instructions related to the program instruction at index 0
// We only send one instruction in test transaction, so we can assume the first
const innerIx = txInfo!.meta?.innerInstructions?.[0]?.instructions;
// Get the inner instructions that match the SPL_NOOP_PROGRAM_ID
const noopInnerIx = innerIx.filter(
instruction =>
txInfo?.transaction.message.staticAccountKeys[
instruction.programIdIndex
].toBase58() === SPL_NOOP_PROGRAM_ID.toBase58(),
);
let noteLog: NoteLog;
for (let i = noopInnerIx.length - 1; i >= 0; i--) {
try {
// Try to decode and deserialize the instruction data
const applicationDataEvent = deserializeApplicationDataEvent(
Buffer.from(bs58.decode(noopInnerIx[i]?.data!)),
);
// Get the application data
const applicationData = applicationDataEvent.fields[0].applicationData;
// Deserialize the application data into NoteLog instance
noteLog = deserialize(
NoteLogBorshSchema,
NoteLog,
Buffer.from(applicationData),
);
if (noteLog !== undefined) {
break;
}
} catch (__) {}
}
return noteLog;
}
```
There are 3 main things in the above file:
1. `NoteLog` - a class representing the note log we’ll find in the Noop program
logs. We’ve also added the borsh schema as `NoteLogBorshSchema` for
deserialization.
2. `getHash` - a function that creates a hash of the note and note owner so we
can compare it to what we find on the Merkle tree
3. `getNoteLog` - a function that looks through the provided transaction’s logs,
finds the Noop program logs, then deserializes and returns the corresponding
Note log.
#### 8. Write client tests
Now that we’ve got our packages installed and utility file ready, let’s dig into
the tests themselves. We’re going to create four of them:
1. Create Note Tree - this will create the Merkle tree we’ll be using to store
note hashes
2. Add Note - this will call our `append_note` instruction
3. Add Max Size Note - this will call our `append_note` instruction with a note
that maxes out the 1232 bytes allowed in a single transaction
4. Update First Note - this will call our `update_note` instruction to modify
the first note we added
The first test is mostly just for setup. In the last three tests, we’ll be
asserting each time that the note hash on the tree matches what we would expect
given the note text and signer.
Let’s start with our imports. There are quite a few from Anchor,
`@orbition-network/web3.js`, `@orbition-network/spl-account-compression`, and our own utils file.
```typescript theme={null}
import * as anchor from "@coral-xyz/anchor";
import { Program } from "@coral-xyz/anchor";
import { CompressedNotes } from "../target/types/compressed_notes";
import {
Keypair,
Transaction,
PublicKey,
sendAndConfirmTransaction,
Connection,
} from "@orbition-network/web3.js";
import {
ValidDepthSizePair,
createAllocTreeIx,
SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
SPL_NOOP_PROGRAM_ID,
ConcurrentMerkleTreeAccount,
} from "@orbition-network/spl-account-compression";
import { getHash, getNoteLog } from "./utils";
import { assert } from "chai";
```
Next, we’ll want to set up the state variables we’ll be using throughout our
tests. This includes the default Anchor setup as well as generating a Merkle
tree keypair, the tree authority, and some notes.
```typescript theme={null}
describe("compressed-notes", () => {
const provider = anchor.AnchorProvider.env();
anchor.setProvider(provider);
const connection = new Connection(
provider.connection.rpcEndpoint,
"confirmed", // has to be confirmed for some of the methods below
);
const wallet = provider.wallet as anchor.Wallet;
const program = anchor.workspace.CompressedNotes as Program;
// Generate a new keypair for the Merkle tree account
const merkleTree = Keypair.generate();
// Derive the PDA to use as the tree authority for the Merkle tree account
// This is a PDA derived from the Note program, which allows the program to sign for appends instructions to the tree
const [treeAuthority] = PublicKey.findProgramAddressSync(
[merkleTree.publicKey.toBuffer()],
program.programId,
);
const firstNote = "hello world";
const secondNote = "0".repeat(917);
const updatedNote = "updated note";
// TESTS GO HERE
});
```
Finally, let’s start with the tests themselves. First the `Create Note Tree`
test. This test will do two things:
1. Allocate a new account for the Merkle tree with a max depth of 3, max buffer
size of 8, and canopy depth of 0
2. Initialize this new account using our program’s `createNoteTree` instruction
```typescript theme={null}
it("Create Note Tree", async () => {
const maxDepthSizePair: ValidDepthSizePair = {
maxDepth: 3,
maxBufferSize: 8,
};
const canopyDepth = 0;
// instruction to create new account with required space for tree
const allocTreeIx = await createAllocTreeIx(
connection,
merkleTree.publicKey,
wallet.publicKey,
maxDepthSizePair,
canopyDepth,
);
// instruction to initialize the tree through the Note program
const ix = await program.methods
.createNoteTree(maxDepthSizePair.maxDepth, maxDepthSizePair.maxBufferSize)
.accounts({
merkleTree: merkleTree.publicKey,
treeAuthority: treeAuthority,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
})
.instruction();
const tx = new Transaction().add(allocTreeIx, ix);
await sendAndConfirmTransaction(connection, tx, [wallet.payer, merkleTree]);
});
```
Next, we’ll create the `Add Note` test. It should call `append_note` with
`firstNote`, then check that the onchain hash matches our computed hash and that
the note log matches the text of the note we passed into the instruction.
```typescript theme={null}
it("Add Note", async () => {
const txSignature = await program.methods
.appendNote(firstNote)
.accounts({
merkleTree: merkleTree.publicKey,
treeAuthority: treeAuthority,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
})
.rpc();
const noteLog = await getNoteLog(connection, txSignature);
const hash = getHash(firstNote, provider.publicKey);
assert(hash === Buffer.from(noteLog.leafNode).toString("hex"));
assert(firstNote === noteLog.note);
});
```
Next, we’ll create the `Add Max Size Note` test. It is the same as the previous
test, but with the second note.
```typescript theme={null}
it("Add Max Size Note", async () => {
// Size of note is limited by max transaction size of 1232 bytes, minus additional data required for the instruction
const txSignature = await program.methods
.appendNote(secondNote)
.accounts({
merkleTree: merkleTree.publicKey,
treeAuthority: treeAuthority,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
})
.rpc();
const noteLog = await getNoteLog(connection, txSignature);
const hash = getHash(secondNote, provider.publicKey);
assert(hash === Buffer.from(noteLog.leafNode).toString("hex"));
assert(secondNote === noteLog.note);
});
```
Lastly, we’ll create the `Update First Note` test. This is slightly more complex
than adding a note. We’ll do the following:
1. Get the Merkle tree root as it’s required by the instruction.
2. Call the `update_note` instruction of our program, passing in the index 0
(for the first note), the Merkle tree root, the first note, and the updated
data. Remember, it needs the first note and the root because the program must
verify the entire proof path for the note’s leaf before it can be updated.
```typescript theme={null}
it("Update First Note", async () => {
const merkleTreeAccount =
await ConcurrentMerkleTreeAccount.fromAccountAddress(
connection,
merkleTree.publicKey,
);
const rootKey = merkleTreeAccount.tree.changeLogs[0].root;
const root = Array.from(rootKey.toBuffer());
const txSignature = await program.methods
.updateNote(0, root, firstNote, updatedNote)
.accounts({
merkleTree: merkleTree.publicKey,
treeAuthority: treeAuthority,
logWrapper: SPL_NOOP_PROGRAM_ID,
compressionProgram: SPL_ACCOUNT_COMPRESSION_PROGRAM_ID,
})
.rpc();
const noteLog = await getNoteLog(connection, txSignature);
const hash = getHash(updatedNote, provider.publicKey);
assert(hash === Buffer.from(noteLog.leafNode).toString("hex"));
assert(updatedNote === noteLog.note);
});
```
That’s it, congrats! Go ahead and run `anchor test` and you should get four
passing tests.
If you’re running into issues, feel free to go back through some of the demo or
look at the full solution code in the
[Compressed Notes repository](https://github.com/unboxed-software/anchor-compressed-notes).
## Challenge
Now that you’ve practiced the basics of state compression, add a new instruction
to the Compressed Notes program. This new instruction should allow users to
delete an existing note. keep in mind that you can’t remove a leaf from the
tree, so you’ll need to decide what “deleted” looks like for your program. Good
luck!
If you'd like a very simple example of a delete function, check out the
[`solution` branch on GitHub](https://github.com/Unboxed-Software/anchor-compressed-notes/tree/solution).
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=60f6b072-eaeb-469c-b32e-5fea4b72d1d1)!
# Close Mint Extension
Source: https://docs.orbition.network/courses/token-extensions-for-mints/close-mint
Create a mint that can be closed once the tokens are burnt.
## Summary
* The original Token Program only allowed closing token accounts, but not mint
accounts.
* Token Extensions Program includes the `close mint` extension which allows mint
accounts to be closed.
* To close a mint with the `close mint` extension, the supply of said mint needs
to be 0.
* The `mintCloseAuthority` can be updated by calling `setAuthority`
## Overview
The original Token Program only allows owners to close token accounts, not mint
accounts. So if you create a mint, you'll never be able to close the account.
This has resulted in a lot of wasted space on the blockchain. To remedy this,
the Token Extensions Program introduced the `close mint` extension. This simply
allows a mint account to be closed and the lamports refunded. The only caveat,
is the supply of said mint needs to be 0.
This extension is a nice improvement for developers, who may have thousands of
mint accounts that could be cleaned up and be refunded for. Additionally it's
great for NFT holders who wish to burn their NFT. They will now be able to
recuperate all of the costs, ie closing the mint, metadata and token accounts.
Whereas before, if someone burned an NFT would only recuperate the metadata and
token account's rents. Note, the burner would also have to be the
`mintCloseAuthority`.
The `close mint` extension, adds an additional field `mintCloseAuthority` to the
mint account. This is the address of the authority to actually close the
account.
Again, for a mint to be closed with this extension, the supply has to be 0. So
if any of this token is minted, it will have to be burned first.
### Create Mint with Close Authority
Initializing the mint with the close authority extension involves three
instructions:
* `SystemProgram.createAccount`
* `createInitializeMintCloseAuthorityInstruction`
* `createInitializeMintInstruction`
The first instruction `SystemProgram.createAccount` allocates space on the
blockchain for the mint account. However like all Token Extensions Program
mints, we need to calculate the size and cost of the mint. This can be
accomplished by using `getMintLen` and `getMinimumBalanceForRentExemption`. In
this case, we'll call `getMintLen` with only the
`ExtensionType.MintCloseAuthority`.
To get the mint length and create account instruction, do the following:
```ts theme={null}
const extensions = [ExtensionType.MintCloseAuthority];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer,
newAccountPubkey: mint,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
});
```
The second instruction `createInitializeMintCloseAuthorityInstruction`
initializes the close authority extension. The only notable parameter is the
`mintCloseAuthority` in the second position. This is the address that can close
the mint.
```ts theme={null}
const initializeMintCloseAuthorityInstruction =
createInitializeMintCloseAuthorityInstruction(
mint,
authority,
TOKEN_2024_PROGRAM_ID,
);
```
The last instruction `createInitializeMintInstruction` initializes the mint.
```ts theme={null}
const initializeMintInstruction = createInitializeMintInstruction(
mint,
decimals,
payer.publicKey,
null,
TOKEN_2024_PROGRAM_ID,
);
```
Finally we add the instructions to the transaction and submit it to the Orbition Native Chain
network.
```typescript theme={null}
const mintTransaction = new Transaction().add(
createAccountInstruction,
initializeMintCloseAuthorityInstruction,
initializeMintInstruction,
);
const signature = await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
```
When the transaction is sent, a new mint account is created with the specified
close authority.
### Close Mint with Close Authority
To close a mint with the `close mint` extension, all that is needed is to call
the `closeAccount` function.
Remember, that to close the mint account, the total supply has to be 0. So if
any tokens exist, they have to be burned first. You can do this with the `burn`
function.
The `closeAccount` function works for mints and token
accounts alike.
```ts theme={null}
// burn tokens to 0
const burnSignature = await burn(
connection, // connection - Connection to use
payer, // payer -Payer of the transaction fees
sourceAccount, // account - Account to burn tokens from
mintKeypair.publicKey, // mint - Mint for the account
sourceKeypair, // owner - Account owner
sourceAccountInfo.amount, // amount - Amount to burn
[], // multiSigners - Signing accounts if `owner` is a multisig
{ commitment: "finalized" }, // confirmOptions - Options for confirming the transaction
TOKEN_2024_PROGRAM_ID, // programId - SPL Token program account
);
// account can be closed as total supply is now 0
await closeAccount(
connection, // connection - Connection to use
payer, // payer - Payer of the transaction fees
mintKeypair.publicKey, // account - Account to close
payer.publicKey, // destination - Account to receive the remaining balance of the closed account
payer, // authority - Authority which is allowed to close the account
[], // multiSigners - Signing accounts if `authority` is a multisig
{ commitment: "finalized" }, // confirmOptions - Options for confirming the transaction
TOKEN_2024_PROGRAM_ID, // programIdSPL Token program account
);
```
### Update Close Mint Authority
To change the `closeMintAuthority` you can call the `setAuthority` function and
pass in the right accounts, as well as the `authorityType`, which in this case
is `AuthorityType.CloseMint`
```ts theme={null}
/**
* Assign a new authority to the account
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param account Address of the account
* @param currentAuthority Current authority of the specified type
* @param authorityType Type of authority to set
* @param newAuthority New authority of the account
* @param multiSigners Signing accounts if `currentAuthority` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await setAuthority(
connection,
payer,
mint,
currentAuthority,
AuthorityType.CloseMint,
newAuthority,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
## Lab
In this lab, we'll create a mint with the `close mint` extension. We will then
mint some of the tokens and see what happens when we try to close it with a
non-zero supply (hint, the close transaction will fail). Lastly, we will burn
the supply and close the account.
### 1. Getting Started
To get started, create an empty directory named `close-mint` and navigate to it.
We'll be initializing a brand new project. Run `npm init` and follow through the
prompts.
Next, we'll need to add our dependencies. Run the following to install the
required packages:
```bash theme={null}
npm i @orbition-network-developers/helpers @orbition-network/spl-token @orbition-network/web3.js esrun dotenv typescript
```
Create a directory named `src`. In this directory, create a file named
`index.ts`. This is where we will run checks against the rules of this
extension. Paste the following code in `index.ts`:
```ts theme={null}
import { Connection, Keypair, LAMPORTS_PER_SOL } from "@orbition-network/web3.js";
import { initializeKeypair } from "@orbition-network-developers/helpers";
// import { createClosableMint } from './create-mint' // - uncomment this in a later step
import {
TOKEN_2024_PROGRAM_ID,
burn,
closeAccount,
createAccount,
getAccount,
getMint,
mintTo,
} from "@orbition-network/spl-token";
import dotenv from "dotenv";
dotenv.config();
/**
* Create a connection and initialize a keypair if one doesn't already exists.
* If a keypair exists, airdrop a OBN if needed.
*/
const connection = new Connection("http://127.0.0.1:8899");
const payer = await initializeKeypair(connection);
console.log(`public key: ${payer.publicKey.toBase58()}`);
const mintKeypair = Keypair.generate();
const mint = mintKeypair.publicKey;
console.log("\nmint public key: " + mintKeypair.publicKey.toBase58() + "\n\n");
// CREATE A MINT WITH CLOSE AUTHORITY
// MINT TOKEN
// VERIFY SUPPLY
// TRY CLOSING WITH NON ZERO SUPPLY
// BURN SUPPLY
// CLOSE MINT
```
`index.ts` creates a connection to the specified validator node and calls
`initializeKeypair`. This is where we'll end up calling the rest of our script
once we've written it.
Go ahead and run the script. You should see the `payer` and `mint` public key
logged to your terminal.
```bash theme={null}
esrun src/index.ts
```
If you run into an error in `initializeKeypair` with airdropping, follow the
next step.
#### 2. Run validator node
For the sake of this guide, we'll be running our own validator node.
In a separate terminal, run the following command: `solana-test-validator`. This
will run the node and also log out some keys and values. The value we need to
retrieve and use in our connection is the JSON RPC URL, which in this case is
`http://127.0.0.1:8899`. We then use that in the connection to specify to use
the local RPC URL.
```typescript theme={null}
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
```
Alternatively, if you’d like to use testnet or testnet, import the
`clusterApiUrl` from `@orbition-network/web3.js` and pass it to the connection as such:
```typescript theme={null}
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
```
If you decide to use testnet, and have issues with airdropping OBN, feel free to
add the `keypairPath` parameter to `initializeKeypair`. You can get this from
running `solana config get` in your terminal. And then go to
[faucet.orbition.network](https://faucet.orbition.network/) and airdrop some OBN to your
address. You can get your address from running `solana address` in your
terminal.
### 3. Create a mint with close authority
Let's create a closable mint by creating the function `createClosableMint` in a
new file `src/create-mint.ts`.
To create a closable mint, we need several instructions:
* `getMintLen`: Gets the space needed for the mint account
* `SystemProgram.getMinimumBalanceForRentExemption`: Tells us the cost of the
rent for the mint account
* `SystemProgram.createAccount`: Creates the instruction to allocates space on
Orbition Native Chain for the mint account
* `createInitializeMintCloseAuthorityInstruction`: Creates the instruction to
initialize the close mint extension - this takes the `closeMintAuthority` as a
parameter.
* `createInitializeMintInstruction`: Creates the instruction to initialize the
mint
* `sendAndConfirmTransaction`: Sends the transaction to the blockchain
We'll call all of these functions in turn. But before that, let's define the
inputs to our `createClosableMint` function:
* `connection: Connection` : The connection object
* `payer: Keypair` : Payer for the transaction
* `mintKeypair: Keypair` : Keypair for new mint
* `decimals: number` : Mint decimals
Putting it all together we get:
```ts theme={null}
import {
sendAndConfirmTransaction,
Connection,
Keypair,
SystemProgram,
Transaction,
TransactionSignature,
} from "@orbition-network/web3.js";
import {
ExtensionType,
createInitializeMintInstruction,
getMintLen,
TOKEN_2024_PROGRAM_ID,
createInitializeMintCloseAuthorityInstruction,
} from "@orbition-network/spl-token";
export async function createClosableMint(
connection: Connection,
payer: Keypair,
mintKeypair: Keypair,
decimals: number,
): Promise {
const extensions = [ExtensionType.MintCloseAuthority];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
console.log("Creating a transaction with close mint instruction...");
const mintTransaction = new Transaction().add(
SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
}),
createInitializeMintCloseAuthorityInstruction(
mintKeypair.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
),
createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
null,
TOKEN_2024_PROGRAM_ID,
),
);
console.log("Sending transaction...");
const signature = await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
return signature;
}
```
Now let's call this function in `src/index.ts`. First you'll need to import our
new function. Then paste the following under the right comment section:
```ts theme={null}
// CREATE A MINT WITH CLOSE AUTHORITY
const decimals = 9;
await createClosableMint(connection, payer, mintKeypair, decimals);
```
This will create a transaction with close mint instruction.
Feel free to run this and check that everything is working:
```bash theme={null}
esrun src/index.ts
```
### 4. Closing the mint
We're going to close the mint, but first, lets explore what happens when we have
a supply when trying to close (hint, it'll fail).
To do this, we are going to mint some tokens, try to close, then burn the tokens
and then actually close.
#### 4.1 Mint a token
In `src/index.ts`, create an account and mint 1 token to that account.
We can accomplish this by calling 2 functions: `createAccount` and `mintTo`:
```ts theme={null}
// MINT TOKEN
/**
* Creating an account and mint 1 token to that account
*/
console.log("Creating an account...");
const sourceKeypair = Keypair.generate();
const sourceAccount = await createAccount(
connection,
payer,
mint,
sourceKeypair.publicKey,
undefined,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
console.log("Minting 1 token...\n\n");
const amount = 1 * LAMPORTS_PER_SOL;
await mintTo(
connection,
payer,
mint,
sourceAccount,
payer,
amount,
[payer],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
Now we can verify that the mint supply is non-zero by fetching the mint info.
Underneath the minting functions, add the following code block:
```ts theme={null}
// VERIFY SUPPLY
/**
* Get mint information to verify supply
*/
const mintInfo = await getMint(
connection,
mintKeypair.publicKey,
"finalized",
TOKEN_2024_PROGRAM_ID,
);
console.log("Initial supply: ", mintInfo.supply);
```
Let's run the script and check the initial supply:
```bash theme={null}
esrun src/index.ts
```
You should see the following in your terminal:
```bash theme={null}
Initial supply: 1000000000n
```
#### 4.2 Closing the mint with non zero supply
Now we'll attempt to close the mint when supply is non-zero. We know this is
going to fail, since the `close mint` extension requires a non-zero supply. So
to see the resulting error message, we'll wrap the `closeAccount` function in a
try catch and log out the error:
```ts theme={null}
// TRY CLOSING WITH NON ZERO SUPPLY
/**
* Try closing the mint account when supply is not 0
*
* Should throw `SendTransactionError`
*/
try {
await closeAccount(
connection,
payer,
mintKeypair.publicKey,
payer.publicKey,
payer,
[],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
} catch (e) {
console.log(
"Close account fails here because the supply is not zero. Check the program logs:",
(e as any).logs,
"\n\n",
);
}
```
Give this a run:
```bash theme={null}
esrun src/index.ts
```
We'll see that the program throws an error along with the program logs. You
should see the following:
```bash theme={null}
Close account fails here because the supply is not zero.
```
#### 4.3 Burning the supply
Let's burn the whole supply so we can actually close the mint. We do this by
calling `burn`:
```ts theme={null}
// BURN SUPPLY
const sourceAccountInfo = await getAccount(
connection,
sourceAccount,
"finalized",
TOKEN_2024_PROGRAM_ID,
);
console.log("Burning the supply...");
const burnSignature = await burn(
connection,
payer,
sourceAccount,
mintKeypair.publicKey,
sourceKeypair,
sourceAccountInfo.amount,
[],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
#### 4.4 Close the mint
With no tokens in circulation, we can now close the mint. At this point, we can
simply call `closeAccount`, however, for the sake of visualizing how this works,
we'll do the following:
* Retrieve Mint Information: Initially, we fetch and inspect the mint's details, particularly focusing on the supply, which should be zero at this stage. This shows that the mint is eligible to be closed.
* Verify Account Status: Next, we confirm the status of the account to ensure that it is still open and active.
* Close the Account: Once we've verified the account's open status, we proceed to close the mint account.
* Confirm Closure: Finally, after invoking the `closeAccount` function, we check the account status once more to confirm that it has indeed been closed successfully.
We can accomplish all of this with the following functions:
* `getMint`: Grabs the mint account and deserializes the information
* `getAccountInfo`: Grabs the mint account, so we can check it exists - we'll
call this before and after the close.
* `closeAccount`: Closes the mint
Putting this all together we get:
```ts theme={null}
// CLOSE MINT
const mintInfo = await getMint(
connection,
mintKeypair.publicKey,
"finalized",
TOKEN_2024_PROGRAM_ID,
);
console.log("After burn supply: ", mintInfo.supply);
const accountInfoBeforeClose = await connection.getAccountInfo(
mintKeypair.publicKey,
"finalized",
);
console.log("Account closed? ", accountInfoBeforeClose === null);
console.log("Closing account after burning the supply...");
const closeSignature = await closeAccount(
connection,
payer,
mintKeypair.publicKey,
payer.publicKey,
payer,
[],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
const accountInfoAfterClose = await connection.getAccountInfo(
mintKeypair.publicKey,
"finalized",
);
console.log("Account closed? ", accountInfoAfterClose === null);
```
Run the script one last time.
```bash theme={null}
esrun src/index.ts
```
You should see the whole process of creating a closable mint, minting a token,
trying to close, burning the token, and finally closing the account.
That's it! We have successfully created a mint with close authority. If you get
stuck at any point, you can find working code in the `solution` branch of
[this repository](https://github.com/Unboxed-Software/solana-lab-close-mint-account/tree/solution).
## Challenge
For the challenge, try and create your own mint and mint to several token
accounts, then create a script to burn all of those token accounts, then close
the mint.
# Default Account State
Source: https://docs.orbition.network/courses/token-extensions-for-mints/default-account-state
Create token that requires interaction with a specific service to use.
## Summary
* The `default state` extension enables developers to set new token accounts for
a mint with this extension to be frozen by default, requiring interaction with
a specific service to unfreeze and utilize the tokens.
* There are three states of token accounts: Initialized, Uninitialized, and
Frozen, which determine how a token account can be interacted with.
* When a token account is frozen, the balance cannot change.
* The `freezeAuthority` is the only address that can freeze and thaw a token
account
* The `default state` can be updated using `updateDefaultAccountState`
* The lab demonstrates creating a mint with the `default state` extension and
creating a new token account which is set to a frozen state upon creation. The
lab includes tests to ensure the extension works as intended for both minting
and transferring tokens in frozen and thawed states.
## Overview
The `default state` extension allows developers to force all new token accounts
to be in one of two states: "Initialized" or "Frozen". Most usefully, with this
extension any new token accounts created can be set to frozen. When a token
account is frozen, it's balance cannot change. Meaning it cannot be minted to,
transferred from or burned. Only the `freezeAuthority` can thaw a frozen
account.
Imagine you're a Orbition Native Chain game dev, and you only want players of your game to
interact with your in-game token. You can make the player, sign up for the game
to thaw their token account and allow them to play and trade with other players.
This works because of the `default state` extension, where it is set that all
new token accounts are frozen.
#### Types of States
There are 3 types of state with the default account state extension:
* Uninitialized: This state indicates that the token account has been created
but not yet initialized through the Token Program.
* Initialized: An account in the Initialized state has been properly set up
through the Token Program. This means it has a specified mint and an owner has
been assigned.
* Frozen: A Frozen account is one that has been temporarily disabled from
performing certain operations, specifically transferring and minting tokens.
```ts theme={null}
/** Token account state as stored by the program */
export enum AccountState {
Uninitialized = 0,
Initialized = 1,
Frozen = 2,
}
```
However, `default state` only deals with the latter two: `Initialized` and
`Frozen`. When you freeze an account, the state is `Frozen`, when you thaw, it
is `Initialized`.
### Adding default account state
Initializing a mint with transfer fee involves three instructions:
* `SystemProgram.createAccount`
* `createInitializeTransferFeeConfigInstruction`
* `createInitializeMintInstruction`
The first instruction `SystemProgram.createAccount` allocates space on the
blockchain for the mint account. This instruction accomplishes three things:
* Allocates `space`
* Transfers `lamports` for rent
* Assigns to it's owning program
To grab the size of the mint account, we call `getMintLen`, and to grab the
lamports needed for the space, we call `getMinimumBalanceForRentExemption`.
```typescript theme={null}
const mintLen = getMintLen([ExtensionType.DefaultAccountState]);
// Minimum lamports required for Mint Account
const lamports = await connection.getMinimumBalanceForRentExemption(mintLen);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLen,
lamports,
programId: TOKEN_2024_PROGRAM_ID,
});
```
The second
instruction `createInitializeDefaultAccountStateInstruction` initializes the
default account state extension.
```typescript theme={null}
const initializeDefaultAccountStateInstruction =
createInitializeDefaultAccountStateInstruction(
mintKeypair.publicKey, // Mint
defaultState, // Default State
TOKEN_2024_PROGRAM_ID,
);
```
The third instruction `createInitializeMintInstruction` initializes the mint.
```typescript theme={null}
const initializeMintInstruction = createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
);
```
Lastly, add all of these instructions to a transaction and send it to the
blockchain.
```ts theme={null}
const transaction = new Transaction().add(
createAccountInstruction,
initializeDefaultAccountStateInstruction,
initializeMintInstruction,
);
return await sendAndConfirmTransaction(connection, transaction, [
payer,
mintKeypair,
]);
```
### Updating the Default Account State
You can always change the default account state assuming you have the authority
to do so. To do this, simply call `updateDefaultAccountState`.
```ts theme={null}
/**
* Update the default account state on a mint
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param mint Mint to modify
* @param state New account state to set on created accounts
* @param freezeAuthority Freeze authority of the mint
* @param multiSigners Signing accounts if `freezeAuthority` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
export async function updateDefaultAccountState(
connection: Connection,
payer: Signer,
mint: PublicKey,
state: AccountState,
freezeAuthority: Signer | PublicKey,
multiSigners: Signer[] = [],
confirmOptions?: ConfirmOptions,
programId = TOKEN_2024_PROGRAM_ID,
): Promise;
```
### Updating the Freeze Authority
Lastly, you may want to update the `freezeAuthority` to another account. Say you
want to handle the freezing and thawing by a program for example. You can do
this, by calling `setAuthority`, adding in the correct accounts and passing in
the `authorityType`, which in this case would be `AuthorityType.FreezeAccount`.
```ts theme={null}
/**
* Assign a new authority to the account
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param account Address of the account
* @param currentAuthority Current authority of the specified type
* @param authorityType Type of authority to set
* @param newAuthority New authority of the account
* @param multiSigners Signing accounts if `currentAuthority` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await setAuthority(
connection,
payer,
mint,
currentAuthority,
AuthorityType.FreezeAccount,
newAuthority,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
## Lab
In this lab we will be creating a mint which all new token accounts are frozen
upon creation by using the `default state` extension. We will then write tests
to check if the extension is working as intended by attempting to mint and
transfer the tokens in a frozen and thawed account state.
#### 1. Setup Environment
To get started, create an empty directory named `default-account-state` and
navigate to it. We'll be initializing a brand new project. Run `npm init` and
follow through the prompts.
Next, we'll need to add our dependencies. Run the following to install the
required packages:
```bash theme={null}
npm i @orbition-network-developers/helpers @orbition-network/spl-token @orbition-network/web3.js esrun dotenv typescript
```
Create a directory named `src`. In this directory, create a file named
`index.ts`. This is where we will run checks against the rules of this
extension. Paste the following code in `index.ts`:
```ts theme={null}
import {
AccountState,
TOKEN_2024_PROGRAM_ID,
getAccount,
mintTo,
thawAccount,
transfer,
createAccount,
} from "@orbition-network/spl-token";
import { Connection, Keypair, PublicKey } from "@orbition-network/web3.js";
// import { createTokenExtensionMintWithDefaultState } from "./mint-helper"; // This will be uncommented later
import { initializeKeypair, makeKeypairs } from "@orbition-network-developers/helpers";
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
const payer = await initializeKeypair(connection);
const [mintKeypair, ourTokenAccountKeypair, otherTokenAccountKeypair] =
makeKeypairs(3);
const mint = mintKeypair.publicKey;
const decimals = 2;
const defaultState = AccountState.Frozen;
const ourTokenAccount = ourTokenAccountKeypair.publicKey;
// To satisfy the transferring tests
const otherTokenAccount = otherTokenAccountKeypair.publicKey;
const amountToMint = 1000;
const amountToTransfer = 50;
// CREATE MINT WITH DEFAULT STATE
// CREATE TEST TOKEN ACCOUNTS
// TEST: MINT WITHOUT THAWING
// TEST: MINT WITH THAWING
// TEST: TRANSFER WITHOUT THAWING
// TEST: TRANSFER WITH THAWING
```
`index.ts` creates a connection to the specified validator node and calls
`initializeKeypair`. It also has a few variables we will be using in the rest of
this lab. The `index.ts` is where we'll end up calling the rest of our script
once we've written it.
If you run into an error in `initializeKeypair` with airdropping, follow the
next step.
#### 2. Run validator node
For the sake of this guide, we'll be running our own validator node.
In a separate terminal, run the following command: `solana-test-validator`. This
will run the node and also log out some keys and values. The value we need to
retrieve and use in our connection is the JSON RPC URL, which in this case is
`http://127.0.0.1:8899`. We then use that in the connection to specify to use
the local RPC URL.
```typescript theme={null}
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
```
Alternatively, if you’d like to use testnet or testnet, import the
`clusterApiUrl` from `@orbition-network/web3.js` and pass it to the connection as such:
```typescript theme={null}
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
```
If you decide to use testnet, and have issues with airdropping sol. Feel free to
add the `keypairPath` parameter to `initializeKeypair`. You can get this from
running `solana config get` in your terminal. And then go to
[faucet.orbition.network](https://faucet.orbition.network/) and airdrop some sol to your
address. You can get your address from running `solana address` in your
terminal.
#### 3. Helpers
When we pasted the `index.ts` code from earlier, we added the following helpers:
* `initializeKeypair`: This function creates the keypair for the `payer` and
also airdrops some OBN to it
* `makeKeypairs`: This function creates keypairs without airdropping any OBN
Additionally we have some initial accounts:
* `payer`: Used to pay for and be the authority for everything
* `mintKeypair`: Our mint that will have the `default state` extension
* `ourTokenAccountKeypair`: The token account owned by payer that we'll use for
testing
* `otherTokenAccountKeypair`: Another token used for testing
#### 4. Create Mint with Default account state
When creating a mint token with default state, we must create the account
instruction, initialize the default account state for the mint account and
initialize the mint itself.
Create an asynchronous function named `createTokenExtensionMintWithDefaultState`
in `src/mint-helpers.ts`. This function will create the mint such that all new
token accounts will be “frozen” to start. The function will take the following
arguments:
* `connection` : The connection object
* `payer` : Payer for the transaction
* `mintKeypair` : Keypair for the new mint
* `decimals` : Mint decimals
* `defaultState` : Mint token default state - eg: `AccountState.Frozen`
The first step in creating a mint is reserving space on Orbition Native Chain with the
`SystemProgram.createAccount` method. This requires specifying the payer's
keypair, (the account that will fund the creation and provide OBN for rent
exemption), the new mint account's public key (`mintKeypair.publicKey`), the
space required to store the mint information on the blockchain, the amount of
OBN (lamports) necessary to exempt the account from rent and the ID of the token
program that will manage this mint account (`TOKEN_2024_PROGRAM_ID`).
```typescript theme={null}
const mintLen = getMintLen([ExtensionType.DefaultAccountState]);
// Minimum lamports required for Mint Account
const lamports = await connection.getMinimumBalanceForRentExemption(mintLen);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLen,
lamports,
programId: TOKEN_2024_PROGRAM_ID,
});
```
After the mint account creation, the next step involves initializing it with the
default state. The `createInitializeDefaultAccountStateInstruction` function is
used to generate an instruction that enables the mint to set `defaultState` of
any new token accounts.
```typescript theme={null}
const initializeDefaultAccountStateInstruction =
createInitializeDefaultAccountStateInstruction(
mintKeypair.publicKey,
defaultState,
TOKEN_2024_PROGRAM_ID,
);
```
Next, lets add the mint instruction by calling `createInitializeMintInstruction`
and passing in the required arguments. This function is provided by the SPL
Token package and it constructs a transaction instruction that initializes a new
mint.
```typescript theme={null}
const initializeMintInstruction = createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey, // Designated Mint Authority
payer.publicKey, // Designated Freeze Authority
TOKEN_2024_PROGRAM_ID,
);
```
Lastly, let's add all of the instructions to a transaction and send it to the
blockchain:
```typescript theme={null}
const transaction = new Transaction().add(
createAccountInstruction,
initializeDefaultAccountStateInstruction,
initializeMintInstruction,
);
return await sendAndConfirmTransaction(connection, transaction, [
payer,
mintKeypair,
]);
```
Putting it all together, the final `src/mint-helpers.ts` file will look like
this:
```ts theme={null}
import {
AccountState,
ExtensionType,
TOKEN_2024_PROGRAM_ID,
createInitializeDefaultAccountStateInstruction,
createInitializeMintInstruction,
getMintLen,
} from "@orbition-network/spl-token";
import {
Connection,
Keypair,
SystemProgram,
Transaction,
sendAndConfirmTransaction,
} from "@orbition-network/web3.js";
/**
* Creates the token mint with the default state
* @param connection
* @param payer
* @param mintKeypair
* @param decimals
* @param defaultState
* @returns signature of the transaction
*/
export async function createTokenExtensionMintWithDefaultState(
connection: Connection,
payer: Keypair,
mintKeypair: Keypair,
decimals: number = 2,
defaultState: AccountState,
): Promise {
const mintLen = getMintLen([ExtensionType.DefaultAccountState]);
// Minimum lamports required for Mint Account
const lamports = await connection.getMinimumBalanceForRentExemption(mintLen);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLen,
lamports,
programId: TOKEN_2024_PROGRAM_ID,
});
const initializeDefaultAccountStateInstruction =
createInitializeDefaultAccountStateInstruction(
mintKeypair.publicKey,
defaultState,
TOKEN_2024_PROGRAM_ID,
);
const initializeMintInstruction = createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey, // Designated Mint Authority
payer.publicKey, // Designated Freeze Authority
TOKEN_2024_PROGRAM_ID,
);
const transaction = new Transaction().add(
createAccountInstruction,
initializeDefaultAccountStateInstruction,
initializeMintInstruction,
);
return await sendAndConfirmTransaction(connection, transaction, [
payer,
mintKeypair,
]);
}
```
#### 6. Test Setup
Now that we have the ability to create a mint with a default state for all of
it's new token accounts, let's write some tests to see how it functions.
#### 6.1 Create Mint with Default State
Let's first create a mint with the default state of `frozen`. To do this we call
the `createTokenExtensionMintWithDefaultState` function we just created in out
`index.ts` file:
```ts theme={null}
// CREATE MINT WITH DEFAULT STATE
await createTokenExtensionMintWithDefaultState(
connection,
payer,
mintKeypair,
decimals,
defaultState,
);
```
#### 6.2 Create Test Token Accounts
Now, let's create two new Token accounts to test with. We can accomplish this by
calling the `createAccount` helper provided by the SPL Token library. We will
use the keypairs we generated at the beginning: `ourTokenAccountKeypair` and
`otherTokenAccountKeypair`.
```typescript theme={null}
// CREATE TEST TOKEN ACCOUNTS
// Transferring from account
await createAccount(
connection,
payer,
mint,
payer.publicKey,
ourTokenAccountKeypair,
undefined,
TOKEN_2024_PROGRAM_ID,
);
// Transferring to account
await createAccount(
connection,
payer,
mint,
payer.publicKey,
otherTokenAccountKeypair,
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
#### 7 Tests
Now let's write some tests to show the interactions that can be had with the
`default state` extension.
We'll write four tests in total:
* Minting without thawing the recipient's account
* Minting with thawing the recipient's account
* Transferring without thawing the recipient's account
* Transferring with thawing the recipient's account
#### 7.1 Minting without thawing the recipient's account
This test will attempt to mint a token to `ourTokenAccount` without thawing the
account. This test is expected to fail as the account will be frozen on the mint
attempt. Remember: when a token account is frozen, the balance cannot change.
To do this, let's wrap a `mintTo` function in a `try catch` and print out the
respected result:
```typescript theme={null}
// TEST: MINT WITHOUT THAWING
try {
// Attempt to mint without thawing
await mintTo(
connection,
payer,
mint,
ourTokenAccount,
payer.publicKey,
amountToMint,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.error("Should not have minted...");
} catch (error) {
console.log(
"✅ - We expected this to fail because the account is still frozen.",
);
}
```
Test this by running the script:
```bash theme={null}
esrun src/index.ts
```
We should see the following error logged out in the terminal, meaning the
extension is working as intended.
`✅ - We expected this to fail because the account is still frozen.`
#### 7.2 Minting with thawing the recipient's account
This test will attempt to mint a token after thawing the token account. This
test is expected to pass as the account will be thawed on the mint attempt.
We can create this test by calling `thawAccount` and then `mintTo`:
```typescript theme={null}
// TEST: MINT WITH THAWING
// Unfreeze frozen token
await thawAccount(
connection,
payer,
ourTokenAccount,
mint,
payer.publicKey,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
// Mint tokens to tokenAccount
await mintTo(
connection,
payer,
mint,
ourTokenAccount,
payer.publicKey,
amountToMint,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const ourTokenAccountWithTokens = await getAccount(
connection,
ourTokenAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log(
`✅ - The new account balance is ${Number(ourTokenAccountWithTokens.amount)} after thawing and minting.`,
);
```
Go ahead and run the script, the transaction should succeed.
```bash theme={null}
esrun src/index.ts
```
#### 7.3 Transferring without thawing the recipient's account
Now that we’ve tested minting, we can test transferring our tokens frozen and
not. First lets test a transfer without thawing the recipient's token account.
Remember, by default, the `otherTokenAccountKeypair` is frozen due to the
extension.
Again, we expect this test to fail, since the `otherTokenAccountKeypair` is
frozen and it's balance cannot change.
To test this, let's wrap a `transfer` function in a `try catch`:
```typescript theme={null}
// TEST: TRANSFER WITHOUT THAWING
try {
await transfer(
connection,
payer,
ourTokenAccount,
otherTokenAccount,
payer,
amountToTransfer,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.error("Should not have minted...");
} catch (error) {
console.log(
"✅ - We expected this to fail because the account is still frozen.",
);
}
```
Run the test and see the results:
```bash theme={null}
esrun src/index.ts
```
#### 7.4 Transferring with thawing the recipient's account
The last test we'll create tests transferring tokens after thawing the token
account we will be transferring to. This test is expected to pass, since all
token accounts will now be thawed.
We'll do this by calling `thawAccount` and then `transfer`:
```typescript theme={null}
// TEST: TRANSFER WITH THAWING
// Unfreeze frozen token
await thawAccount(
connection,
payer,
otherTokenAccount,
mint,
payer.publicKey,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
await transfer(
connection,
payer,
ourTokenAccount,
otherTokenAccount,
payer,
amountToTransfer,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const otherTokenAccountWithTokens = await getAccount(
connection,
otherTokenAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log(
`✅ - The new account balance is ${Number(
otherTokenAccountWithTokens.amount,
)} after thawing and transferring.`,
);
```
Run all of the tests one last time and see the results:
```bash theme={null}
esrun src/index.ts
```
Remember the key takeaways:
* The `default state` extension, enforces the default state on *all* new token
accounts.
* Frozen account's balance cannot change.
Congratulations! We’ve just created and tested a mint using the default account
extension!
## Challenge
Add tests for burning tokens from frozen and thawed token accounts (hint, one
will fail, one will succeed).
To get you started:
```ts theme={null}
// TEST: Burn tokens in frozen account
await freezeAccount(
connection,
payer,
ourTokenAccount,
mint,
payer.publicKey,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
await burn(
connection,
payer,
ourTokenAccount,
mint,
payer.publicKey,
1,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
# Non-Transferable Token
Source: https://docs.orbition.network/courses/token-extensions-for-mints/non-transferable-token
Create tokens that can't be transferred. Certificates, identity, ticketing and more.
## Summary
* In the original Token Program, creating non-transferrable (sometimes called
"soul-bound") tokens is impossible
* The Token Extension Program's `non-transferrable token` enables
non-transferrable tokens
## Overview
In the Token Program, it's impossible to create a token that cannot be
transferred away. While this may seem unimportant, there are several reasons one
may want to issue a non-transferrable (or "soul-bound") token.
Take the following example: Say you are a Orbition Native Chain game dev, and your new game,
"Bits and Bytes", wants to award achievements to the players. Achievements are
not transferrable, and you want their hard work to be proudly displayed in their
wallet. The solution is to send them a non-transferable NFT. However, in the
Token Program, this is not possible. However, it is in the Token Extension
Program! Enter, the `non-transferable` extension.
Token Extension Program has the `non-transferable` extension which can be used
to create non-transferable mints. These mints can be burned, but they can't be
transferred.
### Creating non-transferable mint account
Initializing a non-transferable mint involves three instruction:
* `SystemProgram.createAccount`
* `createInitializeNonTransferableMintInstruction`
* `createInitializeMintInstruction`
The first instruction `SystemProgram.createAccount` allocates space on the
blockchain for the mint account. This instruction accomplishes three things:
* Allocates `space`
* Transfers `lamports` for rent
* Assigns to itself it's owning program
Like all other extensions, you'll need to calculate the space and lamports
needed for the mint account. You can do this by calling: `getMintLen` and
`getMinimumBalanceForRentExemption`.
```ts theme={null}
const extensions = [ExtensionType.NonTransferable];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
});
```
The second instruction `createInitializeNonTransferableMintInstruction`
initializes the non-transferable extension.
```ts theme={null}
const initializeNonTransferableMintInstruction =
createInitializeNonTransferableMintInstruction(
mintKeypair.publicKey,
TOKEN_2024_PROGRAM_ID,
);
```
The third instruction `createInitializeMintInstruction` initializes the mint.
```ts theme={null}
const initializeMintInstruction = createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
null, // Confirmation Config
TOKEN_2024_PROGRAM_ID,
);
```
Lastly, add all of the instructions to a transaction and send to Orbition Native Chain.
```ts theme={null}
const mintTransaction = new Transaction().add(
createAccountInstruction,
initializeNonTransferableMintInstruction,
initializeMintInstruction,
);
await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
```
And that's it! You now have a mint account, that when minted, cannot be
transferred. This extension gets more exciting when you mix it with the
`metadata` and `metadata-pointer` extensions to create soul-bound NFTs.
## Lab
In this lab, we will create a non-transferable token and then see what happens
when we try to transfer it (hint: it will fail the transfer).
#### 1. Getting started
To get started, create an empty directory named `non-transferable-token` and
navigate to it. We'll be initializing a brand new project. Run `npm init` and
follow through the prompts.
Next, we'll need to add our dependencies. Run the following to install the
required packages:
```bash theme={null}
npm i @orbition-network-developers/helpers @orbition-network/spl-token @orbition-network/web3.js esrun dotenv typescript
```
Create a directory named `src`. In this directory, create a file named
`index.ts`. This is where we will run checks against the rules of this
extension. Paste the following code in `index.ts`:
```ts theme={null}
import { Connection, Keypair } from "@orbition-network/web3.js";
import { initializeKeypair } from "@orbition-network-developers/helpers";
import dotenv from "dotenv";
import {
createAccount,
mintTo,
TOKEN_2024_PROGRAM_ID,
} from "@orbition-network/spl-token";
// import { createNonTransferableMint } from './create-mint';
dotenv.config();
/**
* Create a connection and initialize a keypair if one doesn't already exists.
* If a keypair exists, airdrop a sol if needed.
*/
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
const payer = await initializeKeypair(connection);
console.log(`public key: ${payer.publicKey.toBase58()}`);
const mintKeypair = Keypair.generate();
const mint = mintKeypair.publicKey;
console.log("\nmint public key: " + mintKeypair.publicKey.toBase58() + "\n\n");
// CREATE MINT
// CREATE SOURCE ACCOUNT AND MINT TOKEN
// CREATE DESTINATION ACCOUNT FOR TRANSFER
// TRY TRANSFER
```
This file has a main function that creates a connection to the specified
validator node and calls `initializeKeypair`. This main function is where we'll
end up calling the rest of our script once we've written it.
Go ahead and run the script. You should see the `mint` public key logged to your
terminal.
```bash theme={null}
esrun src/index.ts
```
If you run into an error in `initializeKeypair` with airdropping, follow the
next step.
#### 2. Setting up dev environment (optional)
If you are having issues with airdropping testnet OBN. You can either:
1. Add the `keypairPath` parameter to `initializeKeypair` and get some testnet
OBN from [Orbition Native Chain's faucet.](https://faucet.orbition.network/)
2. Run a local validator by doing the following:
In a separate terminal, run the following command: `solana-test-validator`. This
will run the node and also log out some keys and values. The value we need to
retrieve and use in our connection is the JSON RPC URL, which in this case is
`http://127.0.0.1:8899`. We then use that in the connection to specify to use
the local RPC URL.
```typescript theme={null}
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
```
#### 3. Create a non-transferable mint
Let's create the function `createNonTransferableMint` in a new file
`src/create-mint.ts`.
Inside the file, create the function `createNonTransferableMint` with the
following arguments:
* `connection` : The connection object
* `payer` : Payer for the transaction
* `mintKeypair` : Keypair for new mint
* `decimals` : Mint decimals
Inside the function, we'll call the following:
* `getMintLen` - to get the space needed for the mint account
* `getMinimumBalanceForRentExemption` - to get the amount of lamports needed for
the mint account
* `createAccount` - Allocates space on the blockchain for the mint account
* `createInitializeNonTransferableMintInstruction` - initializes the extension
* `createInitializeMintInstruction` - initializes the mint
* `sendAndConfirmTransaction` - sends the transaction to the blockchain
```typescript theme={null}
import {
sendAndConfirmTransaction,
Connection,
Keypair,
SystemProgram,
Transaction,
TransactionSignature,
} from "@orbition-network/web3.js";
import {
ExtensionType,
createInitializeMintInstruction,
getMintLen,
TOKEN_2024_PROGRAM_ID,
createInitializeNonTransferableMintInstruction,
} from "@orbition-network/spl-token";
export async function createNonTransferableMint(
connection: Connection,
payer: Keypair,
mintKeypair: Keypair,
decimals: number,
): Promise {
const extensions = [ExtensionType.NonTransferable];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
console.log("Creating a transaction with non-transferable instruction...");
const mintTransaction = new Transaction().add(
SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
}),
createInitializeNonTransferableMintInstruction(
mintKeypair.publicKey,
TOKEN_2024_PROGRAM_ID,
),
createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
null,
TOKEN_2024_PROGRAM_ID,
),
);
const signature = await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
return signature;
}
```
Now let's invoke this function in `src/index.ts` to create the non-transferable
mint:
```typescript theme={null}
// CREATE MINT
const decimals = 9;
await createNonTransferableMint(connection, payer, mintKeypair, decimals);
```
The script should run with no errors
```bash theme={null}
esrun src/index.ts
```
The non-transferable mint has been set up correctly and will be created when we
run `npm start`. Let’s move on to the next step and create a source account and
mint a token to it.
#### 4. Mint token
Let's test that we can't actually transfer tokens created from this mint. To do
this, we need to mint a token to an account.
Let's do this in `src/index.ts`. Let's create a source account and mint one
non-transferable token.
We can accomplish this in two functions:
* `getOrCreateAssociatedTokenAccount`: from the `@orbition-network/spl-token` library,
this creates an associated token account (ATA) for the given mint and owner.
* `mintTo`: This function will mint an `amount` of tokens to the given token
account.
```typescript theme={null}
// CREATE PAYER ATA AND MINT TOKEN
console.log("Creating an Associated Token Account...");
const ata = (
await getOrCreateAssociatedTokenAccount(
connection,
payer,
mint,
payer.publicKey,
undefined,
undefined,
undefined,
TOKEN_2024_PROGRAM_ID,
)
).address;
console.log("Minting 1 token...");
const amount = 1 * 10 ** decimals;
await mintTo(
connection,
payer,
mint,
ata,
payer,
amount,
[payer],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
const tokenBalance = await connection.getTokenAccountBalance(ata, "finalized");
console.log(
`Account ${ata.toBase58()} now has ${tokenBalance.value.uiAmount} token.`,
);
```
Run the script and confirm a token has been minted to an account:
```bash theme={null}
esrun src/index.ts
```
#### 5. Attempt to transfer a non-transferable token
Lastly, let's try and actually transfer the token somewhere else. First we need
to create a token account to transfer to, and then we want to try and transfer.
In `src/index.ts`, we will create a destination account and try to transfer the
non-transferable token to this account.
We can accomplish this in two functions:
* `createAccount`: This will create a token account for a given mint and the
keypair of said account. So instead of using an ATA here, let's generate a new
keypair as the token account. We're doing this just to show different options
of accounts.
* `transferChecked`: This will attempt to transfer the token.
First, the `createAccount` function:
```typescript theme={null}
// CREATE DESTINATION ACCOUNT FOR TRANSFER
console.log("Creating a destination account...\n\n");
const destinationKeypair = Keypair.generate();
const destinationAccount = await createAccount(
connection,
payer,
mintKeypair.publicKey,
destinationKeypair.publicKey,
undefined,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
Now, the `transferChecked` function:
```typescript theme={null}
// TRY TRANSFER
console.log("Attempting to transfer non-transferable mint...");
try {
const signature = await transferChecked(
connection,
payer,
ata,
mint,
destinationAccount,
ata,
amount,
decimals,
[destinationKeypair],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
} catch (e) {
console.log(
"This transfer is failing because the mint is non-transferable. Check out the program logs: ",
(e as any).logs,
"\n\n",
);
}
```
Now let's run everything and see what happens:
```
esrun src/index.ts
```
You should get an error message at the very end that says
`Transfer is disabled for this mint`. This is indicating that the token we are
attempting to transfer is in fact non-transferable!
```bash theme={null}
Attempting to transfer non-transferable mint...
This transfer is failing because the mint is non-transferable. Check out the program logs: [
'Program TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb invoke [1]',
'Program log: Instruction: TransferChecked',
'Program log: Transfer is disabled for this mint',
'Program TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb consumed 3910 of 200000 compute units',
'Program TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb failed: custom program error: 0x25'
]
```
That's it! We have successfully created a non-transferable mint. If you are
stuck at any point, you can find the working code on the `solution` branch of
[this repository](https://github.com/Unboxed-Software/solana-lab-non-transferable-token/tree/solution).
## Challenge
For the challenge, create your own non-transferable token with the metadata
extension and keep a “soulbound” NFT to yourself.
# Metadata and Metadata Pointer Extension
Source: https://docs.orbition.network/courses/token-extensions-for-mints/token-extensions-metadata
Include token metadata directly inside the token mint account.
## Summary
* The `metadata pointer` extension associates a token mint directly to a
metadata account. This happens by storing the metadata account's address in
the mint. This metadata account address can be an external metadata account,
like Metaplex, or can be the mint itself if using the `metadata` extension.
* The `metadata` mint extension allows embedding of metadata directly into mint
accounts through the Token Extensions Program. This is always accompanied with
a self-referencing `metadata pointer`. This facilitates embedding
comprehensive token information at the minting stage.
* These extensions enhance the interoperability of tokens across different
applications and platforms by standardizing how metadata is associated and
accessed.
* Directly embedding or pointing to metadata can streamline transactions and
interactions by reducing the need for additional lookups or external calls.
## Overview
The Token Extensions Program streamlines metadata on Orbition Native Chain. Without the Token
Extensions Program, developers store metadata in metadata accounts using a
metadata onchain program; mainly `Metaplex`. However, this has some drawbacks.
For example the mint account to which the metadata is "attached" has no
awareness of the metadata account. To determine if an account has metadata, we
have to PDA the mint and the `Metaplex` program together and query the network
to see if a Metadata account exists. Additionally, to create and update this
metadata you have to use a secondary program (i.e. `Metaplex`). These processes
introduces vender lock in and increased complexity. Token Extension Programs's
Metadata extensions fix this by introducing two extensions:
* `metadata-pointer` extension: Adds two simple fields in the mint account
itself: a publicKey pointer to the account that holds the metadata for the
token following the
[Token-Metadata Interface](https://github.com/orbition-network/solana-program-library/tree/master/token-metadata/interface),
and the authority to update this pointer.
* `metadata` extension: Adds the fields described in the
[Token-Metadata Interface](https://github.com/orbition-network/solana-program-library/tree/master/token-metadata/)
which allows us to store the metadata in the mint itself.
### Metadata-Pointer extension:
Since multiple metadata programs exist, a mint can have numerous accounts
claiming to describe the mint, making it complicated to know which one is the
mint's "official" metadata. To resolve this, the `metadata-pointer` extension
adds a `publicKey` field to the mint account called `metadataAddress`, which
points to the account that holds the metadata for this token. To avoid imitation
mints claiming to be a stablecoin, a client can now check whether the mint and
the metadata point to each other.
The extension adds two new fields to the mint account to accomplish this:
* `metadataAddress`: Holds the metadata account address for this token; it can
point to itself if you use the `metadata` extension.
* `authority`: The authority that can set the metadata address.
The extension also introduces three new helper functions:
* `createInitializeMetadataPointerInstruction`
* `createUpdateMetadataPointerInstruction`
* `getMetadataPointerState`
The function `createInitializeMetadataPointerInstruction` will return the
instruction that will set the metadata address in the mint account.
This function takes four parameters:
* `mint`: the mint account that will be created
* `authority`: the authority that can set the metadata address
* `metadataAddress`: the account address that holds the metadata
* `programId`: the SPL Token program ID (in this case, it will be the Token
Extension program ID)
```ts theme={null}
function createInitializeMetadataPointerInstruction(
mint: PublicKey,
authority: PublicKey | null,
metadataAddress: PublicKey | null,
programId: PublicKey,
);
```
The `createUpdateMetadataPointerInstruction` function returns an instruction
that will update the mint account's metadata address. You can update the
metadata pointer at any point if you hold the authority.
This function takes five parameters:
* `mint`: the mint account that will be created.
* `authority`: the authority that can set the metadata address
* `metadataAddress`: the account address that holds the metadata
* `multiSigners`: the multi-signers that will sign the transaction
* `programId`: the SPL Token program ID (in this case, it will be the Token
Extension program ID)
```ts theme={null}
function createUpdateMetadataPointerInstruction(
mint: PublicKey,
authority: PublicKey,
metadataAddress: PublicKey | null,
multiSigners: (Signer | PublicKey)[] = [],
programId: PublicKey = TOKEN_2024_PROGRAM_ID,
);
```
The `getMetadataPointerState` function will return the `MetadataPointer` state
for the given `Mint` object. We can get this using the `getMint` function.
```ts theme={null}
function getMetadataPointerState(mint: Mint): Partial | null;
```
```ts theme={null}
export interface MetadataPointer {
/** Optional authority that can set the metadata address */
authority: PublicKey | null;
/** Optional Account Address that holds the metadata */
metadataAddress: PublicKey | null;
}
```
#### Create NFT with metadata-pointer
To create an NFT with the `metadata-pointer` extension, we need two new
accounts: the `mint` and the `metadataAccount`.
The `mint` is usually a new `Keypair` created by `Keypair.generate()`. The
`metadataAccount` can be the `mint`'s `publicKey` if using the metadata mint
extension or another metadata account like from Metaplex.
At this point, the `mint` is only a `Keypair`, but we need to save space for it
on the blockchain. All accounts on the Orbition Native Chain blockchain owe rent proportional
to the size of the account, and we need to know how big the mint account is in
bytes. We can use the `getMintLen` method from the `@orbition-network/spl-token` library.
The metadata-pointer extension increases the size of the mint account by adding
two new fields: `metadataAddress` and `authority`.
```ts theme={null}
const mintLen = getMintLen([ExtensionType.MetadataPointer]);
const lamports = await connection.getMinimumBalanceForRentExemption(mintLen);
```
To create and initialize the `mint` with the metadata pointer, we need several
instructions in a particular order:
1. Create the `mint` account, which reserves space on the blockchain with
`SystemProgram.createAccount`
2. Initialize the metadata pointer extension with
`createInitializeMetadataPointerInstruction`
3. Initialize the mint itself with `createInitializeMintInstruction`
```ts theme={null}
const createMintAccountInstructions = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
lamports,
newAccountPubkey: mint.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
space: mintLen,
});
const initMetadataPointerInstructions =
createInitializeMetadataPointerInstruction(
mint.publicKey,
payer.publicKey,
metadataAccount,
TOKEN_2024_PROGRAM_ID,
);
const initMintInstructions = createInitializeMintInstruction(
mint.publicKey,
decimals,
payer.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
);
```
To create the NFT, add the instructions to a transaction and send it to the
Orbition Native Chain network:
```ts theme={null}
const transaction = new Transaction().add(
createMintAccountInstructions,
initMetadataPointerInstructions,
initMintInstructions,
);
const sig = await sendAndConfirmTransaction(connection, transaction, [
payer,
mint,
]);
```
### Metadata extension:
The `metadata` extension is an exciting addition to the Token Extensions
Program. This extension allows us to store the metadata directly *in* the mint
itself! This eliminates the need for a separate account, greatly simplifying the
handling of metadata.
The added fields and functions in the metadata extension follow the
[Token-Metadata Interface](https://github.com/orbition-network/solana-program-library/tree/master/token-metadata/interface)
When a mint is initialized with the metadata extension, it will store these
extra fields:
```rust theme={null}
type Pubkey = [u8; 32];
type OptionalNonZeroPubkey = Pubkey; // if all zeroes, interpreted as `None`
pub struct TokenMetadata {
/// The authority that can sign to update the metadata
pub update_authority: OptionalNonZeroPubkey,
/// The associated mint, used to counter spoofing to be sure that metadata
/// belongs to a particular mint
pub mint: Pubkey,
/// The longer name of the token
pub name: String,
/// The shortened symbol for the token
pub symbol: String,
/// The URI pointing to richer metadata
pub uri: String,
/// Any additional metadata about the token as key-value pairs. The program
/// must avoid storing the same key twice.
pub additional_metadata: Vec<(String, String)>,
}
```
With these added fields, the `@orbition-network/spl-token-metadata` library has been
updated with the following functions to help out:
* `createInitializeInstruction`
* `createUpdateFieldInstruction`
* `createRemoveKeyInstruction`
* `createUpdateAuthorityInstruction`
* `createEmitInstruction`
* `pack`
* `unpack`
Additionally, the @orbition-network/spl-token library introduces a new function and two
constants:
* `getTokenMetadata`
* `LENGTH_SIZE`: a constant number of bytes of the length of the data
* `TYPE_SIZE`: a constant number of bytes of the type of the data
The function `createInitializeInstruction` initializes the metadata in the
account and sets the primary metadata fields (name, symbol, URI). The function
then returns an instruction that will set the metadata fields in the mint
account.
This function takes eight parameters:
* `mint`: the mint account that will be initialize
* `metadata`: the metadata account that will be created
* `mintAuthority`: the authority that can mint tokens
* `updateAuthority`: the authority that can sign to update the metadata
* `name`: the longer name of the token
* `symbol`: the shortened symbol for the token, also known as the ticker
* `uri`: the token URI pointing to richer metadata
* `programId`: the SPL Token program ID (in this case it will be the Token
Extension program ID)
```ts theme={null}
export interface InitializeInstructionArgs {
programId: PublicKey;
metadata: PublicKey;
updateAuthority: PublicKey;
mint: PublicKey;
mintAuthority: PublicKey;
name: string;
symbol: string;
uri: string;
}
export function createInitializeInstruction(
args: InitializeInstructionArgs,
): TransactionInstruction;
```
The function `createUpdateFieldInstruction` returns the instruction that creates
or updates a field in a token-metadata account.
This function takes five parameters:
* `metadata`: the metadata account address.
* `updateAuthority`: the authority that can sign to update the metadata
* `field`: the field that we want to update, this is either one of the built in
`Field`s or a custom field stored in the `additional_metadata` field
* `value`: the updated value of the field
* `programId`: the SPL Token program Id (in this case it will be the Token
Extension program Id)
```ts theme={null}
export enum Field {
Name,
Symbol,
Uri,
}
export interface UpdateFieldInstruction {
programId: PublicKey;
metadata: PublicKey;
updateAuthority: PublicKey;
field: Field | string;
value: string;
}
export function createUpdateFieldInstruction(
args: UpdateFieldInstruction,
): TransactionInstruction;
```
The function `createRemoveKeyInstruction` returns the instruction that removes
the `additional_metadata` field from a token-metadata account.
This function takes five parameters:
* `metadata`: the metadata account address
* `updateAuthority`: the authority that can sign to update the metadata
* `field`: the field that we want to remove
* `programId`: the SPL Token program ID (in this case it will be the Token
Extension program ID)
* `idempotent`: When true, instruction will not error if the key does not exist
```ts theme={null}
export interface RemoveKeyInstructionArgs {
programId: PublicKey;
metadata: PublicKey;
updateAuthority: PublicKey;
key: string;
idempotent: boolean;
}
export function createRemoveKeyInstruction(
args: RemoveKeyInstructionArgs,
): TransactionInstruction;
```
The function `createUpdateAuthorityInstruction` returns the instruction that
updates the authority of a token-metadata account.
This function takes four parameters:
* `metadata`: the metadata account address
* `oldAuthority`: the current authority that can sign to update the metadata
* `newAuthority`: the new authority that can sign to update the metadata
* `programId`: the SPL Token program ID (in this case it will be the Token
Extension program ID)
```ts theme={null}
export interface UpdateAuthorityInstructionArgs {
programId: PublicKey;
metadata: PublicKey;
oldAuthority: PublicKey;
newAuthority: PublicKey | null;
}
export function createUpdateAuthorityInstruction(
args: UpdateAuthorityInstructionArgs,
): TransactionInstruction;
```
The function `createEmitInstruction` "emits" or logs out token-metadata in the
expected TokenMetadata state format. This is a required function for metadata
programs that want to follow the TokenMetadata interface. The emit instruction
allows indexers and other off-chain users to call to get metadata. This also
allows custom metadata programs to store
[metadata in a different format while maintaining compatibility with the Interface standards](https://orbition.network/developers/guides/token-extensions/metadata-pointer#metadata-interface-instructions).
This function takes four parameters:
* `metadata`: the metadata account address
* `programId`: the SPL Token program ID (in this case it will be the Token
Extension program ID)
* `start`: *Optional* the start the metadata
* `end`: *Optional* the end the metadata
```ts theme={null}
export interface EmitInstructionArgs {
programId: PublicKey;
metadata: PublicKey;
start?: bigint;
end?: bigint;
}
export function createEmitInstruction(
args: EmitInstructionArgs,
): TransactionInstruction;
```
The `pack` function encodes metadata into a byte array, while its counterpart,
`unpack`, decodes metadata from a byte array. These operations are essential for
determining the metadata's byte size, crucial for allocating adequate storage
space.
```ts theme={null}
export interface TokenMetadata {
// The authority that can sign to update the metadata
updateAuthority?: PublicKey;
// The associated mint, used to counter spoofing to be sure that metadata belongs to a particular mint
mint: PublicKey;
// The longer name of the token
name: string;
// The shortened symbol for the token
symbol: string;
// The URI pointing to richer metadata
uri: string;
// Any additional metadata about the token as key-value pairs
additionalMetadata: [string, string][];
}
export const pack = (meta: TokenMetadata): Uint8Array
export function unpack(buffer: Buffer | Uint8Array): TokenMetadata
```
The function `getTokenMetadata` returns the metadata for the given mint.
It takes four parameters:
* `connection`: Connection to use
* `address`: mint account
* `commitment`: desired level of commitment for querying the state
* `programId`: SPL Token program account (in this case it will be the Token
Extension program ID)
```ts theme={null}
export async function getTokenMetadata(
connection: Connection,
address: PublicKey,
commitment?: Commitment,
programId = TOKEN_2024_PROGRAM_ID,
): Promise;
```
#### Create NFT with metadata extension
Creating an NFT with the metadata extension is just like creating one with the
metadata-pointer with a few extra steps:
1. Gather our needed accounts
2. Find/decide on the needed size of our metadata
3. Create the `mint` account
4. Initialize the pointer
5. Initialize the mint
6. Initialize the metadata in the mint account
7. Add any additional custom fields if needed
First, the `mint` will be a Keypair, usually generated using
`Keypair.generate()`. Then, we must decide what metadata to include and
calculate the total size and cost.
A mint account's size with the metadata and metadata-pointer extensions
incorporate the following:
1. the basic metadata fields: name, symbol, and URI
2. the additional custom fields we want to store as a metadata
3. the update authority that can change the metadata in the future
4. the `LENGTH_SIZE` and `TYPE_SIZE` constants from the `@orbition-network/spl-token`
library - these are sizes associated with mint extensions that are usually
added with the call `getMintLen`, but since the metadata extension is
variable length, they need to be added manually
5. the metadata pointer data (this will be the mint's address and is done for
consistency)
There is no need to allocate more space than what is
necessary if you're anticipating more metadata. The
`createUpdateFieldInstruction` will automatically reallocate space! However,
you'll have to add another `system.transfer` transaction to make sure the mint
account has enough rent.
To determine all of this programmatically, we use the `getMintLen` and `pack`
functions from the `@orbition-network/spl-token` library:
```ts theme={null}
const metadata: TokenMetadata = {
mint: mint.publicKey,
name: tokenName,
symbol: tokenSymbol,
uri: tokenUri,
additionalMetadata: [["customField", "customValue"]],
};
const mintAndPointerLen = getMintLen([ExtensionType.MetadataPointer]); // Metadata extension is variable length, so we calculate it below
const metadataLen = TYPE_SIZE + LENGTH_SIZE + pack(metadata).length;
const totalLen = mintLen + mintAndPointerLen;
const lamports = await connection.getMinimumBalanceForRentExemption(totalLen);
```
To actually create and initialize the `mint` with the metadata and metadata
pointer, we need several instructions in a particular order:
1. Create the `mint` account which reserves space on the blockchain with
`SystemProgram.createAccount`
2. Initialize the metadata pointer extension with
`createInitializeMetadataPointerInstruction`
3. Initialize the mint itself with `createInitializeMintInstruction`
4. Initialize the metadata with `createInitializeInstruction` (this ONLY sets
the basic metadata fields)
5. Optional: Set the custom fields with `createUpdateFieldInstruction` (one
field per call)
```ts theme={null}
const createMintAccountInstructions = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
lamports,
newAccountPubkey: mint.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
space: totalLen,
});
const initMetadataPointerInstructions =
createInitializeMetadataPointerInstruction(
mint.publicKey,
payer.publicKey,
mint.publicKey, // we will point to the mint it self as the metadata account
TOKEN_2024_PROGRAM_ID,
);
const initMintInstructions = createInitializeMintInstruction(
mint.publicKey,
decimals,
payer.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
);
const initMetadataInstruction = createInitializeInstruction({
programId: TOKEN_2024_PROGRAM_ID,
mint: mint.publicKey,
metadata: mint.publicKey,
name: metadata.name,
symbol: metadata.symbol,
uri: metadata.uri,
mintAuthority: payer.publicKey,
updateAuthority: payer.publicKey,
});
const updateMetadataFieldInstructions = createUpdateFieldInstruction({
metadata: mint.publicKey,
updateAuthority: payer.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
field: metadata.additionalMetadata[0][0],
value: metadata.additionalMetadata[0][1],
});
```
Wrap all of these instructions in a transaction to create the embedded NFT:
```ts theme={null}
const transaction = new Transaction().add(
createMintAccountInstructions,
initMetadataPointerInstructions,
initMintInstructions,
initMetadataInstruction,
updateMetadataFieldInstructions, // if you want to add any custom field
);
const signature = await sendAndConfirmTransaction(connection, transaction, [
payer,
mint,
]);
```
Again, the order here matters.
The `createUpdateFieldInstruction` updates only one field
at a time. If you want to have more than one custom field, you will have to call
this method multiple times. Additionally, you can use the same method to update
the basic metadata fields as well:
```ts theme={null}
const updateMetadataFieldInstructions = createUpdateFieldInstruction({
metadata: mint.publicKey,
updateAuthority: payer.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
field: "name", // Field | string
value: "new name",
});
```
## Lab
Now it is time to practice what we have learned so far. In this lab, we'll
create a script that will illustrate how to create an NFT with the `metadata`
and `metadata pointer` extensions.
### 0. Getting started
Let's go ahead and clone our starter code:
```bash theme={null}
git clone https://github.com/Unboxed-Software/solana-lab-token22-metadata.git
cd solana-lab-token22-metadata
git checkout starter
npm install
```
Let's take a look at what's been provided in the `starter` branch.
Along with the NodeJS project being initialized with all of the needed
dependencies, two other files have been provided in the `src/` directory.
* `cat.png`
* `helpers.ts`
* `index.ts`
**`cat.png`** is the image we'll use for the NFT. Feel free to replace it with
your own image.
we are using Irys on testnet to upload files, this is capped
at 100 KiB.
**`helpers.ts`** file provides us with a useful helper function
`uploadOffChainMetadata`.
`uploadOffChainMetadata` is a helper to store the off-chain metadata on Arweave
using Irys (formerly Bundlr). In this lab we will be more focused on the Token
Extensions Program interaction, so this uploader function is provided. It is
important to note that an NFT or any off-chain metadata can be stored anywhere
with any storage provider like [NFT.storage](https://nft.storage/), Orbition Native Chain's
native [ShadowDrive](https://www.shdwdrive.com/), or
[Irys (formerly Bundlr)](https://irys.xyz/). At the end of the day, all you need
is a url to the hosted metadata json file.
This helper has some exported interfaces. These will clean up our functions as
we make them.
```ts theme={null}
export interface CreateNFTInputs {
payer: Keypair;
connection: Connection;
tokenName: string;
tokenSymbol: string;
tokenUri: string;
tokenAdditionalMetadata?: Record;
}
export interface UploadOffChainMetadataInputs {
tokenName: string;
tokenSymbol: string;
tokenDescription: string;
tokenExternalUrl: string;
tokenAdditionalMetadata?: Record;
imagePath: string;
metadataFileName: string;
}
```
**`index.ts`** is where we'll add our code. Right now, the code sets up a
`connection` and initializes a keypair for us to use.
The keypair `payer` will be responsible for every payment we need throughout the
whole process. `payer` will also hold all the authorities, like the mint
authority, mint freeze authority, etc. While it's possible to use a distinct
keypair for the authorities, for simplicity's sake, we'll continue using
`payer`.
Lastly, this lab will all be done on testnet. This is because we are using Irys
to upload metadata to Arweave - the requires a testnet or mainnet connection. If
you are running into airdropping problems:
* Add the `keypairPath` parameter to `initializeKeypair` - path can be gotten by
running `solana config get` in your terminal
* Get the address of your keypair by running `solana address` in your terminal
* Copy the address and airdrop some testnet sol from
[faucet.solana](https://faucet.orbition.network/).
### 1. Uploading the off-chain metadata
In this section we will decide on our NFT metadata and upload our files to
NFT.Storage using the helper functions provided in the starting code.
To upload our off-chain metadata, we need to first prepare an image that will
represent our NFT. We've provided `cat.png`, but feel free to replace it with
your own. Most image types are supported by most wallets. (Again devenet Irys
allows up to 100KiB per file)
Next, let's decide on what metadata our NFT will have. The fields we are
deciding on are `name`, `description`, `symbol`, `externalUrl`, and some
`attributes` (additional metadata). We'll provide some cat adjacent metadata,
but feel free to make up your own.
* `name`: Cat NFT
* `description` = This is a cat
* `symbol` = EMB
* `externalUrl` = [https://orbition.network/](https://orbition.network/)
* `attributes` = `{ species: 'Cat' breed: 'Cool' }`
Lastly we just need to format all of this data and send it to our helper
function `uploadOffChainMetadata` to get the uploaded metadata uri.
When we put all of this together, the `index.ts` file will look as follows:
```ts theme={null}
import { Connection } from "@orbition-network/web3.js";
import { initializeKeypair } from "@orbition-network-developers/helpers";
import { uploadOffChainMetadata } from "./helpers";
import dotenv from "dotenv";
dotenv.config();
const connection = new Connection(clusterApiUrl("testnet"), "finalized");
const payer = await initializeKeypair(connection, {
keypairPath: "your/path/to/keypair.json",
});
const imagePath = "src/cat.png";
const metadataPath = "src/temp.json";
const tokenName = "Cat NFT";
const tokenDescription = "This is a cat";
const tokenSymbol = "EMB";
const tokenExternalUrl = "https://orbition.network/";
const tokenAdditionalMetadata = {
species: "Cat",
breed: "Cool",
};
const tokenUri = await uploadOffChainMetadata(
{
tokenName,
tokenDescription,
tokenSymbol,
imagePath,
metadataPath,
tokenExternalUrl,
tokenAdditionalMetadata,
},
payer,
);
// You can log the URI here and run the code to test it
console.log("Token URI:", tokenUri);
```
Now run `npm run start` in your terminal and test your code. You should see the
URI logged once the uploading is done. If you visit the link you should see a
JSON object that holds all of our off-chain metadata.
### 2. Create NFT function
Creating an NFT involves multiple instructions. As a best practice when writing
scripts that engage with the Orbition Native Chain network, it is best to consolidate all of
these instructions in one transaction due to the atomic nature of transactions.
This ensures either the successful execution of all instructions or a complete
rollback in case of errors. That being said, we're going to make a new function
`createNFTWithEmbeddedMetadata` in a new file called
`src/nft-with-embedded-metadata.ts`.
This function will create an NFT by doing the following:
1. Create the metadata object
2. Allocate the mint
3. Initialize the metadata-pointer making sure that it points to the mint itself
4. Initialize the mint
5. Initialize the metadata inside the mint (that will set name, symbol, and uri
for the mint)
6. Set the additional metadata in the mint
7. Create the associated token account and mint the NFT to it and remove the
mint authority
8. Put all of that in one transaction and send it to the network
9. Fetch and print the token account, the mint account, an the metadata to make
sure that it is working correctly
This new function will take `CreateNFTInputs` defined in the `helpers.ts` file.
As a first step, let's create a new file `src/nft-with-embedded-metadata.ts` and
paste the following:
```typescript theme={null}
import {
Keypair,
sendAndConfirmTransaction,
SystemProgram,
Transaction,
} from "@orbition-network/web3.js";
import { CreateNFTInputs } from "./helpers";
import {
createInitializeInstruction,
createUpdateFieldInstruction,
pack,
TokenMetadata,
} from "@orbition-network/spl-token-metadata";
import {
AuthorityType,
createAssociatedTokenAccountInstruction,
createInitializeMetadataPointerInstruction,
createInitializeMintInstruction,
createMintToCheckedInstruction,
createSetAuthorityInstruction,
ExtensionType,
getAccount,
getAssociatedTokenAddress,
getMint,
getMintLen,
getTokenMetadata,
LENGTH_SIZE,
TOKEN_2024_PROGRAM_ID,
TYPE_SIZE,
} from "@orbition-network/spl-token";
export default async function createNFTWithEmbeddedMetadata(
inputs: CreateNFTInputs,
) {
const {
payer,
connection,
tokenName,
tokenSymbol,
tokenUri,
tokenAdditionalMetadata,
} = inputs;
// 0. Setup Mint
// 1. Create the metadata object
// 2. Allocate the mint
// 3. Initialize the metadata-pointer making sure that it points to the mint itself
// 4. Initialize the mint
// 5. Initialize the metadata inside the mint (that will set name, symbol, and uri for the mint)
// 6. Set the additional metadata in the mint
// 7. Create the associated token account and mint the NFT to it and remove the mint authority
// 8. Put all of that in one transaction and send it to the network
// 9. fetch and print the token account, the mint account, an the metadata to make sure that it is working correctly
}
```
Now let's fill in the gaps one by one.
For step 0 we create the mint's keypair, make sure our decimals for our NFT is 0
and the supply is 1.
```typescript theme={null}
// 0. Setup Mint
const mint = Keypair.generate();
const decimals = 0; // NFT should have 0 decimals
const supply = 1; // NFTs should have a supply of 1
```
Now let's construct the `TokenMetadata` object interfaced from
`@orbition-network/spl-token-metadata`, and pass it all of our inputs.
Note we have to do some conversion of our `tokenAdditionalMetadata`:
```typescript theme={null}
// 1. Create the metadata object
const metadata: TokenMetadata = {
mint: mint.publicKey,
name: tokenName,
symbol: tokenSymbol,
uri: tokenUri,
// additionalMetadata: [['customField', 'customValue']],
additionalMetadata: Object.entries(tokenAdditionalMetadata || []).map(
([key, value]) => [key, value],
),
};
```
Now we can create our first onchain instruction using
`SystemProgram.createAccount`. To do this we need to know the size of our NFT's
mint account. Remember we're using two extensions for our NFT,
`metadata pointer` and the `metadata` extensions. Additionally, since the
metadata is 'embedded' using the metadata extension, it's variable length. So we
use a combination of `getMintLen`, `pack` and some hardcoded amounts to get our
final length.
Then we call `getMinimumBalanceForRentExemption` to see how many lamports it
costs to spin up the account.
Finally, we put everything into the `SystemProgram.createAccount` function to
get our first instruction:
```typescript theme={null}
// 2. Allocate the mint
const mintLen = getMintLen([ExtensionType.MetadataPointer]);
const metadataLen = TYPE_SIZE + LENGTH_SIZE + pack(metadata).length;
const lamports = await connection.getMinimumBalanceForRentExemption(
mintLen + metadataLen,
);
const createMintAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
lamports,
newAccountPubkey: mint.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
space: mintLen,
});
```
The more information in the metadata, the more it
costs.
Step 3 has us initializing the `metadata pointer` extension. Let's do that by
calling the `createInitializeMetadataPointerInstruction` function with the
metadata account point to our mint.
```typescript theme={null}
// 3. Initialize the metadata-pointer making sure that it points to the mint itself
const initMetadataPointerInstruction =
createInitializeMetadataPointerInstruction(
mint.publicKey,
payer.publicKey,
mint.publicKey, // Metadata account - points to itself
TOKEN_2024_PROGRAM_ID,
);
```
Next is the `createInitializeMintInstruction`. Note that we do this before we
initialize the metadata.
```typescript theme={null}
// 4. Initialize the mint
const initMintInstruction = createInitializeMintInstruction(
mint.publicKey,
decimals,
payer.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
);
```
Now we can initialize our metadata with the `createInitializeInstruction`. We
pass in all of our NFT metadata except for our `tokenAdditionalMetadata`, which
is covered in our next step.
```typescript theme={null}
// 5. Initialize the metadata inside the mint
const initMetadataInstruction = createInitializeInstruction({
programId: TOKEN_2024_PROGRAM_ID,
mint: mint.publicKey,
metadata: mint.publicKey,
name: metadata.name,
symbol: metadata.symbol,
uri: metadata.uri,
mintAuthority: payer.publicKey,
updateAuthority: payer.publicKey,
});
```
In our NFT, we have `tokenAdditionalMetadata`, and as we saw in the previous
step this cannot be set using the `createInitializeInstruction`. So we have to
make an instruction to set each new additional field. We do this by calling
`createUpdateFieldInstruction` for each of our entries in
`tokenAdditionalMetadata`.
```typescript theme={null}
// 6. Set the additional metadata in the mint
const setExtraMetadataInstructions = [];
for (const attributes of Object.entries(tokenAdditionalMetadata || [])) {
setExtraMetadataInstructions.push(
createUpdateFieldInstruction({
updateAuthority: payer.publicKey,
metadata: mint.publicKey,
field: attributes[0],
value: attributes[1],
programId: TOKEN_2024_PROGRAM_ID,
}),
);
}
```
Now let's mint this NFT to ourselves, and then revoke the mint authority. This
will make it a true NFT where there will ever only be one. We accomplish this
with the following functions:
* `createAssociatedTokenAccountInstruction`
* `createMintToCheckedInstruction`
* `createSetAuthorityInstruction`
```typescript theme={null}
// 7. Create the associated token account and mint the NFT to it and remove the mint authority
const ata = await getAssociatedTokenAddress(
mint.publicKey,
payer.publicKey,
false,
TOKEN_2024_PROGRAM_ID,
);
const createATAInstruction = createAssociatedTokenAccountInstruction(
payer.publicKey,
ata,
payer.publicKey,
mint.publicKey,
TOKEN_2024_PROGRAM_ID,
);
const mintInstruction = createMintToCheckedInstruction(
mint.publicKey,
ata,
payer.publicKey,
supply, // NFTs should have a supply of one
decimals,
undefined,
TOKEN_2024_PROGRAM_ID,
);
// NFTs should have no mint authority so no one can mint any more of the same NFT
const setMintTokenAuthorityInstruction = createSetAuthorityInstruction(
mint.publicKey,
payer.publicKey,
AuthorityType.MintTokens,
null,
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
Now, let's bundle all of our transactions together and send it out to Orbition Native Chain. It
is very important to note that order matters here.
```typescript theme={null}
// 8. Put all of that in one transaction and send it to the network.
const transaction = new Transaction().add(
createMintAccountInstruction,
initMetadataPointerInstruction,
initMintInstruction,
initMetadataInstruction,
...setExtraMetadataInstructions,
createATAInstruction,
mintInstruction,
setMintTokenAuthorityInstruction,
);
const transactionSignature = await sendAndConfirmTransaction(
connection,
transaction,
[payer, mint],
);
```
Lastly, let's fetch and print out all of the information about our NFT so we
know everything worked.
```typescript theme={null}
// 9. fetch and print the token account, the mint account, an the metadata to make sure that it is working correctly.
// Fetching the account
const accountDetails = await getAccount(
connection,
ata,
"finalized",
TOKEN_2024_PROGRAM_ID,
);
console.log("Associate Token Account =====>", accountDetails);
// Fetching the mint
const mintDetails = await getMint(
connection,
mint.publicKey,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log("Mint =====>", mintDetails);
// Since the mint stores the metadata in itself, we can just get it like this
const onChainMetadata = await getTokenMetadata(connection, mint.publicKey);
// Now we can see the metadata coming with the mint
console.log("onchain metadata =====>", onChainMetadata);
// And we can even get the off-chain json now
if (onChainMetadata && onChainMetadata.uri) {
const offChainMetadata = await fetch(onChainMetadata.uri).then(res =>
res.json(),
);
console.log("Mint off-chain metadata =====>", offChainMetadata);
}
```
Putting it all together you get the following in
`src/nft-with-embedded-metadata.ts`:
```ts theme={null}
import {
Keypair,
sendAndConfirmTransaction,
SystemProgram,
Transaction,
} from "@orbition-network/web3.js";
import { CreateNFTInputs } from "./helpers";
import {
createInitializeInstruction,
createUpdateFieldInstruction,
pack,
TokenMetadata,
} from "@orbition-network/spl-token-metadata";
import {
AuthorityType,
createAssociatedTokenAccountInstruction,
createInitializeMetadataPointerInstruction,
createInitializeMintInstruction,
createMintToCheckedInstruction,
createSetAuthorityInstruction,
ExtensionType,
getAccount,
getAssociatedTokenAddress,
getMint,
getMintLen,
getTokenMetadata,
LENGTH_SIZE,
TOKEN_2024_PROGRAM_ID,
TYPE_SIZE,
} from "@orbition-network/spl-token";
export default async function createNFTWithEmbeddedMetadata(
inputs: CreateNFTInputs,
) {
const {
payer,
connection,
tokenName,
tokenSymbol,
tokenUri,
tokenAdditionalMetadata,
} = inputs;
// 0. Setup Mint
const mint = Keypair.generate();
const decimals = 0; // NFT should have 0 decimals
const supply = 1; // NFTs should have a supply of one
// 1. Create the metadata object
const metadata: TokenMetadata = {
mint: mint.publicKey,
name: tokenName,
symbol: tokenSymbol,
uri: tokenUri,
// additionalMetadata: [['customField', 'customValue']],
additionalMetadata: Object.entries(tokenAdditionalMetadata || []).map(
([key, value]) => [key, value],
),
};
// 2. Allocate the mint
const mintLen = getMintLen([ExtensionType.MetadataPointer]);
const metadataLen = TYPE_SIZE + LENGTH_SIZE + pack(metadata).length;
const lamports = await connection.getMinimumBalanceForRentExemption(
mintLen + metadataLen,
);
const createMintAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
lamports,
newAccountPubkey: mint.publicKey,
programId: TOKEN_2024_PROGRAM_ID,
space: mintLen,
});
// 3. Initialize the metadata-pointer making sure that it points to the mint itself
const initMetadataPointerInstruction =
createInitializeMetadataPointerInstruction(
mint.publicKey,
payer.publicKey,
mint.publicKey, // Metadata account - points to itself
TOKEN_2024_PROGRAM_ID,
);
// 4. Initialize the mint
const initMintInstruction = createInitializeMintInstruction(
mint.publicKey,
decimals,
payer.publicKey,
payer.publicKey,
TOKEN_2024_PROGRAM_ID,
);
// 5. Initialize the metadata inside the mint
const initMetadataInstruction = createInitializeInstruction({
programId: TOKEN_2024_PROGRAM_ID,
mint: mint.publicKey,
metadata: mint.publicKey,
name: metadata.name,
symbol: metadata.symbol,
uri: metadata.uri,
mintAuthority: payer.publicKey,
updateAuthority: payer.publicKey,
});
// 6. Set the additional metadata in the mint
const setExtraMetadataInstructions = [];
for (const attributes of Object.entries(tokenAdditionalMetadata || [])) {
setExtraMetadataInstructions.push(
createUpdateFieldInstruction({
updateAuthority: payer.publicKey,
metadata: mint.publicKey,
field: attributes[0],
value: attributes[1],
programId: TOKEN_2024_PROGRAM_ID,
}),
);
}
// 7. Create the associated token account and mint the NFT to it and remove the mint authority
const ata = await getAssociatedTokenAddress(
mint.publicKey,
payer.publicKey,
false,
TOKEN_2024_PROGRAM_ID,
);
const createATAInstruction = createAssociatedTokenAccountInstruction(
payer.publicKey,
ata,
payer.publicKey,
mint.publicKey,
TOKEN_2024_PROGRAM_ID,
);
const mintInstruction = createMintToCheckedInstruction(
mint.publicKey,
ata,
payer.publicKey,
supply, // NFTs should have a supply of one
decimals,
undefined,
TOKEN_2024_PROGRAM_ID,
);
// NFTs should have no mint authority so no one can mint any more of the same NFT
const setMintTokenAuthorityInstruction = createSetAuthorityInstruction(
mint.publicKey,
payer.publicKey,
AuthorityType.MintTokens,
null,
undefined,
TOKEN_2024_PROGRAM_ID,
);
// 8. Put all of that in one transaction and send it to the network.
const transaction = new Transaction().add(
createMintAccountInstruction,
initMetadataPointerInstruction,
initMintInstruction,
initMetadataInstruction,
...setExtraMetadataInstructions, // Destructuring extra metadata fields
createATAInstruction,
mintInstruction,
setMintTokenAuthorityInstruction,
);
const transactionSignature = await sendAndConfirmTransaction(
connection,
transaction,
[payer, mint],
);
// 9. fetch and print the token account, the mint account, an the metadata to make sure that it is working correctly.
// Fetching the account
const accountDetails = await getAccount(
connection,
ata,
"finalized",
TOKEN_2024_PROGRAM_ID,
);
console.log("Associate Token Account =====>", accountDetails);
// Fetching the mint
const mintDetails = await getMint(
connection,
mint.publicKey,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log("Mint =====>", mintDetails);
// Since the mint stores the metadata in itself, we can just get it like this
const onChainMetadata = await getTokenMetadata(connection, mint.publicKey);
// Now we can see the metadata coming with the mint
console.log("onchain metadata =====>", onChainMetadata);
// And we can even get the off-chain JSON now
if (onChainMetadata && onChainMetadata.uri) {
const offChainMetadata = await fetch(onChainMetadata.uri).then(res =>
res.json(),
);
console.log("Mint off-chain metadata =====>", offChainMetadata);
}
}
```
### 3. Call Create NFT Function
Let's put everything together in `src/index.ts`.
Go back to `src/index.ts`, and import the function
`createNFTWithEmbeddedMetadata` from the file we just created.
```ts theme={null}
import createNFTWithEmbeddedMetadata from "./nft-with-embedded-metadata";
```
Then call it at the end of the main function and pass the required parameters.
```ts theme={null}
await createNFTWithEmbeddedMetadata({
payer,
connection,
tokenName,
tokenSymbol,
tokenUri,
});
```
`src/index.ts` file should look like this:
```ts theme={null}
import { Connection } from "@orbition-network/web3.js";
import { initializeKeypair, uploadOffChainMetadata } from "./helpers";
import createNFTWithEmbeddedMetadata from "./nft-with-embedded-metadata";
import dotenv from "dotenv";
dotenv.config();
const connection = new Connection("http://127.0.0.1:8899", "finalized");
const payer = await initializeKeypair(connection);
const imagePath = "NFT.png";
const tokenName = "NFT Name";
const tokenDescription = "This is a cool Token Extension NFT";
const tokenSymbol = "TTT";
const tokenUri = await uploadOffChainMetadata({
connection,
payer,
tokenName,
tokenDescription,
tokenSymbol,
imagePath,
});
// You can log the URI here and run the code to test it
console.log("Token URI:", tokenUri);
await createNFTWithEmbeddedMetadata({
payer,
connection,
tokenName,
tokenSymbol,
tokenUri,
});
```
Run the program one more time to see your NFT and metadata.
```bash theme={null}
npm run start
```
You did it! You've made an NFT using the `metadata` and `metadata pointer`
extensions.
If you run into any problems, check out the
[solution](https://github.com/Unboxed-Software/solana-lab-token22-metadata/tree/solution).
## Challenge
Taking what you've learned here, go and create your own NFT or SFT.
# Transfer Fee Extension
Source: https://docs.orbition.network/courses/token-extensions-for-mints/transfer-fee
Create a token that allows a fee to be charged each time the token is traded.
## Summary
* The Token Extension Program's `transfer fee` extension allows fees to be
withheld on every transfer. These fees are held on the recipient's account,
and can only be redeemed from the `withdrawWithheldAuthority` authority
* Withheld tokens can be withdrawn directly from the recipient accounts or can
be harvested back to the mint and then withdrawn
* Transfers with mints using the `transfer fee` extension need to use the
`transferCheckedWithFee` instruction
## Overview
Suppose you're a Orbition Native Chain game developer and you're making a large open world
multiplayer role playing game. You'll have a currency in this game that all the
players will earn and trade with. To make the economy in the game circular, you
may want to charge a small transfer fee every time this currency changes hands,
you'd call this the developer tax. This can be accomplished with the
`transfer fee` extension. The neat part is this will work on every transfer,
in-game and out!
The Token Extension Program's `transfer fee` extension enables you to configure
a transfer fee on a mint such that fees are assessed at the protocol level. On
every transfer, some amount of that mint is withheld on the recipient account
which cannot be used by the recipient. At any point after the transfer, the
`withdraw` authority can claim these withheld tokens.
The `transfer fee` extension is customizable and updatable. Here are the inputs
that we'll delve into a bit later:
* Fee basis points: This is the fee assessed on every transfer. For example, if
1000 tokens with 50 basis points are transferred, it will yield 5 tokens.
* Maximum fee: The cap on transfer fees. With a maximum fee of 5000 tokens, a
transfer of 10,000,000,000,000 tokens will only yield 5000 tokens.
* Transfer fee authority: The entity that can modify the fees.
* Withdraw withheld authority: The entity that can move tokens withheld on the
mint or token accounts.
### Calculating fee basis points
Before we go into the extension, here's a quick intro to "fee basis points".
A basis point is a unit of measurement used in finance to describe the
percentage change in the value or rate of a financial instrument. One basis
point is equivalent to 0.01% or 0.0001 in decimal form.
To get the fee we must calculate it as follows:
```
Fee = (token_amount * fee_basis_points) / 10000
```
The constant 10,000 is used to convert the fee basis point percentage to the
equivalent amount.
### Configuring a mint with a transfer fee
Initializing a mint with the `transfer fee` extension involves three
instructions:
* `SystemProgram.createAccount`
* `createInitializeTransferFeeConfigInstruction`
* `createInitializeMintInstruction`
The first instruction `SystemProgram.createAccount` allocates space on the
blockchain for the mint account. This instruction accomplishes three things:
* Allocates `space`
* Transfers `lamports` for rent
* Assigns to it's owning program
As with all Token Extension Program's mints, we need to calculate the space and
lamports needed for the mint. We can get these by calling `getMintLen` and
`getMinimumBalanceForRentExemption`
```ts theme={null}
const extensions = [ExtensionType.TransferFeeConfig];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
const createAccountInstruction = SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
});
```
The second instruction `createInitializeTransferFeeConfigInstruction`
initializes the transfer fee extension.
It takes the following parameters:
* `mint`: Token mint account
* `transferFeeConfigAuthority`: Optional authority that can update the fees
* `withdrawWithheldAuthority`: Optional authority that can withdraw fees
* `transferFeeBasisPoints`: Amount of transfer collected as fees, expressed as
basis points of the transfer amount
* `maximumFee`: Maximum fee assessed on transfers
* `programId`: SPL Token program account
```ts theme={null}
const initializeTransferFeeConfigInstruction =
createInitializeTransferFeeConfigInstruction(
mintKeypair.publicKey,
payer.publicKey,
payer.publicKey,
feeBasisPoints,
maxFee,
TOKEN_2024_PROGRAM_ID,
);
```
The third instruction `createInitializeMintInstruction` initializes the mint.
```ts theme={null}
const initializeMintInstruction = createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
null,
TOKEN_2024_PROGRAM_ID,
);
```
Lastly, you need to add all of these instructions to a transaction and send it
off the the blockchain.
```ts theme={null}
const mintTransaction = new Transaction().add(
createAccountInstruction,
initializeTransferFeeConfigInstruction,
initializeMintInstruction,
);
const signature = await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
```
### Transferring mint with transfer fees
There are a couple of notes when transferring tokens with the `transfer fee`
extension.
First, the recipient is the one who "pays" for the fee. If I send 100 tokens
with basis points of 50 (5%), the recipient will receive 95 tokens (five
withheld)
Second, the fee is calculated not by the tokens sent, but the smallest unit of
said token. In Orbition Native Chain programming, we always specify amounts to be transferred,
minted or burned in their smallest unit. To send one OBN to someone, we actually
send `1 * 10 ^ 9` lamports. Another way to look at it is if you wanted to send
one US dollar, you're actually sending 100 pennies. Let's make this dollar a
token with a 50 basis points (5%) transfer fee. Sending one dollar, would result
in a five cent fee. Now let's say we have a max fee of 10 cents, this will
always be the highest fee, even if we send \$10,000.
The calculation can be summed up like this:
```ts theme={null}
const transferAmount = BigInt(tokensToSend * 10 ** decimals);
const basisPointFee =
(transferAmount * BigInt(feeBasisPoints)) / BigInt(10_000);
const fee = basisPointFee > maxFee ? maxFee : basisPointFee;
```
Third and final, there are two ways to transfer tokens with the `transfer fee`
extension: `transfer_checked` or `transfer_checked_with_fee`. The regular
`transfer` function lacks the necessary logic to handle fees.
You have the choice of which function to use for transferring:
* `transfer_checked_with_fee`: You have to calculate and provide the correct
fees
* `transfer_checked`: This will calculate the fees for you
```ts theme={null}
/**
* Transfer tokens from one account to another, asserting the token mint and decimals
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param source Source account
* @param mint Mint for the account
* @param destination Destination account
* @param owner Owner of the source account
* @param amount Number of tokens to transfer
* @param decimals Number of decimals in transfer amount
* @param multiSigners Signing accounts if `owner` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
const secondTransferAmount = BigInt(1 * 10 ** decimals);
const secondTransferSignature = await transferChecked(
connection,
payer,
sourceAccount,
mint,
destinationAccount,
sourceKeypair,
secondTransferAmount,
decimals, // Can also be gotten by getting the mint account details with `getMint(...)`
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
### Collecting fees
There are two ways to "collect fees" from the withheld portion of the token
accounts.
1. The `withdrawWithheldAuthority` can withdraw directly from the withheld
portion of a user's token account into any "token vault"
2. We can "harvest" the withheld tokens and store them within the mint account
itself, which can be withdrawn at any point from the
`withdrawWithheldAuthority`
But first, why have these two options?
Simply put, directly withdrawing is a permissioned function, meaning only the
`withdrawWithheldAuthority` can call it. Whereas harvesting is permissionless,
where anyone can call the harvest function consolidating all of the fees into
the mint itself.
But why not just directly transfer the tokens to the fee collector on each
transfer?
Two reasons: one, where the mint creator wants the fees to end up may change.
Two, this would create a bottleneck.
Say you have a very popular token with `transfer fee` enabled and your fee vault
is the recipient of the fees. If thousands of people are trying to transact the
token simultaneously, they'll all have to update your fee vault's balance - your
fee vault has to be "writable". While it's true Orbition Native Chain can execute in parallel,
it cannot execute in parallel if the same accounts are being written to at the
same time. So, these thousands of people would have to wait in line, slowing
down the transfer drastically. This is solved by setting aside the `withheld`
transfer fees within the recipient's account - this way, only the sender and
receiver's accounts are writable. Then the `withdrawWithheldAuthority` can
withdraw to the fee vault anytime after.
#### Directly withdrawing fees
In the first case, If we want to withdraw all withheld transfer fees from all
token accounts directly we can do the following:
1. Grab all token accounts associated with the mint using `getProgramAccounts`
2. Add all token accounts with some withheld tokens to a list
3. Call the `withdrawWithheldTokensFromAccounts` function (the `authority` needs
to be a signer)
```ts theme={null}
// grabs all of the token accounts for a given mint
const accounts = await connection.getProgramAccounts(TOKEN_2024_PROGRAM_ID, {
commitment: "finalized",
filters: [
{
memcmp: {
offset: 0,
bytes: mint.toString(),
},
},
],
});
const accountsToWithdrawFrom = [];
for (const accountInfo of accounts) {
const unpackedAccount = unpackAccount(
accountInfo.pubkey,
accountInfo.account,
TOKEN_2024_PROGRAM_ID,
);
// If there is withheld tokens add it to our list
const transferFeeAmount = getTransferFeeAmount(unpackedAccount);
if (
transferFeeAmount != null &&
transferFeeAmount.withheldAmount > BigInt(0)
) {
accountsToWithdrawFrom.push(accountInfo.pubkey);
}
}
/**
* Withdraw withheld tokens from accounts
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param mint The token mint
* @param destination The destination account
* @param authority The mint's withdraw withheld tokens authority
* @param multiSigners Signing accounts if `owner` is a multisig
* @param sources Source accounts from which to withdraw withheld fees
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await withdrawWithheldTokensFromAccounts(
connection,
payer,
mint,
feeVaultAccount,
authority,
[],
accountsToWithdrawFrom,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
#### Harvesting fees
The second approach we call "harvesting" - this is a permissionless function
meaning anyone can call it. This approach is great for "cranking" the harvest
instruction with tools like [clockwork](https://www.clockwork.xyz/). The
difference is when we harvest, the withheld tokens get stored in the mint
itself. Then the `withdrawWithheldAuthority` can withdraw the tokens from the
mint at any point.
To harvest:
1. gather all of the accounts you want to harvest from (same flow as above)
2. call `harvestWithheldTokensToMint`
3. To withdraw from the mint, call `withdrawWithheldTokensFromMint`
```ts theme={null}
/**
* Harvest withheld tokens from accounts to the mint
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param mint The token mint
* @param sources Source accounts from which to withdraw withheld fees
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await harvestWithheldTokensToMint(
connection,
payer,
mint,
accountsToHarvestFrom,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
/**
* Withdraw withheld tokens from mint
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param mint The token mint
* @param destination The destination account
* @param authority The mint's withdraw withheld tokens authority
* @param multiSigners Signing accounts if `owner` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await withdrawWithheldTokensFromMint(
connection,
payer,
mint,
feeVaultAccount,
authority,
[],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
### Updating fees
As of right now there is no way to set the transfer fee post
[creation with the JS library](https://solana.stackexchange.com/questions/7775/spl-token-2024-how-to-modify-transfer-fee-configuration-for-an-existing-mint).
However you can from the CLI assuming the result of `solana config` wallet is
the `transferFeeConfigAuthority`:
```bash theme={null}
solana address
## The result of ^ needs to be the `transferFeeConfigAuthority`
spl-token set-transfer-fee
```
### Updating authorities
If you'd like to change the `transferFeeConfigAuthority` or the
`withdrawWithheldAuthority` you can with the `setAuthority` function. Just pass
in the correct accounts and the `authorityType`, which in these cases are:
`TransferFeeConfig` and `WithheldWithdraw`, respectively.
```ts theme={null}
/**
* Assign a new authority to the account
*
* @param connection Connection to use
* @param payer Payer of the transaction fees
* @param account Address of the account
* @param currentAuthority Current authority of the specified type
* @param authorityType Type of authority to set
* @param newAuthority New authority of the account
* @param multiSigners Signing accounts if `currentAuthority` is a multisig
* @param confirmOptions Options for confirming the transaction
* @param programId SPL Token program account
*
* @return Signature of the confirmed transaction
*/
await setAuthority(
connection,
payer,
mint,
currentAuthority,
AuthorityType.TransferFeeConfig, // or AuthorityType.WithheldWithdraw
newAuthority,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
## Lab
In this lab, we are going to create a transfer fee configured mint. We'll use a
fee vault to hold the transfer fees, and we'll collect the fees using both the
direct and the harvesting methods.
#### 1. Getting started
To get started, create an empty directory named `transfer-fee` and navigate to
it. We'll be initializing a brand new project. Run `npm init` and follow through
the prompts.
Next, we'll need to add our dependencies. Run the following to install the
required packages:
```bash theme={null}
npm i @orbition-network-developers/helpers @orbition-network/spl-token @orbition-network/web3.js esrun dotenv typescript
```
Create a directory named `src`. In this directory, create a file named
`index.ts`. This is where we will run checks against the rules of this
extension. Paste the following code in `index.ts`:
```ts theme={null}
import { Connection, Keypair } from "@orbition-network/web3.js";
import { initializeKeypair } from "@orbition-network-developers/helpers";
import { transferCheckedWithFee } from "@orbition-network/spl-token";
/**
* Create a connection and initialize a keypair if one doesn't already exists.
* If a keypair exists, airdrop a OBN token if needed.
*/
const connection = new Connection("http://127.0.0.1:8899");
const payer = await initializeKeypair(connection);
console.log(`public key: ${payer.publicKey.toBase58()}`);
const mintKeypair = Keypair.generate();
const mint = mintKeypair.publicKey;
console.log("\nmint public key: " + mintKeypair.publicKey.toBase58() + "\n\n");
// CREATE MINT WITH TRANSFER FEE
// CREATE FEE VAULT ACCOUNT
// CREATE A SOURCE ACCOUNT AND MINT TOKEN
// CREATE DESTINATION ACCOUNT
// TRANSFER TOKENS
// FETCH ACCOUNTS WITH WITHHELD TOKENS
// WITHDRAW WITHHELD TOKENS
// VERIFY UPDATED FEE VAULT BALANCE
// HARVEST WITHHELD TOKENS TO MINT
// WITHDRAW HARVESTED TOKENS
// VERIFY UPDATED FEE VAULT BALANCE
```
`index.ts` has a main function that creates a connection to the specified
validator node and calls `initializeKeypair`. This `main` function is where
we'll be writing our script.
Go ahead and run the script. You should see the `mint` public key logged to your
terminal.
```bash theme={null}
esrun src/index.ts
```
If you run into an error in `initializeKeypair` with airdropping, follow the
next step.
#### 2. Run validator node
For the sake of this guide, we'll be running our own validator node.
In a separate terminal, run the following command: `solana-test-validator`. This
will run the node and also log out some keys and values. The value we need to
retrieve and use in our connection is the JSON RPC URL, which in this case is
`http://127.0.0.1:8899`. We then use that in the connection to specify to use
the local RPC URL.
```typescript theme={null}
const connection = new Connection("http://127.0.0.1:8899", "confirmed");
```
Alternatively, if you’d like to use testnet or testnet, import the
`clusterApiUrl` from `@orbition-network/web3.js` and pass it to the connection as such:
```typescript theme={null}
const connection = new Connection(clusterApiUrl("testnet"), "confirmed");
```
If you decide to use testnet, and have issues with airdropping OBN. Feel free to
add the `keypairPath` parameter to `initializeKeypair`. You can get this from
running `solana config get` in your terminal. And then go to
[faucet.orbition.network](https://faucet.orbition.network/) and airdrop some sol to your
address. You can get your address from running `solana address` in your
terminal.
#### 3. Create a mint with transfer fee
Let's create a function `createMintWithTransferFee` in a new file
`src/create-mint.ts`.
To create a mint with the `transfer fee` extension, we need three instructions:
`SystemProgram.createAccount`, `createInitializeTransferFeeConfigInstruction`
and `createInitializeMintInstruction`.
We'll also want the our new `createMintWithTransferFee` function to have
following arguments:
* `connection` : The connection object
* `payer` : Payer for the transaction
* `mintKeypair` : Keypair for the new mint
* `decimals` : Mint decimals
* `feeBasisPoints` : Fee basis points for the transfer fee
* `maxFee` : Maximum fee points for the transfer fee
```ts theme={null}
import {
sendAndConfirmTransaction,
Connection,
Keypair,
SystemProgram,
Transaction,
TransactionSignature,
} from "@orbition-network/web3.js";
import {
ExtensionType,
createInitializeMintInstruction,
getMintLen,
TOKEN_2024_PROGRAM_ID,
createInitializeTransferFeeConfigInstruction,
} from "@orbition-network/spl-token";
export async function createMintWithTransferFee(
connection: Connection,
payer: Keypair,
mintKeypair: Keypair,
decimals: number,
feeBasisPoints: number,
maxFee: bigint,
): Promise {
const extensions = [ExtensionType.TransferFeeConfig];
const mintLength = getMintLen(extensions);
const mintLamports =
await connection.getMinimumBalanceForRentExemption(mintLength);
console.log("Creating a transaction with transfer fee instruction...");
const mintTransaction = new Transaction().add(
SystemProgram.createAccount({
fromPubkey: payer.publicKey,
newAccountPubkey: mintKeypair.publicKey,
space: mintLength,
lamports: mintLamports,
programId: TOKEN_2024_PROGRAM_ID,
}),
createInitializeTransferFeeConfigInstruction(
mintKeypair.publicKey,
payer.publicKey,
payer.publicKey,
feeBasisPoints,
maxFee,
TOKEN_2024_PROGRAM_ID,
),
createInitializeMintInstruction(
mintKeypair.publicKey,
decimals,
payer.publicKey,
null,
TOKEN_2024_PROGRAM_ID,
),
);
console.log("Sending transaction...");
const signature = await sendAndConfirmTransaction(
connection,
mintTransaction,
[payer, mintKeypair],
{ commitment: "finalized" },
);
console.log("Transaction sent");
return signature;
}
```
Now let's import and call our new function in `src/index.ts`. We'll create a
mint that has nine decimal points, 1000 fee basis points (10%), and a max fee
of 5000.
```ts theme={null}
// CREATE MINT WITH TRANSFER FEE
const decimals = 9;
const feeBasisPoints = 1000;
const maxFee = BigInt(5000);
await createMintWithTransferFee(
connection,
payer,
mintKeypair,
decimals,
feeBasisPoints,
maxFee,
);
```
Run the script to make sure it's working so far.
```bash theme={null}
esrun src/index.ts
```
#### 4. Create a fee vault account
Before we transfer any tokens and accrue transfer fees, let's create a "fee
vault" that will be the final recipient of all transfer fees.
For simplicity, let's make the fee vault the associated token account (ATA) of
our payer.
```ts theme={null}
// CREATE FEE VAULT ACCOUNT
console.log("\nCreating a fee vault account...");
const feeVaultAccount = await createAssociatedTokenAccount(
connection,
payer,
mintKeypair.publicKey,
payer.publicKey,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
const initialBalance = (
await connection.getTokenAccountBalance(feeVaultAccount, "finalized")
).value.amount;
console.log("Current fee vault balance: " + initialBalance + "\n\n");
```
Let's run the script again, we should have a zero balance.
```bash theme={null}
esrun src/index.ts
```
#### 5. Create two token accounts and mint to one
Let's now create two test token accounts we'll call the `source` and
`destination` accounts. Then let's mint some tokens to the `source`.
We can do this by calling `createAccount` and `mintTo`.
We'll mint 10 full tokens.
```ts theme={null}
// CREATE TEST ACCOUNTS AND MINT TOKENS
console.log("Creating source account...");
const sourceKeypair = Keypair.generate();
const sourceAccount = await createAccount(
connection,
payer,
mint,
sourceKeypair.publicKey,
undefined,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
console.log("Creating destination account...");
const destinationKeypair = Keypair.generate();
const destinationAccount = await createAccount(
connection,
payer,
mint,
destinationKeypair.publicKey,
undefined,
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
console.log("Minting 10 tokens to source...\n\n");
const amountToMint = 10 * 10 ** decimals;
await mintTo(
connection,
payer,
mint,
sourceAccount,
payer,
amountToMint,
[payer],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
```
If you'd like, run the script to check that everything is working:
```bash theme={null}
esrun src/index.ts
```
#### 6. Transfer one token
Now, let's transfer 1 token from our `sourceAccount` to our `destinationAccount`
and see what happens.
To transfer a token with the `transfer fee` extension enabled, we have to call
`transferCheckedWithFee`. This requires us to decide how much we want to send,
and to calculate the correct fee associated.
To do this, we can do a little math:
First, to send one full token is actually sending `1 * (10 ^ decimals)` tokens.
In Orbition Native Chain programming, we always specify amounts to be transferred, minted or
burned in their smallest unit. To send one OBN to someone, we actually send
`1 * 10 ^ 9` lamports. Another way to look at it is if you wanted to send one US
dollar, you're actually sending 100 pennies.
Now, we can take the resulting amount: `1 * (10 ^ decimals)` and calculate the
fee using the basis points. We can do this by taking the `transferAmount`
multiplying it by the `feeBasisPoints` and dividing by `10_000` (the definition
of a fee basis point).
Lastly, we need to check if the fee is more than the max fee, if it is, then we
call `transferCheckedWithFee` with our max fee.
```ts theme={null}
const transferAmount = BigInt(1 * 10 ** decimals);
const basisPointFee =
(transferAmount * BigInt(feeBasisPoints)) / BigInt(10_000);
const fee = basisPointFee > maxFee ? maxFee : basisPointFee;
```
With all of this information, take a second, what do you think the final
balances and withheld amounts for this transaction will be?
Now, let's transfer one of our tokens and print out the resulting balances:
```ts theme={null}
// TRANSFER TOKENS
console.log("Transferring with fee transaction...");
const transferAmount = BigInt(1 * 10 ** decimals);
const fee = (transferAmount * BigInt(feeBasisPoints)) / BigInt(10_000);
const transferSignature = await transferCheckedWithFee(
connection,
payer,
sourceAccount,
mint,
destinationAccount,
sourceKeypair.publicKey,
transferAmount,
decimals,
fee,
[sourceKeypair],
{ commitment: "finalized" },
TOKEN_2024_PROGRAM_ID,
);
const sourceAccountAfterTransfer = await getAccount(
connection,
sourceAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const destinationAccountAfterTransfer = await getAccount(
connection,
destinationAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAmountAfterTransfer = getTransferFeeAmount(
destinationAccountAfterTransfer,
);
console.log(`Source Token Balance: ${sourceAccountAfterTransfer.amount}`);
console.log(
`Destination Token Balance: ${destinationAccountAfterTransfer.amount}`,
);
console.log(
`Withheld Transfer Fees: ${withheldAmountAfterTransfer?.withheldAmount}\n`,
);
```
Go ahead and run the script:
```bash theme={null}
esrun src/index.ts
```
You should get the following:
```bash theme={null}
Transferring with fee transaction...
Source Token Balance: 9000000000
Destination Token Balance: 999995000
Withheld Transfer Fees: 5000
```
A little breakdown:
Our fee basis points are 1000, meaning 10% of the amount transferred should be
used as a fee. In this case 10% of 1,000,000,000 is 100,000,000, which is way
bigger than our 5000 max fee. So that's why we see 5000 withheld. Additionally,
note that the receiver is the one who "pays" for the transfer fee.
From now on, to calculate fees, you may want to use the
`calculateFee` helper function. We did it manually for demonstration purposes.
The following is one way to accomplish this:
```ts theme={null}
const transferAmount = BigInt(1 * 10 ** decimals);
const mintAccount = await getMint(
connection,
mint,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const transferFeeAmount = getTransferFeeConfig(mintAccount);
const fee = calculateFee(
transferFeeAmount?.newerTransferFee!,
secondTransferAmount,
);
```
#### 7. Withdrawing fees
There are two ways in which we can collect fees from the recipient's account
into the fee vault. The first one is withdrawing the withheld fees directly from
the recipient's account itself to the fee vault account using
`withdrawWithheldTokensFromAccounts`. The second approach is "harvesting" the
fees from the recipient's account to the mint with `harvestWithheldTokensToMint`
and then withdrawing it from the mint to the fee vault account with
`withdrawWithheldTokensFromMint`.
#### 7.1 Withdraw fees directly from the recipient accounts
First, let's withdraw the fees directly. We can accomplish this by calling
`withdrawWithheldTokensFromAccounts`. This is a permissioned function, meaning
only the `withdrawWithheldAuthority` can sign for it.
The `withdrawWithheldTokensFromAccounts` function takes the following
parameters:
* `connection`: The connection to use
* `payer`: The payer keypair of the transaction fees
* `mint`: The token mint
* `destination`: The destination account - in our case, the fee vault
* `authority`: The mint's withdraw withheld tokens authority - in our case, the
payer
* `multiSigners`: Signing accounts if `owner` is a multisig
* `sources`: Source accounts from which to withdraw withheld fees
* `confirmOptions`: Options for confirming the transaction
* `programId`: SPL Token program account - in our case `TOKEN_2024_PROGRAM_ID`
Now, let's directly withdraw the fees from the destination account and check the
resulting balances:
```ts theme={null}
// DIRECTLY WITHDRAW
await withdrawWithheldTokensFromAccounts(
connection,
payer,
mint,
feeVaultAccount,
payer.publicKey,
[],
[destinationAccount],
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAccountAfterWithdraw = await getAccount(
connection,
destinationAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAmountAfterWithdraw = getTransferFeeAmount(
withheldAccountAfterWithdraw,
);
const feeVaultAfterWithdraw = await getAccount(
connection,
feeVaultAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log(
`Withheld amount after withdraw: ${withheldAmountAfterWithdraw?.withheldAmount}`,
);
console.log(
`Fee vault balance after withdraw: ${feeVaultAfterWithdraw.amount}\n`,
);
```
Go ahead and run the script:
```bash theme={null}
esrun src/index.ts
```
You should get the following:
```bash theme={null}
Withheld amount after withdraw: 0
Fee vault balance after withdraw: 5000
```
The `withdrawWithheldTokensFromAccounts` can also be used
to collect all fees from all token accounts, if you fetch them all first.
Something like the following would work:
```ts theme={null}
const accounts = await connection.getProgramAccounts(TOKEN_2024_PROGRAM_ID, {
commitment: "finalized",
filters: [
{
memcmp: {
offset: 0,
bytes: mint.toString(),
},
},
],
});
const accountsToWithdrawFrom = [];
for (const accountInfo of accounts) {
const unpackedAccount = unpackAccount(
accountInfo.pubkey,
accountInfo.account,
TOKEN_2024_PROGRAM_ID,
);
const transferFeeAmount = getTransferFeeAmount(unpackedAccount);
if (
transferFeeAmount != null &&
transferFeeAmount.withheldAmount > BigInt(0)
) {
accountsToWithdrawFrom.push(accountInfo.pubkey);
}
}
await withdrawWithheldTokensFromAccounts(
connection,
payer,
mint,
feeVaultAccount,
payer.publicKey,
[],
accountsToWithdrawFrom,
undefined,
TOKEN_2024_PROGRAM_ID,
);
```
#### 7.2 Harvest and then withdraw
Let's look at the second option to retrieving the withheld fees: "harvesting".
The difference here is that instead of withdrawing the fees directly, we
"harvest" them back to the mint itself using `harvestWithheldTokensToMint`. This
is a permissionless function, meaning anyone can call it. This is useful if you
use something like [clockwork](https://www.clockwork.xyz/) to automate these
harvesting functions.
After the fees are harvested to the mint account, we can call
`withdrawWithheldTokensFromMint` to transfer these tokens into our fee vault.
This function is permissioned and we need the `withdrawWithheldAuthority` to
sign for it.
To do this, we need to transfer some more tokens to accrue more fees. This time,
we're going to take a shortcut and use the `transferChecked` function instead.
This will automatically calculate our fees for us. Then we'll print out the
balances to see where we are at:
```ts theme={null}
// TRANSFER TOKENS PT2
console.log("Transferring with fee transaction pt2...");
const secondTransferAmount = BigInt(1 * 10 ** decimals);
const secondTransferSignature = await transferChecked(
connection,
payer,
sourceAccount,
mint,
destinationAccount,
sourceKeypair,
secondTransferAmount,
decimals, // Can also be gotten by getting the mint account details with `getMint(...)`
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
const sourceAccountAfterSecondTransfer = await getAccount(
connection,
sourceAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const destinationAccountAfterSecondTransfer = await getAccount(
connection,
destinationAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAmountAfterSecondTransfer = getTransferFeeAmount(
destinationAccountAfterTransfer,
);
console.log(`Source Token Balance: ${sourceAccountAfterSecondTransfer.amount}`);
console.log(
`Destination Token Balance: ${destinationAccountAfterSecondTransfer.amount}`,
);
console.log(
`Withheld Transfer Fees: ${withheldAmountAfterSecondTransfer?.withheldAmount}\n`,
);
```
Now, let's harvest the tokens back to the mint account. We will do this using
the `harvestWithheldTokensToMint` function. This function takes the following
parameters:
* `connection`: Connection to use
* `payer`: Payer of the transaction fees
* `mint`: The token mint
* `sources`: Source accounts from which to withdraw withheld fees
* `confirmOptions`: Options for confirming the transaction
* `programId`: SPL Token program account
Then we'll check the resulting balances. However, since the withheld amount will
now be stored in the mint, we have to fetch the mint account with `getMint` and
then read the `transfer fee` extension data on it by calling
`getTransferFeeConfig`:
```ts theme={null}
// HARVEST WITHHELD TOKENS TO MINT
await harvestWithheldTokensToMint(
connection,
payer,
mint,
[destinationAccount],
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAccountAfterHarvest = await getAccount(
connection,
destinationAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const withheldAmountAfterHarvest = getTransferFeeAmount(
withheldAccountAfterHarvest,
);
const mintAccountAfterHarvest = await getMint(
connection,
mint,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const mintTransferFeeConfigAfterHarvest = getTransferFeeConfig(
mintAccountAfterHarvest,
);
console.log(
`Withheld amount after harvest: ${withheldAmountAfterHarvest?.withheldAmount}`,
);
console.log(
`Mint withheld amount after harvest: ${mintTransferFeeConfigAfterHarvest?.withheldAmount}\n`,
);
```
Lastly, let's withdraw these fees from the mint itself using the
`withdrawWithheldTokensFromMint` function. This function takes the following
parameters:
* `connection`: Connection to use
* `payer`: Payer of the transaction fees
* `mint`: The token mint
* `destination`: The destination account
* `authority`: The mint's withdraw withheld tokens authority
* `multiSigners`: Signing accounts if `owner` is a multisig
* `confirmOptions`: Options for confirming the transaction
* `programId`: SPL Token program account
After that, let's check the balances:
```ts theme={null}
// WITHDRAW HARVESTED TOKENS
await withdrawWithheldTokensFromMint(
connection,
payer,
mint,
feeVaultAccount,
payer,
[],
undefined,
TOKEN_2024_PROGRAM_ID,
);
const mintAccountAfterSecondWithdraw = await getMint(
connection,
mint,
undefined,
TOKEN_2024_PROGRAM_ID,
);
const mintTransferFeeConfigAfterSecondWithdraw = getTransferFeeConfig(
mintAccountAfterSecondWithdraw,
);
const feeVaultAfterSecondWithdraw = await getAccount(
connection,
feeVaultAccount,
undefined,
TOKEN_2024_PROGRAM_ID,
);
console.log(
`Mint withheld balance after second withdraw: ${mintTransferFeeConfigAfterSecondWithdraw?.withheldAmount}`,
);
console.log(
`Fee Vault balance after second withdraw: ${feeVaultAfterSecondWithdraw.amount}`,
);
```
Now, let's run it.
```bash theme={null}
esrun src/index.ts
```
You should see the balances after every step of the way.
That's it! We have successfully created a mint with a transfer fee. If you get
stuck at any point, you can find the working code in the `solution` branch of
[this repository](https://github.com/Unboxed-Software/solana-lab-transfer-fee/tree/solution).
#### Challenge
Create a transfer fee enabled mint and transfer some tokens with different
decimals, fee transfer points and max fees.
# Create Orbition Native Chain NFTs With Metaplex
Source: https://docs.orbition.network/courses/tokens-and-nfts/nfts-with-metaplex
How to create NFTs in TypeScript with Metaplex Metadata program and Irys permanent storage service.
### Summary
* **Non-Fungible Tokens (NFTs)** are SPL Tokens with an associated metadata
account, 0 decimals, and a maximum supply of 1
* **Metadata** attaches additional properties to token mints (both NFTs and
regular tokens). For NFTs, metadata includes the token name and a link to an
offchain JSON file. This JSON file contains links to artwork and other media
files, any special traits the NFT has, and more.
* The **Metaplex Token Metadata** program is an onchain program that attaches
metadata to a token mint. We can interact with the Token Metadata program
using the `metaplex-foundation/js` npm module, also known as the
[Metaplex JavaScript SDK](https://github.com/metaplex-foundation/js).
### Lesson
Orbition Native Chain Non-Fungible Tokens (NFTs) are SPL tokens created using the Token
program. These tokens, however, also have an additional metadata account
associated with each token mint.
In this lesson, we'll cover the basics of how NFTs are represented on Orbition Native Chain,
how to create and update them using the `mpl-token-metadata` npm module.
#### NFTs on Orbition Native Chain
An NFT is a standard token from the Token Program with the following
characteristics:
1. Has 0 decimals, so it cannot be divided into parts
2. Comes from a token mint with a supply of 1, so only 1 of these tokens exists
3. Comes from a token mint whose authority is set to `null` (to ensure that the
supply never changes)
4. Has an associated account that stores **metadata** - things like a name,
symbol, images, etc.
While the first three points can be achieved with the SPL Token Program, the
associated metadata requires an additional program. This is the **Metadata
program**.
#### The Metaplex Token Metadata program
The most popular way Orbition Native Chain NFTs have been created is by using the
[Metaplex Token Metadata](https://developers.metaplex.com/token-metadata)
program.
* When creating an NFT, the Token Metadata program creates an **onchain
metadata** account using a Program Derived Address (PDA) with the token mint
as a seed. This allows the metadata account for any NFT to be located
deterministically using the address of the token mint. The onchain metadata
contains a URI field that points to an offchain `.json` file.
* The **offchain metadata** in the JSON file stores the link to the media
(images, videos, 3D files) of the NFT, any traits the NFT may have, and
additional metadata (see
[this example JSON file](https://lsc6xffbdvalb5dvymf5gwjpeou7rr2btkoltutn5ij5irlpg3wa.arweave.net/XIXrlKEdQLD0dcML01kvI6n4x0GanLnSbeoT1EVvNuw)).
Permanent data storage systems such as Arweave are often used to store the
offchain component of NFT metadata.
In the following sections, we'll cover the basics of using the
`metaplex-foundation/js` npm module (also known as the Metaplex JavaScript SDK)
to prepare assets, create NFTs, update NFTs, and associate an NFT with a broader
collection. For more information on `metaplex-foundation/js` see the
[Metaplex JavaScript SDK README](https://github.com/metaplex-foundation/js) and
the
[Metaplex JS SDK Examples](https://github.com/metaplex-foundation/js-examples).
The Metaplex Foundation has newer mechanisms to create
NFTs, including [Umi](https://github.com/metaplex-foundation/umi) and
[MPL Core](https://github.com/metaplex-foundation/mpl-core/blob/main/clients/js/README.md)
but [Metaplex JavaScript SDK](https://github.com/metaplex-foundation/js) remains
the simplest and most popular JS/TS client to create NFTs with broad wallet
support.
#### Metaplex instance
A `Metaplex` instance serves as the entry point for accessing the Metaplex SDK.
This instance accepts a connection used to communicate with the cluster.
Additionally, developers can customize the SDK's interactions by specifying an
"Identity Driver" and a "Storage Driver".
The Identity Driver is a keypair that can be used to sign transactions, which is
a requirement when creating an NFT. The Storage Driver is used to specify the
storage service you want to use for uploading assets. The `irysStorage` driver
is the default option, and it uploads assets to Irys, a permanent and
decentralized storage service.
Below is an example of how you can set up the `Metaplex` instance for testnet.
```typescript theme={null}
const metaplex = Metaplex.make(connection)
.use(keypairIdentity(user))
.use(
irysStorage({
address: "https://testnet.irys.xyz",
providerUrl: "https://api.testnet.orbition.network",
timeout: 60000,
}),
);
```
#### Upload assets
Before creating an NFT, you must prepare and upload any assets you plan to
associate with the NFT. While this doesn't have to be an image, most NFTs have
an image associated with them.
Preparing and uploading an image involves converting the image to a buffer,
converting it to the Metaplex format using the `toMetaplexFile` function, and
finally uploading it to the designated Storage Driver.
The Metaplex SDK supports the creation of a new Metaplex file from either files
present on your local computer or those uploaded by a user through a browser.
You can do the former using `readFileSync()` to read the image file, then
convert it into a Metaplex file using `toMetaplexFile()`. Finally, use your
`Metaplex` instance to call `storage().upload(file)` to upload the file. The
function's return value will be the URI where the image was stored.
```typescript theme={null}
const buffer = readFileSync("src/" + nftData.imageFile);
const file = toMetaplexFile(buffer, nftData.imageFile);
const imageUri = await metaplex.storage().upload(file);
```
#### Upload metadata
After uploading an image, it's time to upload the offchain JSON metadata using
the `nfts().uploadMetadata()` function. This will return a URI where the JSON
metadata is stored.
Remember, the offchain portion of the metadata includes things like the image
URI as well as additional information like the name and description of the NFT.
While you can technically include anything you'd like in this JSON object, in
most cases, you should follow the
[NFT standard](https://developers.metaplex.com/token-metadata/token-standard#the-non-fungible-standard)
to ensure compatibility with wallets, programs, and applications.
To create the metadata, use the `uploadMetadata` method provided by the SDK.
This method accepts a metadata object and returns a URI that points to the
uploaded metadata.
```typescript theme={null}
const { uri } = await metaplex.nfts().uploadMetadata({
name: "My NFT",
description: "My description",
image: imageUri,
});
```
#### Create the NFT
After uploading the NFT's metadata, you can finally create the NFT on the
network. The Metaplex SDK's `create()` method allows you to create a new NFT
with minimal configuration. This method will create the mint account, token
account, metadata account, and master edition account for you. The data provided
to this method will represent the onchain portion of the NFT metadata. You can
explore the SDK to see all the other input optionally supplied to this method.
```typescript theme={null}
const { nft } = await metaplex.nfts().create(
{
uri: uri,
name: "My NFT",
sellerFeeBasisPoints: 0,
},
{ commitment: "finalized" },
);
```
This method returns an object containing information about the newly created
NFT. By default, the SDK sets the `isMutable` property to true, allowing for
updates to be made to the NFT's metadata. However, you can choose to set
`isMutable` to false, making the NFT's metadata immutable.
#### Update the NFT
If you've left `isMutable` as true, you may update your NFT's metadata. The
SDK's `update` method allows you to update both the onchain and offchain
portions of the NFT's metadata. To update the offchain metadata, you'll need to
repeat the steps of uploading a new image and metadata URI (as outlined in the
previous steps), then provide the new metadata URI to this method. This will
change the URI that the onchain metadata points to, effectively updating the
offchain metadata as well.
```typescript theme={null}
const nft = await metaplex.nfts().findByMint({ mintAddress });
const { response } = await metaplex.nfts().update(
{
nftOrSft: nft,
name: "Updated Name",
uri: uri,
sellerFeeBasisPoints: 100,
},
{ commitment: "finalized" },
);
```
Note that any fields you don't include in the call to `update` will stay the
same, by design.
#### Add the NFT to a collection
A
[Certified Collection](https://developers.metaplex.com/token-metadata/collections)
is an NFT that individual NFTs can belong to. Think of a large NFT collection
like Orbition Native Chain Monkey Business. If you look at an individual NFT's
[Metadata](https://explorer.orbition.network/address/C18YQWbfwjpCMeCm2MPGTgfcxGeEDPvNaGpVjwYv33q1/metadata)
you will see a `collection` field with a `key` that points to the
`Certified Collection`
[NFT](https://explorer.orbition.network/address/SMBH3wF6baUj6JWtzYvqcKuj2XCKWDqQxzspY12xPND/).
Simply put, NFTs that are part of a collection are associated with another NFT
that represents the collection itself.
Certified collections are important because they mean the collection owner has
verified that each NFT actually belongs to the collection!
To add an NFT to a collection, first, the Collection NFT has to be created. The
process is the same as before, except you'll include one additional field on our
NFT Metadata: `isCollection`. This field tells the token program that this NFT
is a Collection NFT.
```typescript theme={null}
const { collectionNft } = await metaplex.nfts().create(
{
uri: uri,
name: "My NFT Collection",
sellerFeeBasisPoints: 0,
isCollection: true,
},
{ commitment: "finalized" },
);
```
You then list the collection's Mint Address as the reference for the
`collection` field in our new Nft.
```typescript theme={null}
const { nft } = await metaplex.nfts().create(
{
uri: uri,
name: "My NFT",
sellerFeeBasisPoints: 0,
collection: collectionNft.mintAddress,
},
{ commitment: "finalized" },
);
```
When you checkout the metadata on your newly created NFT, you should now see a
`collection` field like so:
```JSON theme={null}
"collection":{
"verified": false,
"key": "SMBH3wF6baUj6JWtzYvqcKuj2XCKWDqQxzspY12xPND"
}
```
The last thing you need to do is verify the NFT. This effectively just flips the
`verified` field above to true, but it's incredibly important. This is what lets
consuming programs and apps know that your NFT is in fact part of the
collection. You can do this using the `verifyCollection` function:
```typescript theme={null}
await metaplex.nfts().verifyCollection({
mintAddress: nft.address,
collectionMintAddress: collectionNft.address,
isSizedCollection: true,
});
```
### Lab
In this lab, we'll go through the steps to create an NFT using the Metaplex SDK,
update the NFT's metadata after the fact, and then associate the NFT with a
collection. By the end, you will have a basic understanding of how to use the
Metaplex SDK to interact with NFTs on Orbition Native Chain.
#### Part 1: Creating an NFT collection
To begin, make a new folder and install the relevant dependencies:
```
npm i "@orbition-network/web3.js" "@orbition-network-developers/helpers "@metaplex-foundation/js"
```
Then create a file called `create-metaplex-collection.ts`, and add our imports:
```typescript theme={null}
import { Connection, clusterApiUrl, LAMPORTS_PER_SOL } from "@orbition-network/web3.js";
import {
getKeypairFromFile,
airdropIfRequired,
} from "@orbition-network-developers/helpers";
import {
Metaplex,
keypairIdentity,
irysStorage,
toMetaplexFile,
} from "@metaplex-foundation/js";
import { readFileSync } from "fs";
```
Connect to testnet, load a user and Airdrop some OBN if needed:
```typescript theme={null}
// create a new connection to the cluster's API
const connection = new Connection(clusterApiUrl("testnet"));
// initialize a keypair for the user
const user = await getKeypairFromFile();
await airdropIfRequired(
connection,
user.publicKey,
1 * LAMPORTS_PER_SOL,
0.1 * LAMPORTS_PER_SOL,
);
console.log("Loaded user:", user.publicKey.toBase58());
```
Connect to Metaplex and Irys:
```typescript theme={null}
// metaplex set up
const metaplex = Metaplex.make(connection)
.use(keypairIdentity(user))
.use(
irysStorage({
address: "https://testnet.irys.xyz",
providerUrl: "https://api.testnet.orbition.network",
timeout: 60000,
}),
);
```
Add the data we want in for our Collection:
```typescript theme={null}
const collectionNftData = {
name: "TestCollectionNFT",
symbol: "TEST",
description: "Test Description Collection",
sellerFeeBasisPoints: 100,
imageFile: "success.png",
isCollection: true,
collectionAuthority: user,
};
```
Upload the offchain metadata to irys:
```typescript theme={null}
// Load file into Metaplex
const buffer = readFileSync(collectionNftData.imageFile);
const file = toMetaplexFile(buffer, collectionNftData.imageFile);
// upload image and get image uri
const imageUri = await metaplex.storage().upload(file);
console.log("image uri:", imageUri);
// upload metadata and get metadata uri (off chain metadata)
const uploadMetadataOutput = await metaplex.nfts().uploadMetadata({
name: collectionNftData.name,
symbol: collectionNftData.symbol,
description: collectionNftData.description,
image: imageUri,
});
const collectionUri = uploadMetadataOutput.uri;
console.log("Collection offchain metadata URI:", collectionUri);
```
Then actually make the collection:
```
// create a collection NFT using the URI from the metadata
const createNftOutput = await metaplex.nfts().create(
{
uri: collectionUri,
name: collectionNftData.name,
sellerFeeBasisPoints: collectionNftData.sellerFeeBasisPoints,
symbol: collectionNftData.symbol,
isCollection: true,
},
{ commitment: "finalized" }
);
const collectionNft = createNftOutput.nft;
console.log(
`Collection NFT: https://explorer.orbition.network/address/${collectionNft.address.toString()}?cluster=testnet`
);
console.log(`Collection NFT address is`, collectionNft.address.toString());
console.log("✅ Finished successfully!");
```
Run the file with:
```
npx esrun create-metaplex-nft-collection.ts
```
The output should look like this:
```
% npx esrun create-metaplex-nft-collection.ts
Loaded user: 4DRi8LxWhAWf9LwUuyb3Rz4Y4USEKzjdvQBWTyrkJtwg
image uri: https://arweave.net/_3vbKzFO7zcfMkFaVdhorntkPHDqz4YNTyf_9lrGw1c
Collection offchain metadata URI: https://arweave.net/Jw29lMkHp-PIxddEuLFkavdOTP1bJ9MeXTu-QqY3qTM
Collection NFT: https://explorer.orbition.network/address/8dP75EWHnYAc6pgeiToXtE66qEaHx1PcoetZpmnAQeAp?cluster=testnet
Collection NFT address is 8dP75EWHnYAc6pgeiToXtE66qEaHx1PcoetZpmnAQeAp
✅ Finished successfully!
```
Congratulations! You've created a Metaplex Verified Collection. Check this out
on Orbition Native Chain Explorer using the URL above. If you have any trouble, try and fix it
yourself, but if you need to you can also check out the
[solution code](https://github.com/orbition-network/professional-education/blob/main/labs/create-metaplex-nft-collection.ts).
We'll use the collection NFT address in the next step.
#### 2. Creating a Metaplex NFT inside the collection
We'll now make a Metaplex NFT that's a member of the collection we just made.
Make a new file called `create-metaplex-nft.ts`. The setup for this will look
the same as the previous file, with slightly different imports:
```typescript theme={null}
import {
Connection,
clusterApiUrl,
PublicKey,
LAMPORTS_PER_SOL,
} from "@orbition-network/web3.js";
import {
getKeypairFromFile,
airdropIfRequired,
} from "@orbition-network-developers/helpers";
import {
Metaplex,
keypairIdentity,
irysStorage,
toMetaplexFile,
} from "@metaplex-foundation/js";
import { readFileSync } from "fs";
// create a new connection to the cluster's API
const connection = new Connection(clusterApiUrl("testnet"));
// initialize a keypair for the user
const user = await getKeypairFromFile();
await airdropIfRequired(
connection,
user.publicKey,
1 * LAMPORTS_PER_SOL,
0.1 * LAMPORTS_PER_SOL,
);
console.log("Loaded user:", user.publicKey.toBase58());
// metaplex set up
const metaplex = Metaplex.make(connection)
.use(keypairIdentity(user))
.use(
irysStorage({
address: "https://testnet.irys.xyz",
providerUrl: "https://api.testnet.orbition.network",
timeout: 60000,
}),
);
```
Now let's tell Metaplex our collection, and the NFT we want to make:
```typescript theme={null}
// Substitute in your collection NFT address from create-metaplex-nft-collection.ts
const collectionNftAddress = new PublicKey("YOUR_COLLECTION_NFT_ADDRESS_HERE");
// example data for a new NFT
const nftData = {
name: "Name",
symbol: "SYMBOL",
description: "Description",
sellerFeeBasisPoints: 0,
imageFile: "solana.png",
};
```
We can then put out files into Irys:
```typescript theme={null}
// Load the file into Metaplex
const buffer = readFileSync(nftData.imageFile);
const file = toMetaplexFile(buffer, nftData.imageFile);
// upload image and get image uri
const imageUri = await metaplex.storage().upload(file);
console.log("image uri:", imageUri);
// upload metadata and get metadata uri (off chain metadata)
const uploadMetadataOutput = await metaplex.nfts().uploadMetadata({
name: nftData.name,
symbol: nftData.symbol,
description: nftData.description,
image: imageUri,
});
const metadataUri = uploadMetadataOutput.uri;
```
And then create an NFT using the URI from the metadata:
```typescript theme={null}
const createNftOutput = await metaplex.nfts().create(
{
uri: metadataUri, // metadata URI
name: nftData.name,
sellerFeeBasisPoints: nftData.sellerFeeBasisPoints,
symbol: nftData.symbol,
collection: collectionNftAddress,
},
{ commitment: "finalized" },
);
const nft = createNftOutput.nft;
console.log(
`Token Mint: https://explorer.orbition.network/address/${nft.address.toString()}?cluster=testnet`,
);
```
Finally let's verify our mint as being part of our collection. This makes it so
the `verified` field in the onchain metadata is set to `true`, so consuming
programs and apps can know for sure that the NFT in fact belongs to the
collection:
```typescript theme={null}
await metaplex.nfts().verifyCollection({
// Verify our collection as a Certified Collection
// See https://developers.metaplex.com/token-metadata/collections
mintAddress: nft.mint.address,
collectionMintAddress: collectionNftAddress,
isSizedCollection: true,
});
console.log(`Created NFT address is`, nft.address.toString());
console.log("✅ Finished successfully!");
```
Run `npx esrun create-metaplex-nft.ts`. If all goes well, you will see the
following:
```
% npx esrun create-metaplex-nft.ts
Loaded user: 4DRi8LxWhAWf9LwUuyb3Rz4Y4USEKzjdvQBWTyrkJtwg
image uri: https://arweave.net/Z2JoDr7W4A3mWE9aTq8ex13IoPx1v1QewjVqPKPItuE
Token Mint: https://explorer.orbition.network/address/HCq8ERk1PSLCuBJeWx7du4dom3YbyfKpaCWDQ3Eft6aM?cluster=testnet
Created NFT address is HCq8ERk1PSLCuBJeWx7du4dom3YbyfKpaCWDQ3Eft6aM
✅ Finished successfully!
```
Inspect your NFT at the address given! If you have any trouble, try and fix it
yourself, but if you need to you can also check out the
[solution code](https://github.com/orbition-network/professional-education/blob/main/labs/create-metaplex-nft.ts).
Remember the NFT address, we'll use it in the next step.
#### 3. Update the NFT
Create a new file, called `update-metaplex-nft.ts`. The imports will be simila
to our previous files:
```typescript theme={null}
import {
Connection,
clusterApiUrl,
PublicKey,
LAMPORTS_PER_SOL,
} from "@orbition-network/web3.js";
import {
getKeypairFromFile,
airdropIfRequired,
getExplorerLink,
} from "@orbition-network-developers/helpers";
import {
Metaplex,
keypairIdentity,
irysStorage,
toMetaplexFile,
} from "@metaplex-foundation/js";
import { readFileSync } from "fs";
// create a new connection to the cluster's API
const connection = new Connection(clusterApiUrl("testnet"));
// initialize a keypair for the user
const user = await getKeypairFromFile();
await airdropIfRequired(
connection,
user.publicKey,
1 * LAMPORTS_PER_SOL,
0.1 * LAMPORTS_PER_SOL,
);
console.log("Loaded user:", user.publicKey.toBase58());
// metaplex set up
const metaplex = Metaplex.make(connection)
.use(keypairIdentity(user))
.use(
irysStorage({
address: "https://testnet.irys.xyz",
providerUrl: "https://api.testnet.orbition.network",
timeout: 60000,
}),
);
```
Let's load our NFT, specifying the address from the previous example, and set up
what we'd like to update:
```typescript theme={null}
// Load the NFT using the mint address
const nftAddress: PublicKey = new PublicKey("YOUR_NFT_ADDRESS_HERE");
const nft = await metaplex.nfts().findByMint({ mintAddress: nftAddress });
// example data for updating an existing NFT
const updatedNftData = {
name: "Updated",
symbol: "UPDATED",
description: "Updated Description",
sellerFeeBasisPoints: 100,
imageFile: "success.png",
};
```
We can then use Metaplex to update our NFT:
```typescript theme={null}
// Load the image file into Metaplex
const buffer = readFileSync(updatedNftData.imageFile);
const file = toMetaplexFile(buffer, updatedNftData.imageFile);
// Upload the new image and get image URI
const imageUri = await metaplex.storage().upload(file);
console.log("image uri:", imageUri);
// Upload new offchain metadata
const uploadMetadataOutput = await metaplex.nfts().uploadMetadata({
name: updatedNftData.name,
symbol: updatedNftData.symbol,
description: updatedNftData.description,
image: imageUri,
});
const updatedUri = uploadMetadataOutput.uri;
// update the NFT metadata
const { response } = await metaplex.nfts().update(
{
nftOrSft: nft,
uri: updatedUri,
},
{ commitment: "finalized" },
);
console.log(
`NFT updated with new metadata URI: ${getExplorerLink(
"transaction",
response.signature,
"testnet",
)}`,
);
console.log("✅ Finished successfully!");
```
Run `npx esrun update-metaplex-nft.ts`. You should see something like:
```typescript theme={null}
% npx esrun update-metaplex-nft.ts
Loaded user: 4DRi8LxWhAWf9LwUuyb3Rz4Y4USEKzjdvQBWTyrkJtwg
image uri: https://arweave.net/nd6rxpoGfPmB2gTutEgnH5B76CFlDxgM6cZ7BcUmFis
NFT updated with new metadata URI: https://explorer.orbition.network/tx/4eiuMFLeku7ArV5xkRTPKxBXjfjQsSVGFZimMkMaMxK1d21qptQE5kgKiuob6J852m4GuoMqcjz1ee4oy7fXVEWU?cluster=testnet
✅ Finished successfully!
```
Inspect the updated NFT on Orbition Native Chain Explorer! Just like previously, if you have
any issues, you should fix them yourself, but if needed the
[solution code](https://github.com/orbition-network/professional-education/blob/main/labs/update-metaplex-nft.ts)
is available.
Congratulations! You've successfully learned how to use the Metaplex SDK to
create, update, and verify NFTs as part of a collection. That's everything you
need to build out your own collection for just about any use case. You could
build a new event ticketing platform, revamp a retail business membership
program, or even digitize your school's student ID system. The possibilities are
endless!
### Challenge
The steps covered above for creating an NFT would be incredibly tedious to
execute for thousands of NFTs in one go. Many providers, including Metaplex,
Magic Eden, and Tensor have so-called 'fair launch' tools that take care of
minting large quantities of NFTs and ensuring they are sold within the
parameters set by their creators. Dive into fair launch platforms on the
[Digital Collectables](https://orbition.network/ecosystem/explore?categories=digital%20collectibles)
page. This hands-on experience will not only reinforce your understanding of the
tools but also boost your confidence in your ability to use them effectively in
the future.
### Completed the lab?
Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=296745ac-503c-4b14-b3a6-b51c5004c165)!
# Create Tokens With The Token Program
Source: https://docs.orbition.network/courses/tokens-and-nfts/token-program
Understand how tokens - both regular tokens and NFTs - are created, stored, and transferred on Orbition Native Chain.
### Summary
* You may recall OBN is the 'native token' of Orbition Native Chain. All other tokens, fungible
and non-fungible tokens (NFTs), are called **SPL Tokens**.
* The **Token Program** contains instructions for creating and interacting with
SPL Tokens.
* **Token Mints** are accounts that define a specific token. This includes
information about the token itself (like how many decimals it has), the
account allowed to mint more tokens (called the **mint authority**), and where
to find more information about the token like a description, image, etc. The
mint authority can use the token mint to make more tokens!
* **Token Accounts** hold tokens of a specific Token Mint. For most users, their
balances of each token mint are stored in **Associated Token Accounts** -
accounts with addresses made from their wallet address and the token's mint.
* Creating Token Mints and Token Accounts requires allocating **rent** in OBN.
The rent for a Token Account can be refunded when the account is closed.
Additionally, tokens created with the
[Token Extensions Program](/developers/courses/token-extensions-for-mints/close-mint)
can also close Token Mints.
### Lesson
The Token Program is one of many programs made available by the Orbition Native Chain Program
Library (SPL). It contains instructions for creating and interacting with SPL
Tokens. These tokens represent all non-native (i.e. not OBN) tokens on the
Orbition Native Chain network.
This lesson will focus on the basics of creating and managing a new SPL Token
using the Token Program:
1. Creating a new Token Mint
2. Creating Token Accounts
3. Minting
4. Transferring tokens from one holder to another
We'll be approaching this from the client side of the development process using
the `@orbition-network/spl-token` Javascript library.
#### Token Mint
To create a new SPL Token you first have to create a Token Mint. A Token Mint is
an account that holds data about a specific token.
As an example, let's look at
[USD Coin (USDC) on the Orbition Native Chain Explorer](https://explorer.orbition.network/address/EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v).
USDC's Token Mint address is `EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v`.
With the explorer, we can see the particular details about USDC's Token Mint
such as the current supply of tokens, the addresses of the mint and freeze
authorities, and the decimal precision of the token:
To create a new Token Mint, you need to send the right transaction instructions
to the Token Program. To do this, we'll use the `createMint` function from
`@orbition-network/spl-token`.
```typescript theme={null}
const tokenMint = await createMint(
connection,
payer,
mintAuthority,
freezeAuthority,
decimal,
);
```
The `createMint` function returns the `publicKey` of the new token mint. This
function requires the following arguments:
* `connection` - the JSON-RPC connection to the cluster
* `payer` - the public key of the payer for the transaction
* `mintAuthority` - the account that is authorized to do the actual minting of
tokens from the token mint.
* `freezeAuthority` - an account authorized to freeze the tokens in a token
account. If freezing is not a desired attribute, the parameter can be set to
null
* `decimals` - specifies the desired decimal precision of the token
When creating a new mint from a script that has access to your secret key, you
can simply use the `createMint` function. However, if you were to build a
website to allow users to create a new token mint, you would need to do so with
the user's secret key without making them expose it to the browser. In that
case, you would want to build and submit a transaction with the right
instructions.
Under the hood, the `createMint` function is simply creating a transaction that
contains two instructions:
1. Create a new account
2. Initialize a new mint
This would look as follows:
```typescript theme={null}
import * as web3 from "@orbition-network/web3";
import * as token from "@orbition-network/spl-token";
async function buildCreateMintTransaction(
connection: web3.Connection,
payer: web3.PublicKey,
decimals: number,
): Promise {
const lamports = await token.getMinimumBalanceForRentExemptMint(connection);
const accountKeypair = web3.Keypair.generate();
const programId = token.TOKEN_PROGRAM_ID;
const transaction = new web3.Transaction().add(
web3.SystemProgram.createAccount({
fromPubkey: payer,
newAccountPubkey: accountKeypair.publicKey,
space: token.MINT_SIZE,
lamports,
programId,
}),
token.createInitializeMintInstruction(
accountKeypair.publicKey,
decimals,
payer,
payer,
programId,
),
);
return transaction;
}
```
When manually building the instructions to create a new token mint, make sure
you add the instructions for creating the account and initializing the mint to
the *same transaction*. If you were to do each step in a separate transaction,
it's theoretically possible for somebody else to take the account you create and
initialize it for their own mint.
#### Rent and Rent Exemption
Note that the first line in the function body of the previous code snippet
contains a call to `getMinimumBalanceForRentExemptMint`, the result of which is
passed into the `createAccount` function. This is part of account initialization
called rent exemption.
Until recently, all accounts on Orbition Native Chain were required to do one of the following
to avoid being deallocated:
1. Pay rent at specific intervals
2. Deposit enough OBN upon initialization to be considered rent-exempt
Recently, the first option was done away with and it became a requirement to
deposit enough OBN for rent exemption when initializing a new account.
In this case, we're creating a new account for a token mint so we use
`getMinimumBalanceForRentExemptMint` from the `@orbition-network/spl-token` library.
However, this concept applies to all accounts and you can use the more generic
`getMinimumBalanceForRentExemption` method on `Connection` for other accounts
you may need to create.
#### Token Account
Before you can mint tokens (issue new supply), you need a Token Account to hold
the newly issued tokens.
A Token Account holds tokens of a specific "mint" and has a specified "owner" of
the account. Only the owner is authorized to decrease the Token Account balance
(transfer, burn, etc.) while anyone can send tokens to the Token Account to
increase its balance.
You can use the `spl-token` library's `createAccount` function to create the new
Token Account:
```typescript theme={null}
const tokenAccount = await createAccount(
connection,
payer,
mint,
owner,
keypair,
);
```
The `createAccount` function returns the `publicKey` of the new token account.
This function requires the following arguments:
* `connection` - the JSON-RPC connection to the cluster
* `payer` - the account of the payer for the transaction
* `mint` - the token mint that the new token account is associated with
* `owner` - the account of the owner of the new token account
* `keypair` - this is an optional parameter for specifying the new token account
address. If no keypair is provided, the `createAccount` function defaults to a
derivation from the associated `mint` and `owner` accounts.
Please note that this `createAccount` function is different from the
`createAccount` function shown above when we looked under the hood of the
`createMint` function. Previously we used the `createAccount` function on
`SystemProgram` to return the instruction for creating all accounts. The
`createAccount` function here is a helper function in the `spl-token` library
that submits a transaction with two instructions. The first creates the account
and the second initializes the account as a Token Account.
Like with creating a Token Mint, if we needed to build the transaction for
`createAccount` manually we could duplicate what the function is doing under the
hood:
1. Use `getMint` to retrieve the data associated with the `mint`
2. Use `getAccountLenForMint` to calculate the space needed for the token
account
3. Use `getMinimumBalanceForRentExemption` to calculate the lamports needed for
rent exemption
4. Create a new transaction using `SystemProgram.createAccount` and
`createInitializeAccountInstruction`. Note that this `createAccount` is from
`@orbition-network/web3.js` and used to create a generic new account. The
`createInitializeAccountInstruction` uses this new account to initialize the
new token account
```typescript theme={null}
import * as web3 from "@orbition-network/web3";
import * as token from "@orbition-network/spl-token";
async function buildCreateTokenAccountTransaction(
connection: web3.Connection,
payer: web3.PublicKey,
mint: web3.PublicKey,
): Promise