Overview
The SPL (Solana Program Library) Token Program 2022 introduces several enhancements and features, with a significant focus on the extension functionality. These extensions allow developers to add additional capabilities and configurations to token accounts and mints without modifying the core SPL Token Program.
Data Structure
The program uses
PodStateWithExtensionsMut
to manage general data in data account. base
: the basic data, likeMint
data defined in original Token Program.
account_type
: type of the account
tlv_data
: extension data, it uses tlv(type, length and value) to store tlv data.
For a data structure to be
BaseState
, it needs to implement three traits:BaseState
: return the account type
PackedSizeOf
: return the size of the account data structure
IsInitialized
: return whether the account has been initialized
/// --- token/program-2022/src/extension/mod.rs --- /// Trait for base states, specifying the associated enum pub trait BaseState: PackedSizeOf + IsInitialized { /// Associated extension type enum, checked at the start of TLV entries const ACCOUNT_TYPE: AccountType; } /// Different kinds of accounts. Note that `Mint`, `Account`, and `Multisig` /// types are determined exclusively by the size of the account, and are not /// included in the account data. `AccountType` is only included if extensions /// have been initialized. #[repr(u8)] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum AccountType { /// Marker for 0 data Uninitialized, /// Mint account with additional extensions Mint, /// Token holding account with additional extensions Account, } /// Encapsulates mutable base state data (mint or account) with possible /// extensions, where the base state is Pod for zero-copy serde. #[derive(Debug, PartialEq)] pub struct PodStateWithExtensionsMut<'data, S: BaseState> { /// Unpacked base data pub base: &'data mut S, /// Writable account type account_type: &'data mut [u8], /// Slice of data containing all TLV data, deserialized on demand tlv_data: &'data mut [u8], } /// --- token/program-2022/src/state.rs --- /// Simplified version of the `Pack` trait which only gives the size of the /// packed struct. Useful when a function doesn't need a type to implement all /// of `Pack`, but a size is still needed. pub trait PackedSizeOf { /// The packed size of the struct const SIZE_OF: usize; } /// --- solana-program-1.18.11/src/program_pack.rs --- /// Check if a program account state is initialized pub trait IsInitialized { /// Is initialized fn is_initialized(&self) -> bool; }
There are three accounts type:
- Uninitialized
- Mint
- Account
Different accounts have different data size. To make different kinds of account distinguishable, SPL uses the max data size as the common data size used by different kinds of accounts. So it can store the account type in the same position followed by tlv data.
tlv
data consists of multiple type, length and value pairs. Because extensions can be added to account dynamically, so SPL can’t fix the specific extensions data structure. And it uses tlv
format to store extensions flexibly. - When we need to add some extension to the account, SPL will iterate the
tlv
data to find an uninitialized area, and record the extension there.
- If we want to update the extension’s data, then SPL also iterates the
tlv
data to matchtype
of each extension to get the corresponding extension’s position, and update the data.
- SPL ensures that each kind of extension can only occur once in the
tlv
.
We can see the introduction of this in the code:
/// Any account with extensions must be at least `Account::LEN`. Both mints and /// accounts can have extensions /// A mint with extensions that takes it past 165 could be indiscernible from an /// Account with an extension, even if we add the account type. For example, /// let's say we have: /// /// Account: 165 bytes... + [2, 0, 3, 0, 100, ....] /// ^ ^ ^ ^ /// acct type extension length data... /// /// Mint: 82 bytes... + 83 bytes of other extension data /// + [2, 0, 3, 0, 100, ....] /// (data in extension just happens to look like this) /// /// With this approach, we only start writing the TLV data after Account::LEN, /// which means we always know that the account type is going to be right after /// that. We do a special case checking for a Multisig length, because those /// aren't extensible under any circumstances. const BASE_ACCOUNT_LENGTH: usize = Account::LEN;
Example
Let’s use setting
mint_close_authority
process as an example to illustrate how the extension works.process_initialize_mint_close_authority
is the entry to initialize mint_close_authority
of a Mint account.Inside:
- get Mint account
- get account data in byte array format
- deserialize the account data
- initialize extension
// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMintCloseAuthority](enum.TokenInstruction.html) /// instruction pub fn process_initialize_mint_close_authority( accounts: &[AccountInfo], close_authority: PodCOption<Pubkey>, ) -> ProgramResult { /// get Mint account let account_info_iter = &mut accounts.iter(); let mint_account_info = next_account_info(account_info_iter)?; /// get account data in byte array format let mut mint_data = mint_account_info.data.borrow_mut(); /// deserialize the account data(byte array) let mut mint = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; /// initialize extension let extension = mint.init_extension::<MintCloseAuthority>(true)?; extension.close_authority = close_authority.try_into()?; Ok(()) }
unpack_uninitialized
unpack_uninitialized
is a function of PodStateWithExtensionsMut::<PodMint>
which is used to deserialize byte array to get PodStateWithExtensionsMut::<PodMint>
which includes:base
: isPodMint
whose structure is exactly same with theMint
defined in original token program. (This is for compatibility consideration). Also it ensures that the base state is not already initialized.
account_type
: account type
tlv
: extension data in tlv format
/// [Mint] data stored as a Pod type #[repr(C)] #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] pub struct PodMint { /// Optional authority used to mint new tokens. The mint authority may only /// be provided during mint creation. If no mint authority is present /// then the mint has a fixed supply and no further tokens may be /// minted. pub mint_authority: PodCOption<Pubkey>, /// Total supply of tokens. pub supply: PodU64, /// Number of base 10 digits to the right of the decimal place. pub decimals: u8, /// If `true`, this structure has been initialized pub is_initialized: PodBool, /// Optional authority to freeze token accounts. pub freeze_authority: PodCOption<Pubkey>, } impl IsInitialized for PodMint { fn is_initialized(&self) -> bool { self.is_initialized.into() } }
/// ---token/program/src/state.rs--- #[repr(C)] #[derive(Clone, Copy, Debug, Default, PartialEq)] pub struct Mint { /// Optional authority used to mint new tokens. The mint authority may only /// be provided during mint creation. If no mint authority is present /// then the mint has a fixed supply and no further tokens may be /// minted. pub mint_authority: COption<Pubkey>, /// Total supply of tokens. pub supply: u64, /// Number of base 10 digits to the right of the decimal place. pub decimals: u8, /// Is `true` if this structure has been initialized pub is_initialized: bool, /// Optional authority to freeze token accounts. pub freeze_authority: COption<Pubkey>, }
Inside the
unpack_uninitialized
:Steps:
- Check Minimum Length and Multisig Length:
ensures that the input slice has a minimum length of
S::SIZE_OF
bytes and is not exactly the length of a multisig account (Multisig::LEN
). If these conditions are not met, the function returns an InvalidAccountData
error.- Split the Input Slice:
base_data
: The firstS::SIZE_OF
bytes, which represent the base state.rest
: The remaining bytes, which potentially contain TLV (Type-Length-Value) data.
The input slice is split into two parts:
- Deserialize the Base State:
The base_data slice is deserialized into a base state of type S using the pod_from_bytes_mut function. This function ensures that the base state is correctly interpreted as a Pod (Plain Old Data) type.
- Check if the Base State is Already Initialized:
The function checks if the base state is already initialized by calling the
is_initialized
method on the deserialized base state. If the base state is already initialized, the function returns an AlreadyInUse
error.- Unpack the Account Type and TLV Data:
The remaining slice (
rest
) is passed to the unpack_uninitialized_type_and_tlv_data_mut
function, which unpacks the account type and TLV data from the slice. This function ensures that the account type is Uninitialized
.- Create and Return the State with Extensions:
- A new instance of
PodStateWithExtensionsMut
is created with the base state, account type, and TLV data. - The function calls
check_account_type_matches_extension_type
to ensure that the account type in the TLV data matches the base state's account type. - Finally, the function returns the newly created state.
/// --- token/program-2022/src/extension/mod.rs --- /// Encapsulates mutable base state data (mint or account) with possible /// extensions, where the base state is Pod for zero-copy serde. #[derive(Debug, PartialEq)] pub struct PodStateWithExtensionsMut<'data, S: BaseState> { /// Unpacked base data pub base: &'data mut S, /// Writable account type account_type: &'data mut [u8], /// Slice of data containing all TLV data, deserialized on demand tlv_data: &'data mut [u8], } impl<'data, S: BaseState + Pod> PodStateWithExtensionsMut<'data, S> { /// ... /// Unpack an uninitialized base state, leaving the extension data as a /// mutable slice /// /// Fails if the base state has already been initialized. pub fn unpack_uninitialized(input: &'data mut [u8]) -> Result<Self, ProgramError> { /// Check Minimum Length and Multisig Length: check_min_len_and_not_multisig(input, S::SIZE_OF)?; /// Split the Input Slice: let (base_data, rest) = input.split_at_mut(S::SIZE_OF); /// Deserialize the Base State: let base = pod_from_bytes_mut::<S>(base_data)?; /// Check if the Base State is Already Initialized: if base.is_initialized() { return Err(TokenError::AlreadyInUse.into()); } /// Unpack the Account Type and TLV Data: let (account_type, tlv_data) = unpack_uninitialized_type_and_tlv_data_mut::<S>(rest)?; /// Create and Return the State with Extensions: let state = Self { base, account_type, tlv_data, }; state.check_account_type_matches_extension_type()?; Ok(state) } /// ... }
check_min_len_and_not_multisig
checks the length of the input is not multisig length./// --- token/program-2022/src/extension/mod.rs --- fn check_min_len_and_not_multisig(input: &[u8], minimum_len: usize) -> Result<(), ProgramError> { if input.len() == Multisig::LEN || input.len() < minimum_len { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }
pod_from_bytes_mut
converts the raw bytes to corresponding structure./// --- libraries/pod/src/bytemuck.rs --- /// Convert a slice of bytes into a mutable `Pod` (zero copy) pub fn pod_from_bytes_mut<T: Pod>(bytes: &mut [u8]) -> Result<&mut T, ProgramError> { bytemuck::try_from_bytes_mut(bytes).map_err(|_| ProgramError::InvalidArgument) }
Get account type and tlv
unpack_uninitialized_type_and_tlv_data_mut
decodes the rest bytes to get AccountType
and tlv
bytes array. It also checks that the account hasn’t been initialized./// --- token/program-2022/src/extension/mod.rs --- fn unpack_uninitialized_type_and_tlv_data_mut<S: BaseState>( rest: &mut [u8], ) -> Result<(&mut [u8], &mut [u8]), ProgramError> { unpack_type_and_tlv_data_with_check_mut::<S, _>(rest, |account_type| { if account_type != AccountType::Uninitialized { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }) }
unpack_type_and_tlv_data_with_check_mut
uses type_and_tlv_indices
function to get the start index of account_type and tlv data in the rest
. Then it will use the check_fn
to check the validity of the account_type
and returns account_type
and tlv
byte array.Note that
AccountType
is represented using 1 byte. So there is some redundant code here (split the rest
, and read the account_type
by range./// --- token/program-2022/src/extension/mod.rs --- fn unpack_type_and_tlv_data_with_check_mut< S: BaseState, F: Fn(AccountType) -> Result<(), ProgramError>, >( rest: &mut [u8], check_fn: F, ) -> Result<(&mut [u8], &mut [u8]), ProgramError> { if let Some((account_type_index, tlv_start_index)) = type_and_tlv_indices::<S>(rest)? { // type_and_tlv_indices() checks that returned indexes are within range let account_type = AccountType::try_from(rest[account_type_index]) .map_err(|_| ProgramError::InvalidAccountData)?; check_fn(account_type)?; let (account_type, tlv_data) = rest.split_at_mut(tlv_start_index); Ok(( &mut account_type[account_type_index..tlv_start_index], tlv_data, )) } else { Ok((&mut [], &mut [])) } } /// --- token/program-2022/src/extension/mod.rs --- /// Different kinds of accounts. Note that `Mint`, `Account`, and `Multisig` /// types are determined exclusively by the size of the account, and are not /// included in the account data. `AccountType` is only included if extensions /// have been initialized. #[repr(u8)] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum AccountType { /// Marker for 0 data Uninitialized, /// Mint account with additional extensions Mint, /// Token holding account with additional extensions Account, }
type_and_tlv_indices
reads the start indexes of account_type
and tlv
.- if the
rest_input
is empty, then it just returnsOk(None)
, the caller will know that there is no extension in this account.
- else
- gets the start indexes of both
account_type
andtlv
based on the data structure of account. Note thatMint
account has less data size, so there are 83 padding zeros, so the start index ofaccount_type
is 83. - ensures tlv data space is not empty.
- check the padding part is filled with zero.
/// --- token/program-2022/src/extension/mod.rs --- fn type_and_tlv_indices<S: BaseState>( rest_input: &[u8], ) -> Result<Option<(usize, usize)>, ProgramError> { if rest_input.is_empty() { Ok(None) } else { /// get the start index of account_type let account_type_index = BASE_ACCOUNT_LENGTH.saturating_sub(S::SIZE_OF); // check padding is all zeroes let tlv_start_index = account_type_index.saturating_add(size_of::<AccountType>()); if rest_input.len() <= tlv_start_index { return Err(ProgramError::InvalidAccountData); } if rest_input[..account_type_index] != vec![0; account_type_index] { Err(ProgramError::InvalidAccountData) } else { Ok(Some((account_type_index, tlv_start_index))) } } }
Initialize Extension
process_initialize_mint_close_authority
calls init_extension
to initialize extension./// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMintCloseAuthority](enum.TokenInstruction.html) /// instruction pub fn process_initialize_mint_close_authority( accounts: &[AccountInfo], close_authority: PodCOption<Pubkey>, ) -> ProgramResult { let account_info_iter = &mut accounts.iter(); let mint_account_info = next_account_info(account_info_iter)?; let mut mint_data = mint_account_info.data.borrow_mut(); let mut mint: PodStateWithExtensionsMut<PodMint> = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; let extension = mint.init_extension::<MintCloseAuthority>(true)?; extension.close_authority = close_authority.try_into()?; Ok(()) } /// --- token/program-2022/src/extension/mint_close_authority.rs --- /// Close authority extension data for mints. #[repr(C)] #[cfg_attr(feature = "serde-traits", derive(Serialize, Deserialize))] #[cfg_attr(feature = "serde-traits", serde(rename_all = "camelCase"))] #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] pub struct MintCloseAuthority { /// Optional authority to close the mint pub close_authority: OptionalNonZeroPubkey, }
The
init_extension
is responsible for initializing an extension. This function checks if the extension already exists in the TLV (Type-Length-Value) buffer and either initializes it with a default value or returns an error if the extension is already present and overwrite is not allowed.Steps:
- Length Calculation
Calculates the length of the packed extension type
V
.- Buffer Allocation
Allocates space for the extension in the TLV buffer. It calls the
alloc
method with the calculated length and overwrite flag. If successful, returns a mutable slice of the allocated buffer. If it fails (e.g., if there's not enough space or the extension already exists and overwrite is not allowed), it returns an error.- Extension Initialization
- Converts the allocated buffer into a mutable reference of type V and initializes it with the default value of
V
. pod_from_bytes_mut::<V>(buffer)?
: Converts the buffer into a mutable reference of typeV
. This operation can fail if the buffer length does not match the expected size ofV
.extension_ref = V::default();
: Assigns the default value ofV
to the allocated buffer, effectively initializing the extension.
- Return Value
returns mutable reference to extension data space stored in the account(
extension_ref
) which allows caller to modify it./// --- token/program-2022/src/extension/mod.rs --- /// Packs the default extension data into an open slot if not already found /// in the data buffer. If extension is already found in the buffer, it /// overwrites the existing extension with the default state if /// `overwrite` is set. If extension found, but `overwrite` is not set, /// it returns error. fn init_extension<V: Extension + Pod + Default>( &mut self, overwrite: bool, ) -> Result<&mut V, ProgramError> { /// Length Calculation let length = pod_get_packed_len::<V>(); /// Buffer Allocation let buffer = self.alloc::<V>(length, overwrite)?; /// Extension Initialization let extension_ref = pod_from_bytes_mut::<V>(buffer)?; *extension_ref = V::default(); /// Return Value Ok(extension_ref) }
The
alloc
function allocates space for a given extension in a TLV (Type-Length-Value) structure within an account's data buffer, ensuring the extension is placed correctly and handling scenarios where the extension already exists.Steps:
- Account Type Check:
Ensures the extension's associated account type matches the base state's account type.
- Get TLV Data and Indices:
- Retrieves the mutable TLV data buffer.
- Determines the start positions for the type, length, and value fields in the TLV entry.
- Check Buffer Length:
Ensures there is enough space in the TLV buffer to accommodate the new extension.
- Get Existing Extension Type:
- Handle Extension Initialization and Overwriting:
- If Uninitialized or Overwrite: Writes the extension type and length into the TLV data.If overwriting, ensures the new length matches the existing length. Returns a mutable slice of the allocated space for the extension data.
- Else: Returns an error indicating the extension is already initialized and cannot be overwritten.
Note
ExtensionType
and Length
both occupy two bytes./// --- token/program-2022/src/extension/mod.rs --- /// Allocate some space for the extension in the TLV data fn alloc<V: Extension>( &mut self, length: usize, overwrite: bool, ) -> Result<&mut [u8], ProgramError> {' /// Account Type Check: if V::TYPE.get_account_type() != S::ACCOUNT_TYPE { return Err(ProgramError::InvalidAccountData); } /// Get TLV Data and Indices: let tlv_data = self.get_tlv_data_mut(); let TlvIndices { type_start, length_start, value_start, } = get_extension_indices::<V>(tlv_data, true)?; /// Check Buffer Length: if tlv_data[type_start..].len() < add_type_and_length_to_len(length) { return Err(ProgramError::InvalidAccountData); } /// Get Existing Extension Type: let extension_type = ExtensionType::try_from(&tlv_data[type_start..length_start])?; /// Handle Extension Initialization and Overwriting: if extension_type == ExtensionType::Uninitialized || overwrite { // write extension type let extension_type_array: [u8; 2] = V::TYPE.into(); let extension_type_ref = &mut tlv_data[type_start..length_start]; extension_type_ref.copy_from_slice(&extension_type_array); // write length let length_ref = pod_from_bytes_mut::<Length>(&mut tlv_data[length_start..value_start])?; // check that the length is the same if we're doing an alloc // with overwrite, otherwise a realloc should be done if overwrite && extension_type == V::TYPE && usize::from(*length_ref) != length { return Err(TokenError::InvalidLengthForAlloc.into()); } *length_ref = Length::try_from(length)?; let value_end = value_start.saturating_add(length); Ok(&mut tlv_data[value_start..value_end]) } else { // extension is already initialized, but no overwrite permission Err(TokenError::ExtensionAlreadyInitialized.into()) } } /// Extensions that can be applied to mints or accounts. Mint extensions must /// only be applied to mint accounts, and account extensions must only be /// applied to token holding accounts. #[repr(u16)] #[cfg_attr(feature = "serde-traits", derive(Serialize, Deserialize))] #[cfg_attr(feature = "serde-traits", serde(rename_all = "camelCase"))] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum ExtensionType { /// Used as padding if the account size would otherwise be 355, same as a /// multisig Uninitialized, /// ... } /// Length in TLV structure #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] #[repr(transparent)] pub struct Length(PodU16); /// --- libraries/pod/src/primitives.rs --- pub struct PodU16(pub [u8; 2]);
PodStateWithExtensionsMut.get_tlv_data_mut
returns the tlv_data
decoded from account’s data byte array./// --- token/program-2022/src/extension/mod.rs --- impl<'a, S: BaseState> BaseStateWithExtensionsMut<S> for PodStateWithExtensionsMut<'a, S> { fn get_tlv_data_mut(&mut self) -> &mut [u8] { self.tlv_data } fn get_account_type_mut(&mut self) -> &mut [u8] { self.account_type } }
Get Extenstion Indices
The
get_extension_indices
function is a utility function that retrieves the indices of specific fields (type, length, and value) within a TLV (Type-Length-Value) data structure. Because there may be multiple extensions stored in the tlv
, so it basically iterates each extension data, and perform corresponding checks and returns the needed extension’s location information.Parameters
tlv_data: &[u8]
: A slice of bytes representing the TLV data.
init: bool
: A flag indicating whether the function is in initialization mode.
Steps:
- Retrieve TLV Indices:
retrieves the indices for the current position in the TLV data.
- Check Data Length:
ensures that there is enough data for the current TLV entry.
- Retrieve Extension Type:
converts the byte slice to an
ExtensionType
and retrieves its associated account type.- Match Extension Type:
- Extension Found: If the current extension matches
V::TYPE
, return its indices. - Uninitialized Extension: If the extension is uninitialized and
init
is true, return its indices. Otherwise, return an error. - Mismatched Account Type: If the account type does not match
v_account_type
, return an error. Because there must be some error, because all extensions stored intlv
only corresponds to a single account, so there is error whether the specified account is incorrect, or the caller specifies wrong extension type. - Move to Next Extension: If none of the above conditions are met, move to the next extension.
/// --- token/program-2022/src/extension/mod.rs --- fn get_extension_indices<V: Extension>( tlv_data: &[u8], init: bool, ) -> Result<TlvIndices, ProgramError> { let mut start_index = 0; let v_account_type = V::TYPE.get_account_type(); while start_index < tlv_data.len() { /// Retrieve TLV Indices according to start_index: let tlv_indices = get_tlv_indices(start_index); /// Check Data Length: if tlv_data.len() < tlv_indices.value_start { return Err(ProgramError::InvalidAccountData); } /// Retrieve Extension Type: let extension_type = ExtensionType::try_from(&tlv_data[tlv_indices.type_start..tlv_indices.length_start])?; let account_type = extension_type.get_account_type(); /// Match Extension Type: if extension_type == V::TYPE { // found an instance of the extension that we're initializing, return! return Ok(tlv_indices); // got to an empty spot, init here, or error if we're searching, since // nothing is written after an Uninitialized spot } else if extension_type == ExtensionType::Uninitialized { if init { return Ok(tlv_indices); } else { return Err(TokenError::ExtensionNotFound.into()); } } else if v_account_type != account_type { return Err(TokenError::ExtensionTypeMismatch.into()); } else { let length = pod_from_bytes::<Length>( &tlv_data[tlv_indices.length_start..tlv_indices.value_start], )?; let value_end_index = tlv_indices.value_start.saturating_add(usize::from(*length)); start_index = value_end_index; } } Err(ProgramError::InvalidAccountData) } /// Helper function to get the current TlvIndices from the current spot fn get_tlv_indices(type_start: usize) -> TlvIndices { let length_start = type_start.saturating_add(size_of::<ExtensionType>()); let value_start = length_start.saturating_add(pod_get_packed_len::<Length>()); TlvIndices { type_start, length_start, value_start, } } /// --- libraries/pod/src/bytemuck.rs --- /// On-chain size of a `Pod` type pub const fn pod_get_packed_len<T: Pod>() -> usize { std::mem::size_of::<T>() }
Get Already Initialized Extension
get_extension_mut
is used to fetch certain exsited extension and deserialize it into V
.PodStateWithExtensionsMut
implements BaseStateWithExtensionsMut
which implements get_extension_mut
. So we can call PodStateWithExtensionsMut.get_extension_mut
directly./// Trait for mutable base state with extension pub trait BaseStateWithExtensionsMut<S: BaseState>: BaseStateWithExtensions<S> { /// ... /// Unpack a portion of the TLV data as the desired type that allows /// modifying the type fn get_extension_mut<V: Extension + Pod>(&mut self) -> Result<&mut V, ProgramError> { pod_from_bytes_mut::<V>(self.get_extension_bytes_mut::<V>()?) } /// ... } /// --- libraries/pod/src/bytemuck.rs --- /// Convert a slice of bytes into a mutable `Pod` (zero copy) pub fn pod_from_bytes_mut<T: Pod>(bytes: &mut [u8]) -> Result<&mut T, ProgramError> { bytemuck::try_from_bytes_mut(bytes).map_err(|_| ProgramError::InvalidArgument) }
get_extension_bytes_mut
first calls self.get_tlv_data_mut()
to get tlv
data of self. Then it calls get_extension_bytes_mut
to get corresponding extension mutable reference from the tlv
./// --- token/program-2022/src/extension/mod.rs --- /// Unpack a portion of the TLV data as the base mutable bytes fn get_extension_bytes_mut<V: Extension>(&mut self) -> Result<&mut [u8], ProgramError> { get_extension_bytes_mut::<S, V>(self.get_tlv_data_mut()) }
Recall that
PodStateWithExtensionsMut.get_tlv_data_mut
returns the tlv_data
decoded from account’s data byte array./// --- token/program-2022/src/extension/mod.rs --- impl<'a, S: BaseState> BaseStateWithExtensionsMut<S> for PodStateWithExtensionsMut<'a, S> { fn get_tlv_data_mut(&mut self) -> &mut [u8] { self.tlv_data } fn get_account_type_mut(&mut self) -> &mut [u8] { self.account_type } }
get_extension_bytes_mut
constructs corresponding extension mutable reference in the tlv
.Steps:
- Check extension’s corresponding account type matches current account type.
- Calls
get_extension_indices
to get corresponding tlv indices.
Note it sets the
init
to be false, which means it only fetches already exsited extension.- Check correctness of the data length.
- Construct mutable reference to the value and return.
/// --- token/program-2022/src/extension/mod.rs --- fn get_extension_bytes_mut<S: BaseState, V: Extension>( tlv_data: &mut [u8], ) -> Result<&mut [u8], ProgramError> { /// Check extension’s corresponding account type matches current account type. if V::TYPE.get_account_type() != S::ACCOUNT_TYPE { return Err(ProgramError::InvalidAccountData); } /// Calls get_extension_indices to get corresponding tlv indices. let TlvIndices { type_start: _, length_start, value_start, } = get_extension_indices::<V>(tlv_data, false)?; // get_extension_indices has checked that tlv_data is long enough to include // these indices let length = pod_from_bytes::<Length>(&tlv_data[length_start..value_start])?; let value_end = value_start.saturating_add(usize::from(*length)); if tlv_data.len() < value_end { return Err(ProgramError::InvalidAccountData); } /// Construct mutable reference to the value and return. Ok(&mut tlv_data[value_start..value_end]) }
Calculate Account Length
ExtensionType
implements try_calculate_account_len
, which can calculate account data size based on account type and needed extensions.Steps:
- Check whether there is no extension.
In this case, it will not use 165 bytes to store the account(
BASE_ACCOUNT_AND_TYPE_LENGTH
) but uses the BaseState
’s size.- Calculate
tlv
size.
where it filterss duplicate extensions.
- Sum
BASE_ACCOUNT_AND_TYPE_LENGTH
andtlv
size
It uses
BASE_ACCOUNT_AND_TYPE_LENGTH
to ensure the position of account type data is same amoung different account types.- Adjust multisig situation.
If the size of the account happens to be Multisig account’s data size (355 bytes), then it adds an extension type data space to help differentiate it from Multisig account.
/// --- token/program-2022/src/extension/mod.rs --- impl ExtensionType { /// ... /// Get the required account data length for the given ExtensionTypes /// /// Fails if any of the extension types has a variable length pub fn try_calculate_account_len<S: BaseState>( extension_types: &[Self], ) -> Result<usize, ProgramError> { if extension_types.is_empty() { Ok(S::SIZE_OF) } else { let extension_size = Self::try_get_total_tlv_len(extension_types)?; let total_len = extension_size.saturating_add(BASE_ACCOUNT_AND_TYPE_LENGTH); Ok(adjust_len_for_multisig(total_len)) } } /// ... }
ExtensionType.try_get_total_tlv_len
filters duplicate extensions and sums each extension’s data size.ExtensionType::try_get_tlv_len
calculates extension’s data size based extension type.add_type_and_length_to_len
sums type
, length
and value
‘s data sizes to get the size of the whole extension.pod_get_packed_len
calculates tlv
's value
data size./// --- token/program-2022/src/extension/mod.rs --- impl ExtensionType { /// ... /// Get the TLV length for a set of ExtensionTypes /// /// Fails if any of the extension types has a variable length fn try_get_total_tlv_len(extension_types: &[Self]) -> Result<usize, ProgramError> { // dedupe extensions let mut extensions = vec![]; for extension_type in extension_types { if !extensions.contains(&extension_type) { extensions.push(extension_type); } } extensions.iter().map(|e| e.try_get_tlv_len()).sum() } /// Get the TLV length for an ExtensionType /// /// Fails if the extension type has a variable length fn try_get_tlv_len(&self) -> Result<usize, ProgramError> { Ok(add_type_and_length_to_len(self.try_get_type_len()?)) } /// Get the data length of the type associated with the enum /// /// Fails if the extension type has a variable length fn try_get_type_len(&self) -> Result<usize, ProgramError> { if !self.sized() { return Err(ProgramError::InvalidArgument); } Ok(match self { ExtensionType::Uninitialized => 0, ExtensionType::TransferFeeConfig => pod_get_packed_len::<TransferFeeConfig>(), ExtensionType::TransferFeeAmount => pod_get_packed_len::<TransferFeeAmount>(), ExtensionType::MintCloseAuthority => pod_get_packed_len::<MintCloseAuthority>(), ExtensionType::ImmutableOwner => pod_get_packed_len::<ImmutableOwner>(), ExtensionType::ConfidentialTransferMint => { pod_get_packed_len::<ConfidentialTransferMint>() } ExtensionType::ConfidentialTransferAccount => { pod_get_packed_len::<ConfidentialTransferAccount>() } ExtensionType::DefaultAccountState => pod_get_packed_len::<DefaultAccountState>(), ExtensionType::MemoTransfer => pod_get_packed_len::<MemoTransfer>(), ExtensionType::NonTransferable => pod_get_packed_len::<NonTransferable>(), ExtensionType::InterestBearingConfig => pod_get_packed_len::<InterestBearingConfig>(), ExtensionType::CpiGuard => pod_get_packed_len::<CpiGuard>(), ExtensionType::PermanentDelegate => pod_get_packed_len::<PermanentDelegate>(), ExtensionType::NonTransferableAccount => pod_get_packed_len::<NonTransferableAccount>(), ExtensionType::TransferHook => pod_get_packed_len::<TransferHook>(), ExtensionType::TransferHookAccount => pod_get_packed_len::<TransferHookAccount>(), ExtensionType::ConfidentialTransferFeeConfig => { pod_get_packed_len::<ConfidentialTransferFeeConfig>() } ExtensionType::ConfidentialTransferFeeAmount => { pod_get_packed_len::<ConfidentialTransferFeeAmount>() } ExtensionType::MetadataPointer => pod_get_packed_len::<MetadataPointer>(), ExtensionType::TokenMetadata => unreachable!(), ExtensionType::GroupPointer => pod_get_packed_len::<GroupPointer>(), ExtensionType::TokenGroup => pod_get_packed_len::<TokenGroup>(), ExtensionType::GroupMemberPointer => pod_get_packed_len::<GroupMemberPointer>(), ExtensionType::TokenGroupMember => pod_get_packed_len::<TokenGroupMember>(), #[cfg(test)] ExtensionType::AccountPaddingTest => pod_get_packed_len::<AccountPaddingTest>(), #[cfg(test)] ExtensionType::MintPaddingTest => pod_get_packed_len::<MintPaddingTest>(), #[cfg(test)] ExtensionType::VariableLenMintTest => unreachable!(), }) } /// ... } /// Helper function to calculate exactly how many bytes a value will take up, /// given the value's length const fn add_type_and_length_to_len(value_len: usize) -> usize { value_len .saturating_add(size_of::<ExtensionType>()) .saturating_add(pod_get_packed_len::<Length>()) } /// --- libraries/pod/src/bytemuck.rs --- /// On-chain size of a `Pod` type pub const fn pod_get_packed_len<T: Pod>() -> usize { std::mem::size_of::<T>() }
Account Type Initialization
account_type
initialization happens during the initialization of the base state of account.Let’s take
Mint
account initialization as an example. Basically, we should first allocate space and create the Mint
account, and initializes extensions for it. Then we use initialize_mint2
instruction to initialize the base state of the Mint
account.In the last operation of the initialization, it calls
mint.init_account_type()
which initializes the account_type
./// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMint2](enum.TokenInstruction.html) instruction. pub fn process_initialize_mint2( accounts: &[AccountInfo], decimals: u8, mint_authority: &Pubkey, freeze_authority: PodCOption<Pubkey>, ) -> ProgramResult { Self::_process_initialize_mint(accounts, decimals, mint_authority, freeze_authority, false) } fn _process_initialize_mint( accounts: &[AccountInfo], decimals: u8, mint_authority: &Pubkey, freeze_authority: PodCOption<Pubkey>, rent_sysvar_account: bool, ) -> ProgramResult { let account_info_iter = &mut accounts.iter(); let mint_info = next_account_info(account_info_iter)?; let mint_data_len = mint_info.data_len(); let mut mint_data = mint_info.data.borrow_mut(); let rent = if rent_sysvar_account { Rent::from_account_info(next_account_info(account_info_iter)?)? } else { Rent::get()? }; if !rent.is_exempt(mint_info.lamports(), mint_data_len) { return Err(TokenError::NotRentExempt.into()); } let mut mint = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; let extension_types = mint.get_extension_types()?; if ExtensionType::try_calculate_account_len::<Mint>(&extension_types)? != mint_data_len { return Err(ProgramError::InvalidAccountData); } ExtensionType::check_for_invalid_mint_extension_combinations(&extension_types)?; if let Ok(default_account_state) = mint.get_extension_mut::<DefaultAccountState>() { let default_account_state = AccountState::try_from(default_account_state.state) .or(Err(ProgramError::InvalidAccountData))?; if default_account_state == AccountState::Frozen && freeze_authority.is_none() { return Err(TokenError::MintCannotFreeze.into()); } } mint.base.mint_authority = PodCOption::some(*mint_authority); mint.base.decimals = decimals; mint.base.is_initialized = PodBool::from_bool(true); mint.base.freeze_authority = freeze_authority; mint.init_account_type()?; Ok(()) }
BaseStateWithExtensionsMut<S: BaseState>.init_account_type
handles account_type
initialization. It first calls self.get_first_extension_type()
to get the first extension type, then calls self.get_account_type_mut()
to get mutable reference to the account_type
. Note it checks and ensures the corresponding account type of the first extension type matches the base account’s type. Finally, it updates the account_type
to be the account type./// Trait for mutable base state with extension pub trait BaseStateWithExtensionsMut<S: BaseState>: BaseStateWithExtensions<S> { /// ... /// Write the account type into the buffer, done during the base /// state initialization /// Noops if there is no room for an extension in the account, needed for /// pure base mints / accounts. fn init_account_type(&mut self) -> Result<(), ProgramError> { let first_extension_type = self.get_first_extension_type()?; let account_type = self.get_account_type_mut(); if !account_type.is_empty() { if let Some(extension_type) = first_extension_type { let account_type = extension_type.get_account_type(); if account_type != S::ACCOUNT_TYPE { return Err(TokenError::ExtensionBaseMismatch.into()); } } account_type[0] = S::ACCOUNT_TYPE.into(); } Ok(()) } /// ... }
get_first_extension_type
gets the first extension type./// --- token/program-2022/src/extension/mod.rs --- /// Trait for base state with extension pub trait BaseStateWithExtensions<S: BaseState> { /// ... /// Get just the first extension type, useful to track mixed initializations fn get_first_extension_type(&self) -> Result<Option<ExtensionType>, ProgramError> { get_first_extension_type(self.get_tlv_data()) } /// ... } fn get_first_extension_type(tlv_data: &[u8]) -> Result<Option<ExtensionType>, ProgramError> { if tlv_data.is_empty() { Ok(None) } else { let tlv_indices = get_tlv_indices(0); if tlv_data.len() <= tlv_indices.length_start { return Ok(None); } let extension_type = ExtensionType::try_from(&tlv_data[tlv_indices.type_start..tlv_indices.length_start])?; if extension_type == ExtensionType::Uninitialized { Ok(None) } else { Ok(Some(extension_type)) } } } /// Helper function to get the current TlvIndices from the current spot fn get_tlv_indices(type_start: usize) -> TlvIndices { let length_start = type_start.saturating_add(size_of::<ExtensionType>()); let value_start = length_start.saturating_add(pod_get_packed_len::<Length>()); TlvIndices { type_start, length_start, value_start, } }
Get Base State And TLV
PodStateWithExtensions::unpack
handles unpacking data to base
state and tlv
ignoring the account_type
.Step:
- Check the account is not multisig account and, ensure the data size matches the requirement of the account type.
- Split the data byte slice to get
base
state andrest
byte slice individually.
- Deserialize the
base
state byte array to corresponding type.
- Ensure the
base
state is initialized.
- Unpack the
rest
byte slice to gettlv
byte slice.
- Return
base
state andtlv
slice.
/// --- token/program-2022/src/extension/mod.rs --- impl<'data, S: BaseState + Pod> PodStateWithExtensions<'data, S> { /// Unpack base state, leaving the extension data as a slice /// /// Fails if the base state is not initialized. pub fn unpack(input: &'data [u8]) -> Result<Self, ProgramError> { check_min_len_and_not_multisig(input, S::SIZE_OF)?; let (base_data, rest) = input.split_at(S::SIZE_OF); let base = pod_from_bytes::<S>(base_data)?; if !base.is_initialized() { Err(ProgramError::UninitializedAccount) } else { let tlv_data = unpack_tlv_data::<S>(rest)?; Ok(Self { base, tlv_data }) } } } fn check_min_len_and_not_multisig(input: &[u8], minimum_len: usize) -> Result<(), ProgramError> { if input.len() == Multisig::LEN || input.len() < minimum_len { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }
unpack_tlv_data
calls type_and_tlv_indices
to get start index of account_type
and tlv
in the rest
slice. Then it checks the account_type
matches the current account’s type. Lastly it returns the tlv
slice to caller./// --- token/program-2022/src/extension/mod.rs --- fn unpack_tlv_data<S: BaseState>(rest: &[u8]) -> Result<&[u8], ProgramError> { if let Some((account_type_index, tlv_start_index)) = type_and_tlv_indices::<S>(rest)? { // type_and_tlv_indices() checks that returned indexes are within range let account_type = AccountType::try_from(rest[account_type_index]) .map_err(|_| ProgramError::InvalidAccountData)?; check_account_type::<S>(account_type)?; Ok(&rest[tlv_start_index..]) } else { Ok(&[]) } } fn check_account_type<S: BaseState>(account_type: AccountType) -> Result<(), ProgramError> { if account_type != S::ACCOUNT_TYPE { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }