SPL Token Program 2022 Extension

SPL Token Program 2022 Extension

Tags
Published
August 6, 2024
Author

Overview

The SPL (Solana Program Library) Token Program 2022 introduces several enhancements and features, with a significant focus on the extension functionality. These extensions allow developers to add additional capabilities and configurations to token accounts and mints without modifying the core SPL Token Program.

Data Structure

The program uses PodStateWithExtensionsMut to manage general data in data account.
  • base : the basic data, like Mint data defined in original Token Program.
  • account_type: type of the account
  • tlv_data: extension data, it uses tlv(type, length and value) to store tlv data.
For a data structure to be BaseState, it needs to implement three traits:
  1. BaseState : return the account type
  1. PackedSizeOf: return the size of the account data structure
  1. IsInitialized: return whether the account has been initialized
/// --- token/program-2022/src/extension/mod.rs --- /// Trait for base states, specifying the associated enum pub trait BaseState: PackedSizeOf + IsInitialized { /// Associated extension type enum, checked at the start of TLV entries const ACCOUNT_TYPE: AccountType; } /// Different kinds of accounts. Note that `Mint`, `Account`, and `Multisig` /// types are determined exclusively by the size of the account, and are not /// included in the account data. `AccountType` is only included if extensions /// have been initialized. #[repr(u8)] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum AccountType { /// Marker for 0 data Uninitialized, /// Mint account with additional extensions Mint, /// Token holding account with additional extensions Account, } /// Encapsulates mutable base state data (mint or account) with possible /// extensions, where the base state is Pod for zero-copy serde. #[derive(Debug, PartialEq)] pub struct PodStateWithExtensionsMut<'data, S: BaseState> { /// Unpacked base data pub base: &'data mut S, /// Writable account type account_type: &'data mut [u8], /// Slice of data containing all TLV data, deserialized on demand tlv_data: &'data mut [u8], } /// --- token/program-2022/src/state.rs --- /// Simplified version of the `Pack` trait which only gives the size of the /// packed struct. Useful when a function doesn't need a type to implement all /// of `Pack`, but a size is still needed. pub trait PackedSizeOf { /// The packed size of the struct const SIZE_OF: usize; } /// --- solana-program-1.18.11/src/program_pack.rs --- /// Check if a program account state is initialized pub trait IsInitialized { /// Is initialized fn is_initialized(&self) -> bool; }
 
There are three accounts type:
  • Uninitialized
  • Mint
  • Account
Different accounts have different data size. To make different kinds of account distinguishable, SPL uses the max data size as the common data size used by different kinds of accounts. So it can store the account type in the same position followed by tlv data.
tlv data consists of multiple type, length and value pairs. Because extensions can be added to account dynamically, so SPL can’t fix the specific extensions data structure. And it uses tlv format to store extensions flexibly.
  • When we need to add some extension to the account, SPL will iterate the tlv data to find an uninitialized area, and record the extension there.
  • If we want to update the extension’s data, then SPL also iterates the tlv data to match type of each extension to get the corresponding extension’s position, and update the data.
  • SPL ensures that each kind of extension can only occur once in the tlv.
notion image
 
We can see the introduction of this in the code:
/// Any account with extensions must be at least `Account::LEN`. Both mints and /// accounts can have extensions /// A mint with extensions that takes it past 165 could be indiscernible from an /// Account with an extension, even if we add the account type. For example, /// let's say we have: /// /// Account: 165 bytes... + [2, 0, 3, 0, 100, ....] /// ^ ^ ^ ^ /// acct type extension length data... /// /// Mint: 82 bytes... + 83 bytes of other extension data /// + [2, 0, 3, 0, 100, ....] /// (data in extension just happens to look like this) /// /// With this approach, we only start writing the TLV data after Account::LEN, /// which means we always know that the account type is going to be right after /// that. We do a special case checking for a Multisig length, because those /// aren't extensible under any circumstances. const BASE_ACCOUNT_LENGTH: usize = Account::LEN;

Example

Let’s use setting mint_close_authority process as an example to illustrate how the extension works.
process_initialize_mint_close_authority is the entry to initialize mint_close_authority of a Mint account.
Inside:
  1. get Mint account
  1. get account data in byte array format
  1. deserialize the account data
  1. initialize extension
// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMintCloseAuthority](enum.TokenInstruction.html) /// instruction pub fn process_initialize_mint_close_authority( accounts: &[AccountInfo], close_authority: PodCOption<Pubkey>, ) -> ProgramResult { /// get Mint account let account_info_iter = &mut accounts.iter(); let mint_account_info = next_account_info(account_info_iter)?; /// get account data in byte array format let mut mint_data = mint_account_info.data.borrow_mut(); /// deserialize the account data(byte array) let mut mint = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; /// initialize extension let extension = mint.init_extension::<MintCloseAuthority>(true)?; extension.close_authority = close_authority.try_into()?; Ok(()) }

unpack_uninitialized

unpack_uninitialized is a function of PodStateWithExtensionsMut::<PodMint> which is used to deserialize byte array to get PodStateWithExtensionsMut::<PodMint> which includes:
  1. base: is PodMint whose structure is exactly same with the Mint defined in original token program. (This is for compatibility consideration). Also it ensures that the base state is not already initialized.
  1. account_type: account type
  1. tlv: extension data in tlv format
/// [Mint] data stored as a Pod type #[repr(C)] #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] pub struct PodMint { /// Optional authority used to mint new tokens. The mint authority may only /// be provided during mint creation. If no mint authority is present /// then the mint has a fixed supply and no further tokens may be /// minted. pub mint_authority: PodCOption<Pubkey>, /// Total supply of tokens. pub supply: PodU64, /// Number of base 10 digits to the right of the decimal place. pub decimals: u8, /// If `true`, this structure has been initialized pub is_initialized: PodBool, /// Optional authority to freeze token accounts. pub freeze_authority: PodCOption<Pubkey>, } impl IsInitialized for PodMint { fn is_initialized(&self) -> bool { self.is_initialized.into() } }
/// ---token/program/src/state.rs--- #[repr(C)] #[derive(Clone, Copy, Debug, Default, PartialEq)] pub struct Mint { /// Optional authority used to mint new tokens. The mint authority may only /// be provided during mint creation. If no mint authority is present /// then the mint has a fixed supply and no further tokens may be /// minted. pub mint_authority: COption<Pubkey>, /// Total supply of tokens. pub supply: u64, /// Number of base 10 digits to the right of the decimal place. pub decimals: u8, /// Is `true` if this structure has been initialized pub is_initialized: bool, /// Optional authority to freeze token accounts. pub freeze_authority: COption<Pubkey>, }
 
Inside the unpack_uninitialized:
Steps:
  1. Check Minimum Length and Multisig Length:
    1. ensures that the input slice has a minimum length of S::SIZE_OF bytes and is not exactly the length of a multisig account (Multisig::LEN). If these conditions are not met, the function returns an InvalidAccountData error.
  1. Split the Input Slice:
    1. The input slice is split into two parts:
      • base_data: The first S::SIZE_OF bytes, which represent the base state.
      • rest: The remaining bytes, which potentially contain TLV (Type-Length-Value) data.
  1. Deserialize the Base State:
    1. The base_data slice is deserialized into a base state of type S using the pod_from_bytes_mut function. This function ensures that the base state is correctly interpreted as a Pod (Plain Old Data) type.
  1. Check if the Base State is Already Initialized:
    1. The function checks if the base state is already initialized by calling the is_initialized method on the deserialized base state. If the base state is already initialized, the function returns an AlreadyInUse error.
  1. Unpack the Account Type and TLV Data:
    1. The remaining slice (rest) is passed to the unpack_uninitialized_type_and_tlv_data_mut function, which unpacks the account type and TLV data from the slice. This function ensures that the account type is Uninitialized.
  1. Create and Return the State with Extensions:
      • A new instance of PodStateWithExtensionsMut is created with the base state, account type, and TLV data.
      • The function calls check_account_type_matches_extension_type to ensure that the account type in the TLV data matches the base state's account type.
      • Finally, the function returns the newly created state.
/// --- token/program-2022/src/extension/mod.rs --- /// Encapsulates mutable base state data (mint or account) with possible /// extensions, where the base state is Pod for zero-copy serde. #[derive(Debug, PartialEq)] pub struct PodStateWithExtensionsMut<'data, S: BaseState> { /// Unpacked base data pub base: &'data mut S, /// Writable account type account_type: &'data mut [u8], /// Slice of data containing all TLV data, deserialized on demand tlv_data: &'data mut [u8], } impl<'data, S: BaseState + Pod> PodStateWithExtensionsMut<'data, S> { /// ... /// Unpack an uninitialized base state, leaving the extension data as a /// mutable slice /// /// Fails if the base state has already been initialized. pub fn unpack_uninitialized(input: &'data mut [u8]) -> Result<Self, ProgramError> { /// Check Minimum Length and Multisig Length: check_min_len_and_not_multisig(input, S::SIZE_OF)?; /// Split the Input Slice: let (base_data, rest) = input.split_at_mut(S::SIZE_OF); /// Deserialize the Base State: let base = pod_from_bytes_mut::<S>(base_data)?; /// Check if the Base State is Already Initialized: if base.is_initialized() { return Err(TokenError::AlreadyInUse.into()); } /// Unpack the Account Type and TLV Data: let (account_type, tlv_data) = unpack_uninitialized_type_and_tlv_data_mut::<S>(rest)?; /// Create and Return the State with Extensions: let state = Self { base, account_type, tlv_data, }; state.check_account_type_matches_extension_type()?; Ok(state) } /// ... }
 
check_min_len_and_not_multisig checks the length of the input is not multisig length.
/// --- token/program-2022/src/extension/mod.rs --- fn check_min_len_and_not_multisig(input: &[u8], minimum_len: usize) -> Result<(), ProgramError> { if input.len() == Multisig::LEN || input.len() < minimum_len { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }
 
pod_from_bytes_mut converts the raw bytes to corresponding structure.
/// --- libraries/pod/src/bytemuck.rs --- /// Convert a slice of bytes into a mutable `Pod` (zero copy) pub fn pod_from_bytes_mut<T: Pod>(bytes: &mut [u8]) -> Result<&mut T, ProgramError> { bytemuck::try_from_bytes_mut(bytes).map_err(|_| ProgramError::InvalidArgument) }
 

Get account type and tlv

unpack_uninitialized_type_and_tlv_data_mut decodes the rest bytes to get AccountType and tlv bytes array. It also checks that the account hasn’t been initialized.
/// --- token/program-2022/src/extension/mod.rs --- fn unpack_uninitialized_type_and_tlv_data_mut<S: BaseState>( rest: &mut [u8], ) -> Result<(&mut [u8], &mut [u8]), ProgramError> { unpack_type_and_tlv_data_with_check_mut::<S, _>(rest, |account_type| { if account_type != AccountType::Uninitialized { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }) }
 
unpack_type_and_tlv_data_with_check_mut uses type_and_tlv_indices function to get the start index of account_type and tlv data in the rest. Then it will use the check_fn to check the validity of the account_type and returns account_type and tlv byte array.
Note that AccountType is represented using 1 byte. So there is some redundant code here (split the rest, and read the account_type by range.
/// --- token/program-2022/src/extension/mod.rs --- fn unpack_type_and_tlv_data_with_check_mut< S: BaseState, F: Fn(AccountType) -> Result<(), ProgramError>, >( rest: &mut [u8], check_fn: F, ) -> Result<(&mut [u8], &mut [u8]), ProgramError> { if let Some((account_type_index, tlv_start_index)) = type_and_tlv_indices::<S>(rest)? { // type_and_tlv_indices() checks that returned indexes are within range let account_type = AccountType::try_from(rest[account_type_index]) .map_err(|_| ProgramError::InvalidAccountData)?; check_fn(account_type)?; let (account_type, tlv_data) = rest.split_at_mut(tlv_start_index); Ok(( &mut account_type[account_type_index..tlv_start_index], tlv_data, )) } else { Ok((&mut [], &mut [])) } } /// --- token/program-2022/src/extension/mod.rs --- /// Different kinds of accounts. Note that `Mint`, `Account`, and `Multisig` /// types are determined exclusively by the size of the account, and are not /// included in the account data. `AccountType` is only included if extensions /// have been initialized. #[repr(u8)] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum AccountType { /// Marker for 0 data Uninitialized, /// Mint account with additional extensions Mint, /// Token holding account with additional extensions Account, }
 
type_and_tlv_indices reads the start indexes of account_type and tlv.
  • if the rest_input is empty, then it just returns Ok(None), the caller will know that there is no extension in this account.
  • else
    • gets the start indexes of both account_type and tlv based on the data structure of account. Note that Mint account has less data size, so there are 83 padding zeros, so the start index of account_type is 83.
    • ensures tlv data space is not empty.
    • check the padding part is filled with zero.
/// --- token/program-2022/src/extension/mod.rs --- fn type_and_tlv_indices<S: BaseState>( rest_input: &[u8], ) -> Result<Option<(usize, usize)>, ProgramError> { if rest_input.is_empty() { Ok(None) } else { /// get the start index of account_type let account_type_index = BASE_ACCOUNT_LENGTH.saturating_sub(S::SIZE_OF); // check padding is all zeroes let tlv_start_index = account_type_index.saturating_add(size_of::<AccountType>()); if rest_input.len() <= tlv_start_index { return Err(ProgramError::InvalidAccountData); } if rest_input[..account_type_index] != vec![0; account_type_index] { Err(ProgramError::InvalidAccountData) } else { Ok(Some((account_type_index, tlv_start_index))) } } }

Initialize Extension

process_initialize_mint_close_authority calls init_extension to initialize extension.
/// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMintCloseAuthority](enum.TokenInstruction.html) /// instruction pub fn process_initialize_mint_close_authority( accounts: &[AccountInfo], close_authority: PodCOption<Pubkey>, ) -> ProgramResult { let account_info_iter = &mut accounts.iter(); let mint_account_info = next_account_info(account_info_iter)?; let mut mint_data = mint_account_info.data.borrow_mut(); let mut mint: PodStateWithExtensionsMut<PodMint> = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; let extension = mint.init_extension::<MintCloseAuthority>(true)?; extension.close_authority = close_authority.try_into()?; Ok(()) } /// --- token/program-2022/src/extension/mint_close_authority.rs --- /// Close authority extension data for mints. #[repr(C)] #[cfg_attr(feature = "serde-traits", derive(Serialize, Deserialize))] #[cfg_attr(feature = "serde-traits", serde(rename_all = "camelCase"))] #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] pub struct MintCloseAuthority { /// Optional authority to close the mint pub close_authority: OptionalNonZeroPubkey, }
 
The init_extension is responsible for initializing an extension. This function checks if the extension already exists in the TLV (Type-Length-Value) buffer and either initializes it with a default value or returns an error if the extension is already present and overwrite is not allowed.
Steps:
  1. Length Calculation
    1. Calculates the length of the packed extension type V.
  1. Buffer Allocation
    1. Allocates space for the extension in the TLV buffer. It calls the alloc method with the calculated length and overwrite flag. If successful, returns a mutable slice of the allocated buffer. If it fails (e.g., if there's not enough space or the extension already exists and overwrite is not allowed), it returns an error.
  1. Extension Initialization
      • Converts the allocated buffer into a mutable reference of type V and initializes it with the default value of V.
      • pod_from_bytes_mut::<V>(buffer)?: Converts the buffer into a mutable reference of type V. This operation can fail if the buffer length does not match the expected size of V.
      • extension_ref = V::default();: Assigns the default value of V to the allocated buffer, effectively initializing the extension.
  1. Return Value
    1. returns mutable reference to extension data space stored in the account(extension_ref) which allows caller to modify it.
/// --- token/program-2022/src/extension/mod.rs --- /// Packs the default extension data into an open slot if not already found /// in the data buffer. If extension is already found in the buffer, it /// overwrites the existing extension with the default state if /// `overwrite` is set. If extension found, but `overwrite` is not set, /// it returns error. fn init_extension<V: Extension + Pod + Default>( &mut self, overwrite: bool, ) -> Result<&mut V, ProgramError> { /// Length Calculation let length = pod_get_packed_len::<V>(); /// Buffer Allocation let buffer = self.alloc::<V>(length, overwrite)?; /// Extension Initialization let extension_ref = pod_from_bytes_mut::<V>(buffer)?; *extension_ref = V::default(); /// Return Value Ok(extension_ref) }
 
The alloc function allocates space for a given extension in a TLV (Type-Length-Value) structure within an account's data buffer, ensuring the extension is placed correctly and handling scenarios where the extension already exists.
Steps:
  1. Account Type Check:
    1. Ensures the extension's associated account type matches the base state's account type.
  1. Get TLV Data and Indices:
      • Retrieves the mutable TLV data buffer.
      • Determines the start positions for the type, length, and value fields in the TLV entry.
  1. Check Buffer Length:
    1. Ensures there is enough space in the TLV buffer to accommodate the new extension.
  1. Get Existing Extension Type:
  1. Handle Extension Initialization and Overwriting:
    1. If Uninitialized or Overwrite: Writes the extension type and length into the TLV data.If overwriting, ensures the new length matches the existing length. Returns a mutable slice of the allocated space for the extension data.
    2. Else: Returns an error indicating the extension is already initialized and cannot be overwritten.
 
Note ExtensionType and Length both occupy two bytes.
/// --- token/program-2022/src/extension/mod.rs --- /// Allocate some space for the extension in the TLV data fn alloc<V: Extension>( &mut self, length: usize, overwrite: bool, ) -> Result<&mut [u8], ProgramError> {' /// Account Type Check: if V::TYPE.get_account_type() != S::ACCOUNT_TYPE { return Err(ProgramError::InvalidAccountData); } /// Get TLV Data and Indices: let tlv_data = self.get_tlv_data_mut(); let TlvIndices { type_start, length_start, value_start, } = get_extension_indices::<V>(tlv_data, true)?; /// Check Buffer Length: if tlv_data[type_start..].len() < add_type_and_length_to_len(length) { return Err(ProgramError::InvalidAccountData); } /// Get Existing Extension Type: let extension_type = ExtensionType::try_from(&tlv_data[type_start..length_start])?; /// Handle Extension Initialization and Overwriting: if extension_type == ExtensionType::Uninitialized || overwrite { // write extension type let extension_type_array: [u8; 2] = V::TYPE.into(); let extension_type_ref = &mut tlv_data[type_start..length_start]; extension_type_ref.copy_from_slice(&extension_type_array); // write length let length_ref = pod_from_bytes_mut::<Length>(&mut tlv_data[length_start..value_start])?; // check that the length is the same if we're doing an alloc // with overwrite, otherwise a realloc should be done if overwrite && extension_type == V::TYPE && usize::from(*length_ref) != length { return Err(TokenError::InvalidLengthForAlloc.into()); } *length_ref = Length::try_from(length)?; let value_end = value_start.saturating_add(length); Ok(&mut tlv_data[value_start..value_end]) } else { // extension is already initialized, but no overwrite permission Err(TokenError::ExtensionAlreadyInitialized.into()) } } /// Extensions that can be applied to mints or accounts. Mint extensions must /// only be applied to mint accounts, and account extensions must only be /// applied to token holding accounts. #[repr(u16)] #[cfg_attr(feature = "serde-traits", derive(Serialize, Deserialize))] #[cfg_attr(feature = "serde-traits", serde(rename_all = "camelCase"))] #[derive(Clone, Copy, Debug, PartialEq, TryFromPrimitive, IntoPrimitive)] pub enum ExtensionType { /// Used as padding if the account size would otherwise be 355, same as a /// multisig Uninitialized, /// ... } /// Length in TLV structure #[derive(Clone, Copy, Debug, Default, PartialEq, Pod, Zeroable)] #[repr(transparent)] pub struct Length(PodU16); /// --- libraries/pod/src/primitives.rs --- pub struct PodU16(pub [u8; 2]);
 
PodStateWithExtensionsMut.get_tlv_data_mut returns the tlv_data decoded from account’s data byte array.
/// --- token/program-2022/src/extension/mod.rs --- impl<'a, S: BaseState> BaseStateWithExtensionsMut<S> for PodStateWithExtensionsMut<'a, S> { fn get_tlv_data_mut(&mut self) -> &mut [u8] { self.tlv_data } fn get_account_type_mut(&mut self) -> &mut [u8] { self.account_type } }

Get Extenstion Indices

The get_extension_indices function is a utility function that retrieves the indices of specific fields (type, length, and value) within a TLV (Type-Length-Value) data structure. Because there may be multiple extensions stored in the tlv, so it basically iterates each extension data, and perform corresponding checks and returns the needed extension’s location information.
Parameters
  • tlv_data: &[u8]: A slice of bytes representing the TLV data.
  • init: bool: A flag indicating whether the function is in initialization mode.
Steps:
  1. Retrieve TLV Indices:
    1. retrieves the indices for the current position in the TLV data.
  1. Check Data Length:
    1. ensures that there is enough data for the current TLV entry.
  1. Retrieve Extension Type:
    1. converts the byte slice to an ExtensionType and retrieves its associated account type.
  1. Match Extension Type:
      • Extension Found: If the current extension matches V::TYPE, return its indices.
      • Uninitialized Extension: If the extension is uninitialized and init is true, return its indices. Otherwise, return an error.
      • Mismatched Account Type: If the account type does not match v_account_type, return an error. Because there must be some error, because all extensions stored in tlv only corresponds to a single account, so there is error whether the specified account is incorrect, or the caller specifies wrong extension type.
      • Move to Next Extension: If none of the above conditions are met, move to the next extension.
/// --- token/program-2022/src/extension/mod.rs --- fn get_extension_indices<V: Extension>( tlv_data: &[u8], init: bool, ) -> Result<TlvIndices, ProgramError> { let mut start_index = 0; let v_account_type = V::TYPE.get_account_type(); while start_index < tlv_data.len() { /// Retrieve TLV Indices according to start_index: let tlv_indices = get_tlv_indices(start_index); /// Check Data Length: if tlv_data.len() < tlv_indices.value_start { return Err(ProgramError::InvalidAccountData); } /// Retrieve Extension Type: let extension_type = ExtensionType::try_from(&tlv_data[tlv_indices.type_start..tlv_indices.length_start])?; let account_type = extension_type.get_account_type(); /// Match Extension Type: if extension_type == V::TYPE { // found an instance of the extension that we're initializing, return! return Ok(tlv_indices); // got to an empty spot, init here, or error if we're searching, since // nothing is written after an Uninitialized spot } else if extension_type == ExtensionType::Uninitialized { if init { return Ok(tlv_indices); } else { return Err(TokenError::ExtensionNotFound.into()); } } else if v_account_type != account_type { return Err(TokenError::ExtensionTypeMismatch.into()); } else { let length = pod_from_bytes::<Length>( &tlv_data[tlv_indices.length_start..tlv_indices.value_start], )?; let value_end_index = tlv_indices.value_start.saturating_add(usize::from(*length)); start_index = value_end_index; } } Err(ProgramError::InvalidAccountData) } /// Helper function to get the current TlvIndices from the current spot fn get_tlv_indices(type_start: usize) -> TlvIndices { let length_start = type_start.saturating_add(size_of::<ExtensionType>()); let value_start = length_start.saturating_add(pod_get_packed_len::<Length>()); TlvIndices { type_start, length_start, value_start, } } /// --- libraries/pod/src/bytemuck.rs --- /// On-chain size of a `Pod` type pub const fn pod_get_packed_len<T: Pod>() -> usize { std::mem::size_of::<T>() }

Get Already Initialized Extension

get_extension_mut is used to fetch certain exsited extension and deserialize it into V.
PodStateWithExtensionsMut implements BaseStateWithExtensionsMut which implements get_extension_mut. So we can call PodStateWithExtensionsMut.get_extension_mut directly.
/// Trait for mutable base state with extension pub trait BaseStateWithExtensionsMut<S: BaseState>: BaseStateWithExtensions<S> { /// ... /// Unpack a portion of the TLV data as the desired type that allows /// modifying the type fn get_extension_mut<V: Extension + Pod>(&mut self) -> Result<&mut V, ProgramError> { pod_from_bytes_mut::<V>(self.get_extension_bytes_mut::<V>()?) } /// ... } /// --- libraries/pod/src/bytemuck.rs --- /// Convert a slice of bytes into a mutable `Pod` (zero copy) pub fn pod_from_bytes_mut<T: Pod>(bytes: &mut [u8]) -> Result<&mut T, ProgramError> { bytemuck::try_from_bytes_mut(bytes).map_err(|_| ProgramError::InvalidArgument) }
 
get_extension_bytes_mut first calls self.get_tlv_data_mut() to get tlv data of self. Then it calls get_extension_bytes_mut to get corresponding extension mutable reference from the tlv.
/// --- token/program-2022/src/extension/mod.rs --- /// Unpack a portion of the TLV data as the base mutable bytes fn get_extension_bytes_mut<V: Extension>(&mut self) -> Result<&mut [u8], ProgramError> { get_extension_bytes_mut::<S, V>(self.get_tlv_data_mut()) }
 
Recall that PodStateWithExtensionsMut.get_tlv_data_mut returns the tlv_data decoded from account’s data byte array.
/// --- token/program-2022/src/extension/mod.rs --- impl<'a, S: BaseState> BaseStateWithExtensionsMut<S> for PodStateWithExtensionsMut<'a, S> { fn get_tlv_data_mut(&mut self) -> &mut [u8] { self.tlv_data } fn get_account_type_mut(&mut self) -> &mut [u8] { self.account_type } }
 
get_extension_bytes_mut constructs corresponding extension mutable reference in the tlv.
Steps:
  1. Check extension’s corresponding account type matches current account type.
  1. Calls get_extension_indices to get corresponding tlv indices.
    1. Note it sets the init to be false, which means it only fetches already exsited extension.
  1. Check correctness of the data length.
  1. Construct mutable reference to the value and return.
/// --- token/program-2022/src/extension/mod.rs --- fn get_extension_bytes_mut<S: BaseState, V: Extension>( tlv_data: &mut [u8], ) -> Result<&mut [u8], ProgramError> { /// Check extension’s corresponding account type matches current account type. if V::TYPE.get_account_type() != S::ACCOUNT_TYPE { return Err(ProgramError::InvalidAccountData); } /// Calls get_extension_indices to get corresponding tlv indices. let TlvIndices { type_start: _, length_start, value_start, } = get_extension_indices::<V>(tlv_data, false)?; // get_extension_indices has checked that tlv_data is long enough to include // these indices let length = pod_from_bytes::<Length>(&tlv_data[length_start..value_start])?; let value_end = value_start.saturating_add(usize::from(*length)); if tlv_data.len() < value_end { return Err(ProgramError::InvalidAccountData); } /// Construct mutable reference to the value and return. Ok(&mut tlv_data[value_start..value_end]) }

Calculate Account Length

ExtensionType implements try_calculate_account_len, which can calculate account data size based on account type and needed extensions.
Steps:
  1. Check whether there is no extension.
    1. In this case, it will not use 165 bytes to store the account(BASE_ACCOUNT_AND_TYPE_LENGTH) but uses the BaseState’s size.
  1. Calculate tlv size.
    1. where it filterss duplicate extensions.
  1. Sum BASE_ACCOUNT_AND_TYPE_LENGTH and tlv size
    1. It uses BASE_ACCOUNT_AND_TYPE_LENGTH to ensure the position of account type data is same amoung different account types.
  1. Adjust multisig situation.
    1. If the size of the account happens to be Multisig account’s data size (355 bytes), then it adds an extension type data space to help differentiate it from Multisig account.
/// --- token/program-2022/src/extension/mod.rs --- impl ExtensionType { /// ... /// Get the required account data length for the given ExtensionTypes /// /// Fails if any of the extension types has a variable length pub fn try_calculate_account_len<S: BaseState>( extension_types: &[Self], ) -> Result<usize, ProgramError> { if extension_types.is_empty() { Ok(S::SIZE_OF) } else { let extension_size = Self::try_get_total_tlv_len(extension_types)?; let total_len = extension_size.saturating_add(BASE_ACCOUNT_AND_TYPE_LENGTH); Ok(adjust_len_for_multisig(total_len)) } } /// ... }
 
ExtensionType.try_get_total_tlv_len filters duplicate extensions and sums each extension’s data size.
ExtensionType::try_get_tlv_len calculates extension’s data size based extension type.
add_type_and_length_to_len sums type, length and value ‘s data sizes to get the size of the whole extension.
pod_get_packed_len calculates tlv's value data size.
/// --- token/program-2022/src/extension/mod.rs --- impl ExtensionType { /// ... /// Get the TLV length for a set of ExtensionTypes /// /// Fails if any of the extension types has a variable length fn try_get_total_tlv_len(extension_types: &[Self]) -> Result<usize, ProgramError> { // dedupe extensions let mut extensions = vec![]; for extension_type in extension_types { if !extensions.contains(&extension_type) { extensions.push(extension_type); } } extensions.iter().map(|e| e.try_get_tlv_len()).sum() } /// Get the TLV length for an ExtensionType /// /// Fails if the extension type has a variable length fn try_get_tlv_len(&self) -> Result<usize, ProgramError> { Ok(add_type_and_length_to_len(self.try_get_type_len()?)) } /// Get the data length of the type associated with the enum /// /// Fails if the extension type has a variable length fn try_get_type_len(&self) -> Result<usize, ProgramError> { if !self.sized() { return Err(ProgramError::InvalidArgument); } Ok(match self { ExtensionType::Uninitialized => 0, ExtensionType::TransferFeeConfig => pod_get_packed_len::<TransferFeeConfig>(), ExtensionType::TransferFeeAmount => pod_get_packed_len::<TransferFeeAmount>(), ExtensionType::MintCloseAuthority => pod_get_packed_len::<MintCloseAuthority>(), ExtensionType::ImmutableOwner => pod_get_packed_len::<ImmutableOwner>(), ExtensionType::ConfidentialTransferMint => { pod_get_packed_len::<ConfidentialTransferMint>() } ExtensionType::ConfidentialTransferAccount => { pod_get_packed_len::<ConfidentialTransferAccount>() } ExtensionType::DefaultAccountState => pod_get_packed_len::<DefaultAccountState>(), ExtensionType::MemoTransfer => pod_get_packed_len::<MemoTransfer>(), ExtensionType::NonTransferable => pod_get_packed_len::<NonTransferable>(), ExtensionType::InterestBearingConfig => pod_get_packed_len::<InterestBearingConfig>(), ExtensionType::CpiGuard => pod_get_packed_len::<CpiGuard>(), ExtensionType::PermanentDelegate => pod_get_packed_len::<PermanentDelegate>(), ExtensionType::NonTransferableAccount => pod_get_packed_len::<NonTransferableAccount>(), ExtensionType::TransferHook => pod_get_packed_len::<TransferHook>(), ExtensionType::TransferHookAccount => pod_get_packed_len::<TransferHookAccount>(), ExtensionType::ConfidentialTransferFeeConfig => { pod_get_packed_len::<ConfidentialTransferFeeConfig>() } ExtensionType::ConfidentialTransferFeeAmount => { pod_get_packed_len::<ConfidentialTransferFeeAmount>() } ExtensionType::MetadataPointer => pod_get_packed_len::<MetadataPointer>(), ExtensionType::TokenMetadata => unreachable!(), ExtensionType::GroupPointer => pod_get_packed_len::<GroupPointer>(), ExtensionType::TokenGroup => pod_get_packed_len::<TokenGroup>(), ExtensionType::GroupMemberPointer => pod_get_packed_len::<GroupMemberPointer>(), ExtensionType::TokenGroupMember => pod_get_packed_len::<TokenGroupMember>(), #[cfg(test)] ExtensionType::AccountPaddingTest => pod_get_packed_len::<AccountPaddingTest>(), #[cfg(test)] ExtensionType::MintPaddingTest => pod_get_packed_len::<MintPaddingTest>(), #[cfg(test)] ExtensionType::VariableLenMintTest => unreachable!(), }) } /// ... } /// Helper function to calculate exactly how many bytes a value will take up, /// given the value's length const fn add_type_and_length_to_len(value_len: usize) -> usize { value_len .saturating_add(size_of::<ExtensionType>()) .saturating_add(pod_get_packed_len::<Length>()) } /// --- libraries/pod/src/bytemuck.rs --- /// On-chain size of a `Pod` type pub const fn pod_get_packed_len<T: Pod>() -> usize { std::mem::size_of::<T>() }

Account Type Initialization

account_type initialization happens during the initialization of the base state of account.
Let’s take Mint account initialization as an example. Basically, we should first allocate space and create the Mint account, and initializes extensions for it. Then we use initialize_mint2 instruction to initialize the base state of the Mint account.
In the last operation of the initialization, it calls mint.init_account_type() which initializes the account_type.
/// --- token/program-2022/src/processor.rs --- /// Processes an [InitializeMint2](enum.TokenInstruction.html) instruction. pub fn process_initialize_mint2( accounts: &[AccountInfo], decimals: u8, mint_authority: &Pubkey, freeze_authority: PodCOption<Pubkey>, ) -> ProgramResult { Self::_process_initialize_mint(accounts, decimals, mint_authority, freeze_authority, false) } fn _process_initialize_mint( accounts: &[AccountInfo], decimals: u8, mint_authority: &Pubkey, freeze_authority: PodCOption<Pubkey>, rent_sysvar_account: bool, ) -> ProgramResult { let account_info_iter = &mut accounts.iter(); let mint_info = next_account_info(account_info_iter)?; let mint_data_len = mint_info.data_len(); let mut mint_data = mint_info.data.borrow_mut(); let rent = if rent_sysvar_account { Rent::from_account_info(next_account_info(account_info_iter)?)? } else { Rent::get()? }; if !rent.is_exempt(mint_info.lamports(), mint_data_len) { return Err(TokenError::NotRentExempt.into()); } let mut mint = PodStateWithExtensionsMut::<PodMint>::unpack_uninitialized(&mut mint_data)?; let extension_types = mint.get_extension_types()?; if ExtensionType::try_calculate_account_len::<Mint>(&extension_types)? != mint_data_len { return Err(ProgramError::InvalidAccountData); } ExtensionType::check_for_invalid_mint_extension_combinations(&extension_types)?; if let Ok(default_account_state) = mint.get_extension_mut::<DefaultAccountState>() { let default_account_state = AccountState::try_from(default_account_state.state) .or(Err(ProgramError::InvalidAccountData))?; if default_account_state == AccountState::Frozen && freeze_authority.is_none() { return Err(TokenError::MintCannotFreeze.into()); } } mint.base.mint_authority = PodCOption::some(*mint_authority); mint.base.decimals = decimals; mint.base.is_initialized = PodBool::from_bool(true); mint.base.freeze_authority = freeze_authority; mint.init_account_type()?; Ok(()) }
 
BaseStateWithExtensionsMut<S: BaseState>.init_account_type handles account_type initialization. It first calls self.get_first_extension_type() to get the first extension type, then calls self.get_account_type_mut() to get mutable reference to the account_type. Note it checks and ensures the corresponding account type of the first extension type matches the base account’s type. Finally, it updates the account_type to be the account type.
/// Trait for mutable base state with extension pub trait BaseStateWithExtensionsMut<S: BaseState>: BaseStateWithExtensions<S> { /// ... /// Write the account type into the buffer, done during the base /// state initialization /// Noops if there is no room for an extension in the account, needed for /// pure base mints / accounts. fn init_account_type(&mut self) -> Result<(), ProgramError> { let first_extension_type = self.get_first_extension_type()?; let account_type = self.get_account_type_mut(); if !account_type.is_empty() { if let Some(extension_type) = first_extension_type { let account_type = extension_type.get_account_type(); if account_type != S::ACCOUNT_TYPE { return Err(TokenError::ExtensionBaseMismatch.into()); } } account_type[0] = S::ACCOUNT_TYPE.into(); } Ok(()) } /// ... }
 
get_first_extension_type gets the first extension type.
/// --- token/program-2022/src/extension/mod.rs --- /// Trait for base state with extension pub trait BaseStateWithExtensions<S: BaseState> { /// ... /// Get just the first extension type, useful to track mixed initializations fn get_first_extension_type(&self) -> Result<Option<ExtensionType>, ProgramError> { get_first_extension_type(self.get_tlv_data()) } /// ... } fn get_first_extension_type(tlv_data: &[u8]) -> Result<Option<ExtensionType>, ProgramError> { if tlv_data.is_empty() { Ok(None) } else { let tlv_indices = get_tlv_indices(0); if tlv_data.len() <= tlv_indices.length_start { return Ok(None); } let extension_type = ExtensionType::try_from(&tlv_data[tlv_indices.type_start..tlv_indices.length_start])?; if extension_type == ExtensionType::Uninitialized { Ok(None) } else { Ok(Some(extension_type)) } } } /// Helper function to get the current TlvIndices from the current spot fn get_tlv_indices(type_start: usize) -> TlvIndices { let length_start = type_start.saturating_add(size_of::<ExtensionType>()); let value_start = length_start.saturating_add(pod_get_packed_len::<Length>()); TlvIndices { type_start, length_start, value_start, } }

Get Base State And TLV

PodStateWithExtensions::unpack handles unpacking data to base state and tlv ignoring the account_type.
Step:
  • Check the account is not multisig account and, ensure the data size matches the requirement of the account type.
  • Split the data byte slice to get base state and rest byte slice individually.
  • Deserialize the base state byte array to corresponding type.
  • Ensure the base state is initialized.
  • Unpack the rest byte slice to get tlv byte slice.
  • Return base state and tlv slice.
/// --- token/program-2022/src/extension/mod.rs --- impl<'data, S: BaseState + Pod> PodStateWithExtensions<'data, S> { /// Unpack base state, leaving the extension data as a slice /// /// Fails if the base state is not initialized. pub fn unpack(input: &'data [u8]) -> Result<Self, ProgramError> { check_min_len_and_not_multisig(input, S::SIZE_OF)?; let (base_data, rest) = input.split_at(S::SIZE_OF); let base = pod_from_bytes::<S>(base_data)?; if !base.is_initialized() { Err(ProgramError::UninitializedAccount) } else { let tlv_data = unpack_tlv_data::<S>(rest)?; Ok(Self { base, tlv_data }) } } } fn check_min_len_and_not_multisig(input: &[u8], minimum_len: usize) -> Result<(), ProgramError> { if input.len() == Multisig::LEN || input.len() < minimum_len { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }
 
unpack_tlv_data calls type_and_tlv_indices to get start index of account_type and tlv in the rest slice. Then it checks the account_type matches the current account’s type. Lastly it returns the tlv slice to caller.
/// --- token/program-2022/src/extension/mod.rs --- fn unpack_tlv_data<S: BaseState>(rest: &[u8]) -> Result<&[u8], ProgramError> { if let Some((account_type_index, tlv_start_index)) = type_and_tlv_indices::<S>(rest)? { // type_and_tlv_indices() checks that returned indexes are within range let account_type = AccountType::try_from(rest[account_type_index]) .map_err(|_| ProgramError::InvalidAccountData)?; check_account_type::<S>(account_type)?; Ok(&rest[tlv_start_index..]) } else { Ok(&[]) } } fn check_account_type<S: BaseState>(account_type: AccountType) -> Result<(), ProgramError> { if account_type != S::ACCOUNT_TYPE { Err(ProgramError::InvalidAccountData) } else { Ok(()) } }