Real-time streaming can be understood as a constant flow of assets from one wallet to another over a period of time. It simplifies transactions and also enables a trustless environment. Subscription services or freelancers can use streaming transactions to maintain a trustless environment with their customers. There are many uses of this concept. You can check out the SuperFluid protocol built on the Ethereum network as an example. This tutorial is comprised of three parts. First, we will write a Solana program to handle the streaming, then we will create a backend for our protocol. Finally, we will connect it all with our front-end.
A good understanding of the Rust programming language and React and Redux is required to grasp the contents of this tutorial.
The following software is required to complete this tutorial:
- Git, install it from HERE.
- Solana CLI, install it from HERE.
- Solana wallet
- The Rust toolchain, install it from HERE.
- Node.js (v14.18.1+), install it from HERE.
- Postgress, install it from HERE
We will write the tutorial in 3 parts. First, we will write our Solana program then we will create a backend for our protocol, and in the end, we will connect it all with our front-end.
Before getting into the coding, let us briefly review what a streaming protocol means and what instructions we need to create in our Solana program. A streaming protocol creates an escrow account that will keep track of the balance of both parties with the help of time. Initially, all the funds in the escrow account will be owned by the sender. As time passes, ownership of some funds will be transferred to the receiver which means the receiver can then withdraw those funds.
We will need 3 instructions in our Solana program
- To Create a Stream: This can be used the the sender and they will be funding our escrow account, give information about the receiver and define start and end time for the Stream.
- Withdraw funds: This can be used by the receiver to withdraw the funds they have earned.
- Cancel Stream: This may be used by the sender to cancel a stream. This instruction will distribute the owned funds in the escrow account to senders and receiver accounts.
Let's get started with creating our program! Open a terminal in your projects folder and run the following command to create a new Rust project using the library template:
We will be now able to see the
sol-stream-programfolder in our projects directory so we can open it open our code editor. I am using Visual Studio Code (commonly called VSCode) for this tutorial. You can use code editor of your choice - in the
Cargo.tomlfile, add the required dependencies:
Now to download all the crates, we can save the
Cargo.tomlfile and in the terminal run:
We will create our program with the following structure:
The flow of a program using this structure looks like this:
- Someone calls the entrypoint
- The entrypoint forwards the arguments to the processor
- The processor asks
instruction.rsto decode the
instruction_dataargument from the entrypoint function.
- Using the decoded data, the processor will now decide which processing function to use to process the request.
- The processor may use
state.rsto encode state into or decode the state of an account which has been passed into the entrypoint.
The code in
instruction.rsdefines the API of the program.
Now let us create the
error.rsfiles in the
sol-stream-program/srcdirectory. We can register the newly created files in
lib.rsby updating it to include the modules:
Now in the
src/entrypoint.rsfile let's add the code for our entrypoint function and register it by using the
We are importing the required structs, functions and macros from the
solana_programcrate. Then we create a function that will take program_id, accounts, and instruction_data as input parameters and returns a ProgramResult. We then register this function with the
entrypoint!macro in the last line.
Now let us open
instruction.rsand add the following code:
WithdrawFromStreamwould require the some input from initiator, let us create structs for them in
state.rsadd the following code:
CreateStreamwe will want
start_time: The Unix timestamp at which the stream will start.
end_time: The Unix timestamp at which the stream will end.
receiver: Public key of the receiver.
lamports_withdrawn: We allow the receiver to withdraw Lamports when they have ownership, we would also want to keep track of the number of Lamports withdrawn for calculation purposes.
amount_speed: The number of Lamports transferred to the receiver every second.
WithdrawFromStreamwe will want the
amount: The amount of Lamports the receiver wants to withdraw.
Now let us import these structs into
instruction.rsand use them, add this line at the very top of the file:
Now that our enums are complete, let us add a function to unpack the instruction given to our program. At the end of
We have added a function to unpack data since there is only one entrypoint. We have used the first element of
instruction_dataas a tag. Then we have used the
BorshDeserializationderivation which provides us with the
try_from_slicefunction to unpack data. We are using:
- Tag 1 -> Create Stream Instruction.
- Tag 2 -> Withdraw Instruction.
- Tag 3 -> Close Stream Instruction.
Returning an error otherwise. Now we can open the
processor.rsfile and add the logic for instructions in it.
We have imported the required struct and functions from the
solana_programcrate. Then we will create a struct called
Processorand implement it by creating a function named
processwhich will take as input:
program_id: Program ID of program.
accounts: Accounts passed in the transaction.
instruction_data: Instruction data passed for instruction.
This function will return
ProgramResult. In the function, we have unpacked the
instruction_datausing the unpack function we created on
StreamInstructionenum. We have used the
?operator to unwrap valid values or return the error value, propagating them to the calling function. Then we have used a match statement on the instructions and stubbed them out by adding the
todo!()macro to each of them.
Before we move on to defining the instructions, let us update the
We have imported the
Processorand updated the function to call the
processfunction passing the same arguments. For reference, you can check the file HERE.
Now let us go to
processor.rsand remove the todos. For each instruction, we will create a function and write the logic in there.
Update the code in
We have created empty functions
process_withdrawfunctions have a parameter
datawhich would be the struct
Now let's open
errors.rsand write the errors that our program might return in some cases.
Now we can open the
processor.rsfile again and complete
process_create_stream We will parse the public key for admin. I have used my pubkey here as an example, but you can use your public key. We are using the
from_strmethod. In case of an error, we return
PubKeyParseErrorwhich we defined in
Then we will get all the accounts. First we create an iterator and then we can use the
next_account_infofunction to get all the accounts. We will store all the accounts with a corresponding variable name.
Now we can check if the admin account provided was correct or incorrect by comparing its key with the pubkey we provided in our function. If it is incorrect, we return an error.
Now, we can make a transaction of 0.03 lamports (an arbitrary amount) and send those to the admin account.
Now, we can check the given instruction data. When we check the end time, it shouldn't be less than the start time and the start time shouldn't be less than the current time. We can get the current
Clock::get()?.unix_timestamp. We will return an
InvalidStartOrEndTimeerror in case of failure.
Then we can check that the total Lamport deposited in the account should be equal to the amount we want to send + the minimum number of Rent required to create the account on the chain. We can get the minimum amount of balance required by using the
Rent::get()?.minimum_balance(len)method. If this failed we can return the
Then we will check who signed this transaction, and the public key of the receiver is equal to the account provided to us.
Now we are all set to write the stream data to our program account. We will create a
StreamDatastruct and store that in our escrow account. In
state.rsat the end add a new struct:
We have added a new method to create an instance of the struct with the help of
CreateStreamInputand the sender's public key.
Now let's jump back to the
processor.rsfile and complete the function:
We will first create the
escrow_dataand then with the help of the borsh
serializemethod we can write data to
escrow_account. We complete the function by returning the result
Ok(())at the end of the function.
We have stored accounts into variables just like we did in the
process_create_streamfunction. Then we have deserialized data in the
escrow_accountnot that this data is what we saved in the
process_create_streamfunction. Then we perform a check that the receiver of this account is the singer and this
escrow_accountbelongs to him.
Then we can check if the user can withdraw the required Lamports or not. We will get the current time and calculate the total number of Lamports owned by them. By subtracting
lamports_withdrawn, we can keep track of the Lamports that are already withdrawn by the receiver.
Now we can proceed with the transaction and send the token to
receiver_account. We will also make an increment in
lamports_withdrawn. We finish the function by writing the new
escrow_accountand then returning the result
In this function also we will get the accounts and store them in variables. Then we get the
escrow_datajust like we did in the
process_withdrawfunction. We then check if the
sender_accountis the owner of
senderhas signed the transaction or not.
We are closing the escrow account, so we want to transfer the funds to the receiver and sender which they own. So we can calculate the total tokens owned by the receiver.
Now we have the total Lamports that are owned by the receiver. We can send the remaining Lamports to the sender. At last, we set the
escrow_accountbalance to 0, then we can return the result
You can check the complete code of the Solana program on GitHub.
Now to deploy the program, we can use the following command to create a build. In this command, the manifest-path should be the path of your
Cargo.tomlfile. This will output the compiled program in Shared Object format (.so) in the
We will create a new Solana account to deploy the program. Run the following command:
The command will prompt you for a passphrase to secure the recovery seed phrase:
You can choose a passphrase or leave it empty. Continuing will provide both the public key of the account and the seed phrase used to create the private key:
Once complete you will have the
keypair.jsonfile, containing the private and public keys of a new Solana account. It is important to keep your keypair safe. Do not commit this file to a remote code repository. It is best to add this file to a
.gitignoreto prevent this from happening.
Now we are going to request an airdrop of SOL tokens on the Solana Devnet. This command will add 1 SOL token to the account:
If you get insufficient balance while deploying, you can re-run the command to airdrop funds on Devnet.
Now we will use the following command to deploy. Note that the path of
dist/program/program.somight be different in your case. Please check and then run the command.
We will get the program id as output.
We can verify the deployment by checking on the Solana Explorer for Devnet.. We can search for our program id to see any related transactions, including the deployment.
We will be creating our backend with help of the rocket web framework, but before we start writing the code for the backend it would be a great time to understand why we need a backend.
Let's think of a scenario where we do not have a backend for our protocol. We have stored all the data in the Program driven account(PDA) so for fetching all streams we can
getProgramAccountsfunction provided in the
@solana/web3.jspackage and then with the help of the
borshpackage we will deserialize the byte data. Then we can check among all the PDA's data which one of those belongs to a user i.e either they are sending or receiving.
Now let us suppose we have made our app live and it has about 1000 users. If all users have created 2 streams, this means we would have 2000 program driven accounts! Fetching all the accounts just to display the 2 streams for each user would make our protocol slow and with any increase in users, it becomes unusable.
We will use our backend to index our PDA's and fix the scalability issue. Let us create a rust project again but this time with the default template i.e binary (application) template. Open the console in your projects folder and run the following command:
This will generate
sol-stream-backenddirectory. We can go ahead and open it on VS-Code now. For now, our project will look like:
Now open the
Cargo.tomlfile and update it to add the required crates:
- borsh: To serialize and deserialize data
- rocket_cors: To enable cross-origin resource sharing
- solana-client: To fetch all program accounts on the solana blockchain
- serde: For JSON Serialization and Deserialization.
- diesel: For SQL query building
- dotenv: To manage our database_url
We are going to need all of these crates, to download them we can save the
Cargo.tomlfile and in the terminal run:
Now in the
main.rslet us add code to create a "Hello world!" route with rocket.rs.
Before I explain this code run the following command in the terminal:
It will compile and run the program then we can open
http://127.0.0.1:8000/on our browser to see:
Now let's see what code made this happen. In the first line, we have imported
getfrom the rocket. We will use
#[rocket::main]on our function which will transform our function into a regular main function that internally initializes a Rocket-specific tokio runtime and runs the attributed async fn inside of it.
Then inside the function, we will get default cors (cors stands for Cross Object Resource Sharing) in a variable
cors. This line of code attaches our
corsand will start a server with a base
"/"and it will have routes that will be passed in the
routes!macro. As you can see we have two routes and the functions for them are called
route_with_pubkey. I have passed them in the macro. Then we await this call and return
Ok(())so that it compiles.
Now take a look at the
route_with_pubkeyfunctions. We have made use of the
getmacro here. For the
indexfunction, we will return a String "Hello world". In the case of
route_with_pubkeywe will get the pubkey from the URL and return "Hello <pubkey>".
Now let us install
dieselCLI. We can do this by running:
Now in our project folder, we can run:
Now run the following command to create a migration:
This will create a migrations folder with the name
Now we can run the following command which would create
src/schema.rsfile which would contain schema required for
Now we can run the following command to create a
.envfile in our project which would contain the database URL.
main.rs: will contain the main function and we will use it to add all the modules.
models.rs: will contain
routes.rs: will contain all the routes in our app.
schema.rs: is generated by diesel cli.
solana.rs: will contain a function to subscribe to the solana program and get all program accounts.
Create the empty files and let us write the code for our backend.
main.rsat the top add:
We have added
#[macro_use]for diesel. We will create the function to get
PgConnectionwith the help
database_urlwe can get with the dotenv. We have not added the
mainfunction here yet, We will add that later.
Now we can go in
Here we have 2 structs one of them is the same we had in our program
StreamData: Used to get the data from the PDA account.
Stream: This is the struct we would like to store in our database. We will have
pda_accountPubKey as ID. We have changed Pubkey types to
receiveras we can use the
.tostringfunction on pubkey and it would be easy to store that in our database compared to an array of i32. We have also drived
Insertablefrom diesel so we can generate find, insert, update, and other SQL queries with the help of diesel. We have also drived
serdecrate to convert this to JSON format.
Now we will need some more functions in on
Streamso at the end of the file add:
We have added the following functions:
new: function to create a new
Streamwith the help of
pdapub key and
pdadata. We first create the
StreamDatawith the help of the
borshcrate and then we can create a return
get_all_with_sender: function to get all the streams with the sender equal to the given public key.
get_all_with_receiver: function to get all the streams with the receiver equal to the given public key.
id_is_present: function to check if the database contains the particular id.
insert_or_update: function to
inserta new Stream if the
idis not present which we can check with
updateif it is present.
If you want to learn more about diesel you can read HERE.
Now we can move to the
We have added three functions to this file.
get_all_program_accounts: function will return all the program accounts owned by a program.
subscribe_to_program: function to subscribe to the program which would give us notification whenever there is an update in any program-owned account.
get_accounts_and_update: function to get all the program accounts and fill insert or update them in our database.
You can read about Solana RPC HERE.
subscribe_to_programwe have spawned a thread to listen to updates we will receive and in case we receive an error we have called the function again. For each notification in the for loop, we have created the stream with the help of
RPC Notificationyou can see the JSON schema for that HERE. Then we have called the
insert_or_updatefunction which we created in our model.
Now let's go to
We have added create two functions the
indexfunction is the same as we had in our rocket example.
get_all_stream: This function will return Json response we are using
get_all_with_receiverfunction to get them from our indexed table.
Now let us move to
src/main.rsand add the main function. At top of the file add the following Code:
Now add the
In the main function we are getting all the program accounts and populating our database, then we can call the
subscribe_to_programfunction. After this, we are starting the rocket server.
You can see the complete Backend HERE.
Now let's write the frontend part of the app. I have created a React app for the frontend, using Redux for state management. I have already created the UI part of it, which you can find HERE.
You can clone and checkout the
uibranch and run it with the help of the following commands:
Once it is running, you can see the UI in the browser:
Video: UI Sample
Now let us write the functions to connect with the sol-stream-program.
We have created
adminAddressvariables to store the program id and admin address for the program. We have also created a
connectionvariable with the help of the
clustervariable which contains the link to devnet RPC.
Now let's write the
getAllStreamsfunction first which you can find at the very last in the
action/index.jsfile. It just contains a request to our backend client it takes in the string value of
pubkeyas a parameter in it. I have used the axios package for it we can change the base URL in
Now that our
baseURLis set we can write the function in
We are requesting the route "/<pubkey>" and then we dispatch the result. We have added the code in the try and catch block to handle network errors.
Now let's write the functions that will contain the instructions to our program. We will also create a Schema for data Serialization.
For creating the stream instructions we have defined a Schema that describes the data type for variables. We will be using this in the
In the code above we have created a pubkey which we will use to create a Program Drived account(PDA). We have used the function
serializefrom the borsh package to get the Uint8Array of the input data.
We have to append
1in our Uint8Array as we are using the first character as the tag. Please refer to the unpack function in
We can then get the minimum lamports required to make the account by using the
getMinimumBalanceForRentExemptionfunction on the connection variable.
In the above code, We have created 2 instruction:
To the system program to create a Program drived account with the space of
96and lamports which are equal to
0.03(admin cut)+solana rent+total amount user want to send. We are passing these arguments in the
This instruction contains an array of public keys associated with the transaction, specifically the
newAccount(the PDA being created), the signer's public key coming from their connected wallet, and also the receiver and admin addresses. The
programIdis the deployed program address and
datais the instruction data which includes the tag to specify which instruction to execute (CreateStream, WithdrawFromStream or CloseStream).
Then we create a transaction object we have used the
setPayerAndBlockhashTransactionfunction which we will add later. Once we have the transaction object we have to send the transaction with the help of the wallet. We have then dispatched the result. In case of an error, we can just alert the user.
setPayerAndBlockhashTransaction: This function takes in the instructions array and then returns a
You can check out the
CreateStreamfunction HERE, for reference.
Now let's write the
withdrawfunction for this one also we have created the
WithdrawInputclass with schema:
We only have the
amountvariable in the struct for withdrawal input.
This time we aren't creating a PDA so we only have one instruction in this transaction which is to our program. Note that the
streamIdis the address to the
data_to_send, we have to specify 2 at the beginning of the array. As you recall, in our program's
instruction.rsfile we have set the tag 2 for the WithdrawFromStream instruction.
We have dispatched the withdraw result in the end. You can check out the
Withdrawfunction HERE, for reference.
Now let's write the last function in the file to close a stream.
For this function, we do not have any instruction data we are only passing the tag. Then we have packed the instruction by calling
setPayerAndBlockhashTransactionand then dispatched the result.
You can check the frontend code HERE. You can check out the
closeStreamfunction HERE, for reference.
Now for testing it we will need the backend server running. Here is a video walkthrough.
Video: Sol Streaming Protocol
In this tutorial, you learned how to build a Sol token streaming protocol on Solana. We covered the on-chain program's code using the Rust programming language. We built the User Interface with React. And we have created a backend with Rocket framework to index PDA's data.
This tutorial was created by Sushant Chandla.