Deployment

When deploying in cloud environments with firewalls (like Hugging Face Spaces, RunPod), your WebRTC connections may be blocked from making direct connections. In these cases, you need a TURN server to relay the audio/video traffic between users. This guide covers different options for setting up FastRTC to connect to a TURN server.

Tip

The rtc_configuration parameter of the Stream class also be passed to the WebRTC component directly if you're building a standalone gradio app.

Community Server

Hugging Face graciously provides a TURN server for the community. In order to use it, you need to first create a Hugging Face account by going to huggingface.co.

Then navigate to this space and follow the instructions on the page. You just have to click the "Log in" button and then the "Sign Up" button.

Then you can use the get_hf_turn_credentials helper to get your credentials:

from fastrtc import get_hf_turn_credentials, Stream

# Pass a valid access token for your Hugging Face account
# or set the HF_TOKEN environment variable 
credentials = get_hf_turn_credentials(token=None)

Stream(
    handler=...,
    rtc_configuration=credentials,
    modality="audio",
    mode="send-receive"
)

Warning

This is a shared resource so we make no latency/availability guarantees. For more robust options, see the Twilio, Cloudflare and self-hosting options below.

Twilio API

An easy way to do this is to use a service like Twilio.

Create a free account and the install the twilio package with pip (pip install twilio). You can then connect from the WebRTC component like so:

from fastrtc import Stream
from twilio.rest import Client
import os

account_sid = os.environ.get("TWILIO_ACCOUNT_SID")
auth_token = os.environ.get("TWILIO_AUTH_TOKEN")

client = Client(account_sid, auth_token)

token = client.tokens.create()

rtc_configuration = {
    "iceServers": token.ice_servers,
    "iceTransportPolicy": "relay",
}

Stream(
    handler=...,
    rtc_configuration=rtc_configuration,
    modality="audio",
    mode="send-receive"
)

Automatic login

You can log in automatically with the get_twilio_turn_credentials helper

from gradio_webrtc import get_twilio_turn_credentials

# Will automatically read the TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN
# env variables but you can also pass in the tokens as parameters
rtc_configuration = get_twilio_turn_credentials()

Cloudflare Calls API

Cloudflare also offers a managed TURN server with Cloudflare Calls.

Create a free account and head to the Calls section in your dashboard.

Choose Create -> TURN App, give it a name (like fastrtc-demo), and then hit the Create button.

Take note of the Turn Token ID (often exported as TURN_KEY_ID) and API Token (exported as TURN_KEY_API_TOKEN).

You can then connect from the WebRTC component like so:

from fastrtc import Stream
import requests
import os

turn_key_id = os.environ.get("TURN_KEY_ID")
turn_key_api_token = os.environ.get("TURN_KEY_API_TOKEN")
ttl = 86400 # Can modify TTL, here it's set to 24 hours

response = requests.post(
    f"https://rtc.live.cloudflare.com/v1/turn/keys/{turn_key_id}/credentials/generate-ice-servers",
    headers={
        "Authorization": f"Bearer {turn_key_api_token}",
        "Content-Type": "application/json",
    },
    json={"ttl": ttl},  
)
if response.ok:
    rtc_configuration = response.json()
else:
    raise Exception(
        f"Failed to get TURN credentials: {response.status_code} {response.text}"
    )

stream = Stream(
    handler=...,
    rtc_configuration=rtc_configuration,
    modality="audio",
    mode="send-receive",
)

Self Hosting

We have developed a script that can automatically deploy a TURN server to Amazon Web Services (AWS). You can follow the instructions here or this guide.

Prerequisites

Clone the following repository and install the aws cli if you have not done so already (pip install awscli).

Log into your AWS account and create an IAM user with the following permissions:

• AWSCloudFormationFullAccess
• AmazonEC2FullAccess

Create a key pair for this user and write down the "access key" and "secret access key". Then log into the aws cli with these credentials (aws configure).

Finally, create an ec2 keypair (replace your-key-name with the name you want to give it).

aws ec2 create-key-pair --key-name your-key-name --query 'KeyMaterial' --output text > your-key-name.pem

Running the script

Open the parameters.json file and fill in the correct values for all the parameters:

• KeyName: The key file we just created, e.g. your-key-name (omit .pem).
• TurnUserName: The username needed to connect to the server.
• TurnPassword: The password needed to connect to the server.
• InstanceType: One of the following values t3.micro, t3.small, t3.medium, c4.large, c5.large.

Then run the deployment script:

aws cloudformation create-stack \
  --stack-name turn-server \
  --template-body file://deployment.yml \
  --parameters file://parameters.json \
  --capabilities CAPABILITY_IAM

You can then wait for the stack to come up with:

aws cloudformation wait stack-create-complete \
  --stack-name turn-server

Next, grab your EC2 server's public ip with:

aws cloudformation describe-stacks \
  --stack-name turn-server \
  --query 'Stacks[0].Outputs' > server-info.json

The server-info.json file will have the server's public IP and public DNS:

[
    {
        "OutputKey": "PublicIP",
        "OutputValue": "35.173.254.80",
        "Description": "Public IP address of the TURN server"
    },
    {
        "OutputKey": "PublicDNS",
        "OutputValue": "ec2-35-173-254-80.compute-1.amazonaws.com",
        "Description": "Public DNS name of the TURN server"
    }
]

Finally, you can connect to your EC2 server from the gradio WebRTC component via the rtc_configuration argument:

from fastrtc import Stream
rtc_configuration = {
    "iceServers": [
        {
            "urls": "turn:35.173.254.80:80",
            "username": "<my-username>",
            "credential": "<my-password>"
        },
    ]
}
Stream(
    handler=...,
    rtc_configuration=rtc_configuration,
    modality="audio",
    mode="send-receive"
)