Texture Streaming

Last Updated: Jan 10th, 2024

Sometimes we want to render pixel data from a source other than a bitmap like a web cam. Using texture stream we can render pixels from any source.

//Texture wrapper class
class LTexture
{
public:
	//Initializes variables
	LTexture();

	//Deallocates memory
	~LTexture();

	//Loads image at specified path
	bool loadFromFile( std::string path );

	//Loads image into pixel buffer
	bool loadPixelsFromFile( std::string path );

	//Creates image from preloaded pixels
	bool loadFromPixels();

#if defined(SDL_TTF_MAJOR_VERSION)
	//Creates image from font string
	bool loadFromRenderedText( std::string textureText, SDL_Color textColor );
#endif

	//Creates blank texture
	bool createBlank( int width, int height );

	//Deallocates texture
	void free();

	//Set color modulation
	void setColor( Uint8 red, Uint8 green, Uint8 blue );

	//Set blending
	void setBlendMode( SDL_BlendMode blending );

	//Set alpha modulation
	void setAlpha( Uint8 alpha );

	//Renders texture at given point
	void render( int x, int y, SDL_Rect* clip = NULL, double angle = 0.0, SDL_Point* center = NULL, SDL_RendererFlip flip = SDL_FLIP_NONE );

	//Gets image dimensions
	int getWidth();
	int getHeight();

	//Pixel accessors
	Uint32* getPixels32();
	Uint32 getPixel32( Uint32 x, Uint32 y );
	Uint32 getPitch32();
	Uint32 mapRGBA( Uint8 r, Uint8 g, Uint8 b, Uint8 a );
	void copyRawPixels32( void* pixels );
	bool lockTexture();
	bool unlockTexture();
	
private:
	//The actual hardware texture
	SDL_Texture* mTexture;

	//Surface pixels
	SDL_Surface* mSurfacePixels;

	//Raw pixels
	void* mRawPixels;
	int mRawPitch;

	//Image dimensions
	int mWidth;
	int mHeight;
};

Here we're adding more functionality to our texture class. The createBlank function allocates a blank texture that we can copy data to when streaming. The copyRawPixels() function copies in the raw pixel data we want to stream.

What's also important is the lockTexture() which gets a pointer to send pixels to and unlockTexture() which uploads pixel data to the pixels. We also have "mRawPixels" and "mRawPitch" to store the data until we send the data to the GPU.

//A test animation stream
class DataStream
{
    public:
        //Initializes internals
        DataStream();

        //Loads initial data
        bool loadMedia();

        //Deallocator
        void free();

        //Gets current frame data
        void* getBuffer();

    private:
        //Internal data
        SDL_Surface* mImages[ 4 ];
        int mCurrentImage;
        int mDelayFrames;
};

Here is our data stream class. We won't go deep into how it works because we don't really care. When dealing with web cam, video decoding, etc APIs they typically don't go deep into how they work because all we care about is getting the video and audio data from them.

All we really care about is that getBuffer function which gets the current pixels from the data buffer.

bool LTexture::createBlank( int width, int height )
{
    //Get rid of preexisting texture
    free();

    //Create uninitialized texture
    mTexture = SDL_CreateTexture( gRenderer, SDL_PIXELFORMAT_RGBA8888, SDL_TEXTUREACCESS_STREAMING, width, height );
    if( mTexture == NULL )
    {
        printf( "Unable to create streamable blank texture! SDL Error: %s\n", SDL_GetError() );
    }
    else
    {
        mWidth = width;
        mHeight = height;
    }

    return mTexture != NULL;
}

As you can see, all this function does is create a 32bit RGBA texture with stream access. One thing you have to make sure of when creating your texture is that the format of the texture pixels matches the format of the pixels we're streaming.

bool LTexture::lockTexture()
{
    bool success = true;

    //Texture is already locked
    if( mRawPixels != NULL )
    {
        printf( "Texture is already locked!\n" );
        success = false;
    }
    //Lock texture
    else
    {
        if( SDL_LockTexture( mTexture, NULL, &mRawPixels, &mRawPitch ) != 0 )
        {
            printf( "Unable to lock texture! %s\n", SDL_GetError() );
            success = false;
        }
    }

    return success;
}

bool LTexture::unlockTexture()
{
    bool success = true;

    //Texture is not locked
    if( mRawPixels == NULL )
    {
        printf( "Texture is not locked!\n" );
        success = false;
    }
    //Unlock texture
    else
    {
        SDL_UnlockTexture( mTexture );
        mRawPixels = NULL;
        mRawPitch = 0;
    }

    return success;
}

Here are our texture locking and unlocking functions which wrap SDL_LockTexture and SDL_UnlockTexture. SDL_LockTexture() grabs a pointer of pixel data and the pitch of the texture. Note: this pointer is not expected to have the data from the original texture. It may do so on certain platforms like Windows, but on other platforms like MacOS it will not. If you create a texture and then lock it, do not expect it to have the original pixel data you created it with. Expect SDL_LockTexture() to give you a fresh pixel pointer to send new data to the texture with. SDL_LockTexture() also grabs texture pitch in bytes.

Once you're done manipulating the pixel pointer with the data you want, unlocking it will send the pixel data to the GPU.

void LTexture::copyRawPixels32( void* pixels )
{
    //Texture is locked
    if( mRawPixels != NULL )
    {
        //Copy to locked pixels
        memcpy( mRawPixels, pixels, mRawPitch * mHeight );
    }
}

Here is our function to copy in the pixels from the stream into the locked texture pixel pointer. The function assumes that the pixels are from an image the same size as the texture.

bool loadMedia()
{
    //Loading success flag
    bool success = true;

    //Load blank texture
    if( !gStreamingTexture.createBlank( 64, 205 ) )
    {
        printf( "Failed to create streaming texture!\n" );
        success = false;
    }

    //Load data stream
    if( !gDataStream.loadMedia() )
    {        
        printf( "Unable to load data stream!\n" );
        success = false;
    }

    return success;
}

In our media loading, we create a blank streaming texture and start our data stream.

            //While application is running
            while( !quit )
            {
                //Handle events on queue
                while( SDL_PollEvent( &e ) != 0 )
                {
                    //User requests quit
                    if( e.type == SDL_QUIT )
                    {
                        quit = true;
                    }
                }

                //Clear screen
                SDL_SetRenderDrawColor( gRenderer, 0xFF, 0xFF, 0xFF, 0xFF );
                SDL_RenderClear( gRenderer );

                //Copy frame from buffer
                gStreamingTexture.lockTexture();
                gStreamingTexture.copyPixels( gDataStream.getBuffer() );
                gStreamingTexture.unlockTexture();

                //Render frame
                gStreamingTexture.render( ( SCREEN_WIDTH - gStreamingTexture.getWidth() ) / 2, ( SCREEN_HEIGHT - gStreamingTexture.getHeight() ) / 2 );

                //Update screen
                SDL_RenderPresent( gRenderer );
            }

In the main loop rendering we lock our stream texture, copy the pixels from the stream and then unlock the texture. With our texture holding the latest image from the stream, we render the image to the screen.

When dealing with decoding APIs things may get trickier where we have to convert from one format to another but ultimately all we need is a means to get the pixel data and copy it to the screen.

Download the media and source code for this demo here.

Back to Index