Imagination PowerVR SDK Blog

PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]

pvrtune

#1

I wrote a program to render to texture instead of real window. I keep getting this warning:



PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]



The program is modified from HelloAPI and I am using the X11 window structure. The egl part and the message loop, swapWindow part are kept as is. I am wondering if I am doing the right thing.


#2

Hi,



Can you explain what you are trying to achieve? Do you need to send the data to a file, pass it to another process, or something else?



Our driver postpones FBO renders until they are used (i.e. referenced by a draw call). As you’re not rendering anything to the framebuffer, the driver thinks that the FBO isn’t needed yet and is postponing the work. When the driver queues up a sufficient number of FBOs, it kicks a render to free memory. The warning you are seeing is when this event occurs.



You can force the driver to kick the renders earlier by using two FBOs and referencing them in each others render (e.g. draw an offscreen triangle with FBO A’s texture bound when rendering into FBO B). Depending on what you’re trying to achieve, there may be a better way to solve the problem.



Thanks,

Joe


#3

Thanks that’s very helpful.

Could you explain “draw an offscreen triangle with FBO A’s texture bound when rendering into FBO B”? When I bound to FBA A and draw something to it, how can I render to FBO B?


#4

To clarify - off-screen here means the object is outside of the view frustum and will be culled by the GPU. Rendering in this way will force the driver to upload the texture (and kick any outstanding renders needed to complete that texture), but will incur the smallest possible additional GPU overhead.

Triangles are the best off-screen objects to use, as they have the fewest number of vertices possible. You can use any object to do this though.



Here are the steps you could take:



Step 1: (one off for first render)

  1. Bind FBO A
  2. Render your scene into FBO A



    Step 2:
  3. Bind FBO B
  4. Render an off-screen object with FBO A’s texture bound
  5. Render your scene into FBO B
  6. Bind FBO A
  7. Render an off-screen object with FBO B’s texture bound
  8. Render your scene into FBO A
  9. Repeat steps 1-6



    Hope this helps.



    Regards,

    Joe

#5

Thanks for the detailed description. I implemented the logic above but still get the same error “PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]”. I guess I don’t quite understand what the steps above are trying to achieve: what’s the significance of having 2 FBOs and 2 textures? Isn’t it essentially still using one FBO and one texture at a time? If an offscreen drawing could trigger the rendering, why not just use one FBO and the attached texture as color so that when the offscreen drawing is issued, it will be rendered?



What I am trying to do is just to use GPU to do some computing, there are two input textures and one or two output texture(s) - the output can be just a buffer instead of texture. Maybe I can use renderbuffer instead of the output texture? I don’t want to read the output back as at this time the only purpose is to evaluate if the GPU can do the computing in reasonable time.



Another possibility is that I might not render the offscreen triangle correctly. Here is the code:


glBindBuffer(GL_ARRAY_BUFFER, OffScreenTriangleVBO);<br />
glVertexAttribPointer(shaderPosHandle, 3, GL_FLOAT, GL_TRUE, 0, 0);<br />
glEnableVertexAttribArray(shaderPosHandle);<br />
glDrawArrays(GL_TRIANGLES,0,3);
```<br />
<br />
After that I render a simple scene with 2 texture inputs.

#6

When the driver receives a draw call for a primitive that has an FBO’s texture bound, it will check if that FBO’s render has completed. If the texture hasn’t been rendered to yet, the driver will kick the outstanding FBO render.



For this technique to work, the off-screen triangle must be textured with the previous FBO’s texture. You need to use two FBOs so that the current FBO can kick the render of the previous FBO. Swapping between the two FBOs while using this technique should cause the driver to regularly kick FBO renders.



Thanks.

Joe


#7

I still get the warnings after doing the method as you suggested. Not sure if I did it correctly. Here is the main code that is rendering a triangle as the main scene, and the offscreen object with two FBOs:



// Bind the custom vertex attribute “myVertex” to location VERTEX_ARRAY

glBindAttribLocation(uiProgramObject, VERTEX_ARRAY, “myVertex”);



GLuint mVertVBO = 0; // Vertex buffer object handle

int mVertexCount = 6;

// We’re going to draw a rectangle to the screen so create a vertex buffer object for two triangles to form the rectangle

{

// Interleaved vertex data

GLfloat afVertices[] = { -1.0f,-1.0f,0.0f, // Position

1.0f ,-1.0f,0.0f,

1.0f ,1.0f ,0.0f,

1.0f ,1.0f ,0.0f,

-1.0f,1.0f ,0.0f,

-1.0f,-1.0f,0.0f

};



// Generate the vertex buffer object (VBO)

glGenBuffers(1, &mVertVBO);



// Bind the VBO so we can fill it with data

glBindBuffer(GL_ARRAY_BUFFER, mVertVBO);



// Set the buffer’s data

unsigned int uiSize = 6 * (sizeof(GLfloat) * 3); // Calc afVertices size (3 vertices * stride (3 GLfloats per vertex))

glBufferData(GL_ARRAY_BUFFER, uiSize, afVertices, GL_STATIC_DRAW);

}



// offscreen triangle vertexes

GLuint mVertVBOOff;

{

// vertex data

GLfloat mVOff[] = {

1.5f, 1.5f,0.0f, // Position

1.5f , 1.4f,0.0f,

1.6f , 1.4f,0.0f,

};

glGenBuffers(1, &mVertVBOOff);

glBindBuffer(GL_ARRAY_BUFFER, mVertVBOOff);

glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)33, mVOff, GL_STATIC_DRAW);

}



// let’s make sure we turn this bind off

glBindBuffer(GL_ARRAY_BUFFER, 0 );





GLuint texobj[2];

GLuint fid[2];



// Generate a frame buffer to render to the texture.

glGenFramebuffers(2, fid);

// target texture

glGenTextures(2, texobj);



// target texture 1

glBindFramebuffer(GL_FRAMEBUFFER, fid[0]);

// Use tightly packed data

glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

glBindTexture(GL_TEXTURE_2D, texobj[0]);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA,WINDOW_WIDTH,WINDOW_HEIGHT,0,GL_RGBA,GL_UNSIGNED_BYTE, NULL);



// Attach the texture as a color buffer.

glFramebufferTexture2D(

GL_FRAMEBUFFER,

GL_COLOR_ATTACHMENT0,

GL_TEXTURE_2D, texobj[0],

0);



int status = glCheckFramebufferStatus(GL_FRAMEBUFFER);

if(status != GL_FRAMEBUFFER_COMPLETE)

printf(“Error glCheckFramebufferStatus status %xn”, status);

else

printf(“glCheckFramebufferStatus okn”);



// target texture 2

glBindFramebuffer(GL_FRAMEBUFFER, fid[1]);

// Use tightly packed data

glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

glBindTexture(GL_TEXTURE_2D, texobj[1]);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA,WINDOW_WIDTH,WINDOW_HEIGHT,0,GL_RGBA,GL_UNSIGNED_BYTE, NULL);



// Attach the texture as a color buffer.

glFramebufferTexture2D(

GL_FRAMEBUFFER,

GL_COLOR_ATTACHMENT0,

GL_TEXTURE_2D, texobj[1],

0);



status = glCheckFramebufferStatus(GL_FRAMEBUFFER);

if(status != GL_FRAMEBUFFER_COMPLETE)

printf(“Error glCheckFramebufferStatus status %xn”, status);

else

printf(“glCheckFramebufferStatus okn”);









// Actually use the created program

glUseProgram(uiProgramObject);



// Sets the clear color.

// The colours are passed per channel (red,green,blue,alpha) as float values from 0.0 to 1.0

glClearColor(0.6f, 0.8f, 1.0f, 1.0f); // clear blue





glViewport(0, 0, WINDOW_WIDTH, WINDOW_HEIGHT);



// render scene for target 1

glBindFramebuffer(GL_FRAMEBUFFER, fid[0]);



// First gets the location of that variable in the shader using its name

int i32Location = glGetUniformLocation(uiProgramObject, “myPMVMatrix”);

// Then passes the matrix to that variable

glUniformMatrix4fv( i32Location, 1, GL_FALSE, pfIdentity);



glBindBuffer(GL_ARRAY_BUFFER, mVertVBO);

/*

Enable the custom vertex attribute at index VERTEX_ARRAY.

We previously binded that index to the variable in our shader “vec4 MyVertex;”

/

glEnableVertexAttribArray(VERTEX_ARRAY);

// Sets the vertex data to this attribute index

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);





glDrawArrays(GL_TRIANGLES,0,mVertexCount);



glBindBuffer(GL_ARRAY_BUFFER, 0 );







// Draws a triangle for 2200 rames

for(int i = 0; i < 2200; ++i)

{

// start timer after 200 frames

if (i==200)

gettimeofday(&timeNow, NULL);

// Check if the message handler finished the demo

if (bDemoDone) break;



/


Clears the color buffer.

glClear() can also be used to clear the depth or stencil buffer

(GL_DEPTH_BUFFER_BIT or GL_STENCIL_BUFFER_BIT)

*/

glClear(GL_COLOR_BUFFER_BIT);

if (!TestEGLError(“glClear”))

{

exit(120);

}





// render to target 2

glBindFramebuffer(GL_FRAMEBUFFER, fid[1]);

// Attach the texture as a color buffer.

glFramebufferTexture2D(

GL_FRAMEBUFFER,

GL_COLOR_ATTACHMENT0,

GL_TEXTURE_2D, texobj[0],0);

status = glCheckFramebufferStatus(GL_FRAMEBUFFER);

if(status != GL_FRAMEBUFFER_COMPLETE)

printf(“Error glCheckFramebufferStatus status %xn”, status);



// render offscreen obj

glBindBuffer(GL_ARRAY_BUFFER, mVertVBOOff);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);

glDrawArrays(GL_TRIANGLES,0,3);





// render rectangle

// First gets the location of that variable in the shader using its name

int i32Location = glGetUniformLocation(uiProgramObject, “myPMVMatrix”);

// Then passes the matrix to that variable

glUniformMatrix4fv( i32Location, 1, GL_FALSE, pfIdentity);

// set the vertex vbo

glBindBuffer(GL_ARRAY_BUFFER, mVertVBO);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_TRUE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);



glDrawArrays(GL_TRIANGLES,0,mVertexCount);



glBindBuffer(GL_ARRAY_BUFFER, 0 );





//////////////////////////////////////////////////////

// render to target 1

glBindFramebuffer(GL_FRAMEBUFFER, fid[0]);

// Attach the texture as a color buffer.

glFramebufferTexture2D(

GL_FRAMEBUFFER,

GL_COLOR_ATTACHMENT0,

GL_TEXTURE_2D, texobj[1],0);

status = glCheckFramebufferStatus(GL_FRAMEBUFFER);

if(status != GL_FRAMEBUFFER_COMPLETE)

printf(“Error glCheckFramebufferStatus status %xn”, status);



// render offscreen obj

glBindBuffer(GL_ARRAY_BUFFER, mVertVBOOff);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);

glDrawArrays(GL_TRIANGLES,0,3);





// render rectangle

// First gets the location of that variable in the shader using its name

i32Location = glGetUniformLocation(uiProgramObject, “myPMVMatrix”);

// Then passes the matrix to that variable

glUniformMatrix4fv( i32Location, 1, GL_FALSE, pfIdentity);

// set the vertex vbo

glBindBuffer(GL_ARRAY_BUFFER, mVertVBO);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);



glDrawArrays(GL_TRIANGLES,0,mVertexCount);



glBindBuffer(GL_ARRAY_BUFFER, 0 );

}

gettimeofday(&timeEnd, NULL);

elapsed = timeEnd.tv_sec * 1000LL + timeEnd.tv_usec /1000LL - timeNow.tv_sec * 1000LL - timeNow.tv_usec/1000LL;

printf(“FPS %f = %f ms, time=%lldn”, 2000.000f/(elapsed/1000.f), elapsed/2000.0000f, elapsed);







Here is the output from running it:

glCheckFramebufferStatus ok

glCheckFramebufferStatus ok

PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]

PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]

PVR:(Warning): Kicking render due to frag buffer space [691, /buffers.c]

FPS 78.668922 = 12.711500 ms, time=25423



Anything wrong with it?


#8

Hi,



Looking at your code, you don’t seem to be binding the previous FBO texture before rendering your off-screen objects - binding the previous FBOs texture will signal to the driver that the texture is needed, which will cause any outstanding renders for that FBO need to be kicked.

You’re also redundantly reattaching textures to your FBOs and checking for their status. Once the first bind has been done, the FBO will maintain these attachments until they are removed. As an example, our SDK’s Water demo only attaches textures to the FBOs it using during application start up.



If your timing data is accurate, then your render is running very fast. 12ms for 2000 loops (so, 2000*2=4000 FBO renders in total). This seems suspiciously fast, so I think it would be best to run PVRTune to analyse your render and ensure your timing is correct.

If it is correct, then the driver warnings don’t seem to be impacting performance and aren’t worth worrying about.



Thanks,

Joe


#9

According this instruction:



Step 2:

  1. Bind FBO B
  2. Render an off-screen object with FBO A’s texture bound
  3. Render your scene into FBO B
  4. Bind FBO A
  5. Render an off-screen object with FBO B’s texture bound
  6. Render your scene into FBO A
  7. Repeat steps 1-6



    This code:



    // render to target 2

    glBindFramebuffer(GL_FRAMEBUFFER, fid[1]);

    // Attach the texture as a color buffer.

    glFramebufferTexture2D(

    GL_FRAMEBUFFER,

    GL_COLOR_ATTACHMENT0,

    GL_TEXTURE_2D, texobj[0],0);




    Binds FBO B (fid[1)] and attaches texture A (texobj[0]) to FBO B. Then the offscreen triangle is drawn:



    // render offscreen obj

    glBindBuffer(GL_ARRAY_BUFFER, mVertVBOOff);

    glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);

    glEnableVertexAttribArray(VERTEX_ARRAY);

    glDrawArrays(GL_TRIANGLES,0,3);




    The above code corresponds to Step2’s 1 and 2. It does have texture A attached to FBO B and FBO B is bound as the current rendering target. Is it wrong?



    I will get the PVRTune running to report back. Is there any specific item I could look to see if there is anything suspicious? Thanks.

#10

You don’t need to change the FBO texture attachments - what you should do is render an offscreen triangle that is textured with the output of the previous FBO render. The aim here is to ensure the texture for each FBO is referenced by a draw call in the next render.



PVRTune’s graph area shows TA (vertex processing) and 3D (fragment processing) timing data. You can use this timing data to understand the cost of your render.



Thanks,

Joe


#11

When rendering an offscreen triangle, it would be rendered to either FBO A or FBO B, and correspondingly to the attached texture. What do you mean by “render an offscreen triangle that is textured with the output of previous FBO render”? The target texture (texobj) is setup and attached to its corresponding FBO. How can an offscreen object be “textured with the output of previous FBO”?



In other words, FBO A has texobj[0] attached, FBO B has texobj[1] attached. I can only bind FBO A and render to its color buffer that is corresponding to texobj[0], or bind FBO B and render to texobj[1]. I don’t see when FBO A is bound with texobj[0] attached, one can render to texobj[1], as texobj[1] is not an attached texture for FBO A.


#12

The purpose of attaching a texture to an FBO is so that the colour data can be applied to subsequent draw calls. Our RenderToTexture SDK Example shows how this can be done.



Thanks,

Joe


#13

I read the RenderToTexture example and understand what it is doing. It is what I tried to do before too. What I don’t understand is the offscreen rendering part as you described, what do you mean by “render an offscreen triangle that is textured with the output of previous FBO render”?



In the code I provided, I have two target textures attached to the two FBOs. I do not have a source texture to attach to the rectangle. In the RenderToTexture example, it has the source texture rendered to the cube. I used to have a source texture but I did not include in the code I provided. I modified the code to have the source texture too. So the rendering part looks like this now for FBO B:



// render to target 2

glBindFramebuffer(GL_FRAMEBUFFER, fid[1]);



// render offscreen obj

glBindTexture(GL_TEXTURE_2D, mTexture);

glBindBuffer(GL_ARRAY_BUFFER, mVertVBOOff);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_FALSE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);

glDrawArrays(GL_TRIANGLES,0,3);



////// render rectangle

// set the vertex vbo

glBindBuffer(GL_ARRAY_BUFFER, mVertVBO);

glVertexAttribPointer(VERTEX_ARRAY, 3, GL_FLOAT, GL_TRUE, 0, 0);

glEnableVertexAttribArray(VERTEX_ARRAY);



// Bind the source texture

glActiveTexture(GL_TEXTURE0);

glBindTexture(GL_TEXTURE_2D, mTexture);

// Set the sampler texture unit to 0

glUniform1i(gSamplerLoc, 0);



// set the texture coord vbo

glBindBuffer(GL_ARRAY_BUFFER, mTexVBO);

glVertexAttribPointer(VERTEX_TEX_ARRAY, 2, GL_FLOAT, GL_FALSE, 0, 0);

glEnableVertexAttribArray(VERTEX_TEX_ARRAY);



glDrawArrays(GL_TRIANGLES,0,mVertexCount);







Here there are two vertex attributes: VERTEX_ARRAY (0) and VERTEX_TEXC_ARRAY (1). The first one is the vertexes for offscreen triangle and the rectangle, the second one is the vertexes for the source texture.



Essentially the offscreen triangle will not have corresponding texture as it’s off the screen, but the rectangle will have texture mapped to it.



The quoted code implements these:



Step 2:

  1. Bind FBO B
  2. Render an off-screen object with FBO A’s texture bound
  3. Render your scene into FBO B



    Would you please point out if there is anything wrong? Thanks.

#14

Hi Registerme,



Sorry that this thread has gone unanswered for so long - it sort of fell to the bottom of the pile which we’re just getting back on top of (GDC preperation and debrief has given us a huge amount of backlog I’m afraid).



Did you ever get to the bottom of this issue? If not I will attempt to pick up where Joe left off.



Thanks,

Tobias


#15

No I did not. Would appreciate the help.


#16

Ok so just to get up to speed, a couple of questions. If you could clarify on these I should be able to help.



So your two FBOs - do they render with each other’s textures (ignoring the offscreen triangles)? It sounds like FBO B uses the texture from FBO A at least?



Do any of these textures get rendered to the main framebuffer? (Do draw anything to the main framebuffer in fact?)



Thanks,

Tobias


#17

As you can see from my Feb 28 post, for FBO B (fid[1]), the texture from FBO A (mTexture) is rendered with the scene (rectangle). For FBO B (not shown in the post), it renders with texture from FBO B (mTexture1).



I don’t know if this was what you suppose me to do. Actually I don’t quite understand what is FBO A’s texture and FBO B’s texture.


#18

Ok so there’s no render to the main framebuffer then? (The one provided by EGL as default - framebuffer 0)



To clarify: FBO A’s texture = the texture you attached to the framebuffer with a call to glFramebufferTexture2D() for FBO A. The same for FBO B.



So I think what is actually needed here is some sort of flush each frame. Rendering back and forth between two FBOs which don’t output anything can confuse older drivers into assuming they just aren’t needed. So they aren’t rendered until they’re needed or the driver runs out of room to queue them up. In your case, you get the warning because they’ve filled the queue and have to force a flush every now and then.



There are two ways that you can do this. Firstly, rather than just ping-ponging between two FBOs, each frame you could draw something to the main framebuffer using one of the two FBO textures. By doing this and then calling eglSwapBuffers each frame, you should get the render to complete properly without the warning.



Thanks,

Tobias


#19

Back in Feb 25, I posted the code to use glFramebufferTexture2D, but Joe replied stating that this was wrong. Do you see if the code there is what is supposed to be done?



Since I am not interested in getting the data output to the display, I am trying to avoid using the main frame buffer. At certain stage I will need to read the frame back to a buffer in CPU. Having the FBO as render target is more convenient.


#20

So there’s nothing specifically “wrong” with what you’re doing. All that’s happening is that the driver can’t wait any longer before rendering something, so it does. The only negative side effect you’d get from this is that if you rendered to the main screen (if doing so didn’t fix it), you might see some stuttering. If you’re going to do CPU readback (presumably via glReadPixels?) and aren’t interested in real-time performance, then this isn’t such a big deal and you can just ignore it.



If you want to have it work a little more smoothly you can use eglCreateSync and eglClientWaitSync to flush the calls through. using the following code:



//Last Draw Call for each fbo goes here

EGLSyncKHR syncObject = eglCreateSyncKHR(eglGetCurrentDisplay(),EGL_SYNC_FENCE_KHR,NULL);

eglClientWaitSyncKHR(eglGetCurrentDisplay(), syncObject, EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, 0);

//Switch FBO target here.



You will need to query for the EGL_KHR_fence_sync extension using eglQueryString(EGL_EXTENSIONS); and get the functions using eglGetProcAddress() first though.



By doing this you should be able to avoid this warning.



Thanks,

Tobias