I maybe found a solution, but I think that it is a workaround for a bug and shouldn’t be necessary.
If I bind the FBO and do a glReadPixels(0,0,1,1,…) (which costs me about 30-40ms) the problem is gone.
Not sure about the OpenGL spec, but I would assume that glFinish or glFlush should be enough to have valid data in memory.