Collective Dynamics And Coherent Diagnostics Of Microbunched Relativistic Electron Beams