So the I've narrowed down as much as I can the server side lag at the start of every round over nearly all maps on the CSS Pop Maps server.
It appears to be server cmd or cmd update rate for 60-90 seconds at the start of every round?
Ironically it doesn't happen during the warm up rounds.

Refer to number 7 in the link below?

Code:

https://developer.valvesoftware.com/wiki/TF2_Network_Graph

So check out the right hand side of my net_graph in the image below , second entry from the top , it should read 66.3 & after about 90 seconds it will climb back up to normal but rego is rather poor until it climbs back up.
Also makes my lerp go yellow during this period too.

Most noticeable as server fills up , it's not so bad with less than 20 players , but it's started up within last month maybe?

This has been happening a lot on Minigames like ill usually have 90 ping but then it spikes and stays like that for 5 minutes and it does this over and over every so often

[M. Bison]

The increased ping occurs when server-side fps plummets. The server-side fps is visible on net_graph 3/4 (it's the "sv:" value).

There's not much we can do except more frequent restarts.

Edit: You can see the FPS over the course of the day on GameMe: http://wls.gameme.com/overview/1 (look near the bottom).