So, I would question whether Diavolo is actually a user. While King Crimson is presented to us as the ability to erase time, that isn't really what it does at all. Instad, King Crimson does 2 things: -One, it prevents others from perceiving the "erased" time. The time isn't really erased at all - things that occur during the time still happened, it's just that anyone other than Diavolo can't perceive the time affected.
-Secondly, it allows Diavolo to selectively prevent events occurring during "erased" time from affecting him.
So, in reality, King Crimson doesn't destroy time at all - that's just what we're mistakenly told it does.