Damon Salvatore doesn’t leave Elena Gilbert.
He didn’t leave her in 4x09, he set her free.
He doesn’t leave her in 5x10, he lets her go, he relieves her from the burden that he thinks he is.
Because he promised he’d never leave her. This is subtle but this is important.
Which is exactly why I’m not worried