At worst has 2 different meanings across 2 categories:
Adverb · Prep_phrase
under the worst of conditions
"at worst we'll go to jail"
If the worst possible alternative happens.