Outside of the immunity to Charm it was still a decent use of an action/ready action.
I think you could rule either way on if the zombies/lich would jump off a cliff after the gem, since the spell mentions that they specifically stare at and approach you (not the gem).
So you better jump off and cast feather fall.