Flipping the sprocket so it dished outwards helped, but the chain is still misaligned in the pulley and it simply goes for a few more revolutions before slipping and pulling the chain half-off the sprocket and into the tire again. I tried running the chain without a pulley, but even at maximum tension on the bottom half, it still rubs against the bike frame where the tensioner would be mounted.
The only two things I can think of now are either making an angled pulley somehow, or finding a way to fix the pulley bracket to the frame so that it won't slip at all.