Cannot pass Jailor Ogrim

Hello. I began the expansion from the quest gold and then hopped a ship back to the main game as soon as I was able to from the cursed chess set area. Now, when I board a ship to return to "cursed", I begin in the jail again and once I get to jailor ogrim, he has his standard dialog of when you speak to him after leaving the jail. Therefore I cannot get past him. Any ideas on how to get around this?

Not sure there's any way around. Do you have any marked spots in the expansion?

I didn't mark anywhere. And I tried town portal as well to no avail.

Attack the false wall and the one after that, you can get around him