Ogres do attack adventurers. The reason the knights ran past them might be that they were scared. If you are too powerful, ogres run from adventurers too. But it is possible that ogres only attack player adventurers and not knights etc.
There's a fault in your logic, if the ogres were simply scared of the knights then the knights still would have seen them and killed them, it could be that the knights have a problem with aggroing the player-neutral tribes.
