I've had it happen a few times where I've dug right into the mountain, created storage, created training area, had my keeper train to level 7 or 8. Made a library and boom, knights attack. No attacking cows or horses or peasants. I've also had a few where I've slaughtered pretty indiscriminately and nothing attacked me. I figured out pretty quick that those peasants could be the trigger, but I've altered my play style and didn't really change anything. I wonder if some other attack (lizardmen or dragon) attacking peasants is triggering it? Or maybe a tree spirit that you've created by chopping too much wood? Just wondering if those attacks could also cause it.