I think Defensive mons do badly vs Ape innately, though they have ways around it on certain sets. Toxic Stalling and forcing Ape to use Rest is one way to put it on the back foot, esp since Rage Fist isn't being powered up if they don't attack. Ape has to do a lot of setup to get Rage Fist to usable levels of power if it isn't being hit and if it uses it prematurely, it'll be chipped by Rocky Helmet + Toxic. Even if its powered up, it still might not OHKO Amongus or Corv, meaning it'll need to take another round of Rocky Helmet damage before it can KO them. Chesto Rest sets are also really susceptible to chip damage some from Spikes and SR, so despite Ape having great bulk, it still might be put into revenge killing range against some of the stronger attackers like Roaring Moon. If you chip it down enough, Ape will only be able to secure a 1-1 trade in the worst case scenario, which isn't ideal, but still a great trade considering how easy it is for it to Snowball.
I think part of the problem that makes Ape so annoying is that Pokemon like Corviknight run the U-turn / Roost / Defog / Body Press set which is basically just begging for Annihilape to come in and get free Rage Fist Boost / Setup. Running Brave Bird would the Ape match-up a bit less annoying, though that of course has its own downsides.
Having said all that, I still think Ape is pretty busted and has ways to get around a lot of these issues, particularly w/ support from something like Grimsnarl. I also think it have some unexplored potentional sets given Rage Fist's very easy power-up condition. A sub set seems very brutal for slower attackers to deal w/ for example.