A rant on Qifrey and Olruggio's relationship in the early manga (Spoilers up to ch 93)
Broke:
Olruggio's deal with Qifrey was selfish because he gets to have it easy while Qifrey suffers.
Woke:
Olruggio is about to cry talking to Qifrey in chapter 36 and it isn't actually about the apprentices. Not remembering the deal, he thinks the most precious person in his life - whom he at that point had spent days worrying over, sitting by his sickbed - is acting in ways that indicate he doesn't value him as a friend; which hurts him deeply. Olruggio thinks he's not doing a good enough job being reliable or getting Qifrey to trust him, which is why Qifrey keeps leaving him or not confiding in him (Qifrey mentions repeatedly trying to abandon Olly in ch 93 and there's the story in the anthology too), so he gives him the snugstone as proof to Qifrey that he cares , and starts hanging around at the atelier more. But afterwards Qifrey keeps holding secrets from him.
When checking his glasses and noticing the seal on them, Olruggio would (not remembering having engraved them or that Qifrey's has vision loss, since this memory is tied to his secret) think that Qifrey had gone to someone else to have them fixed despite knowing that Olly was readily available and willing to help. Worst case scenario he would realize he had had his memories tampered with and feel betrayed.
Add to this the fact that each memory wipe probably makes these feelings worse, because each time Olly figures Qifrey out an even longer time has passed; making Olly's perceived inability to understand him much worse ("I've been living with him for 5-10-15-20 years and didn't notice this? What is wrong with me?").
In ch. 9 he's not mad at Qifrey for taking in Coco because it is a violation of the law (Olruggio hates the Knight Moralis as much as Qifrey and will lie to them without issue), but because Qifrey had promised him to drop the search for the Brimcaps and settle down with him and the apprentices. Now Qifrey is both breaking that promise, jeopardizing the life they have built, and is making big decisions behind Olruggio's back. He wants Coco out as a way of protest against Qifrey.
With this in mind, I think the reason Olruggio explains to Jujy that he doesn't want apprentices because he (implicitly) can't trust himself to prioritize them over Qifrey, is because he is realizing he was hypocritical in accusing Qifrey of not keeping his priorities straight after the second test. Having Beldaruit comment that Qifrey has become a better person at the Tower of Tomes while Olruggio is still suspicious of him, and Jujy talking to him about how it shouldn't be hard balancing your responsibilities with helping those you care for in Ch 39, Olly probably took that as another affirmation that he was in the wrong for being mad at Qufrey and wasn't doing enough to understand him. So he confronts him in ch 40, tries to communicate, and is mind wiped for his efforts.
I'm sometimes a little concerned that the switch in tone on Olruggio's character following the ch.40 mind wipe is not character development but something deliberate. Because as Ininia says the "wipe your mind of specific thing" spell doesn't just target a specific moment or memory, but every memory related to said thing. So when Qifrey said he erased everything pertaining to his secret, maybe Olruggio just straight up forgot every grievance he had with Qifrey over Qifrey's secrecy and his own self-loathing which was tied to it? Which is why he suddenly started becoming more open to getting closer to everyone at the atelier? If he had just forgotten the moment he confronted Qifrey the grievances would still be there.
Another thing worth noting is that Qifrey always acts surprised when Olruggio mentions being upset with him. Which I think means that Qifrey isn't realizing the impact his behavior has on Olruggio outside of the memory wipes. He says Olruggio keeps finding out his secret because Olruggio is good at sleuthing, but the fact of the matter is that Olruggio starts confronting him because Qifrey is constantly, unintentionally, committing micro-agressions against him.