u/Optimal-Emergency-24

I took down lath and plaster from this wall (it was crumbling bad). This is the first time I’ve ever installed drywall (don’t judge the base plaster work).

I noticed the areas in red and deep behind. Should I have left the keys? Did I miss something? The wall is caved in like 1-1.5 inches in.