Ivy can establish roots in wood and wood composite surfaces. Once the ivy is removed and the roots dry out, the residue can be mostly scraped off but may need sanded to fully remove it. You will most likely need to paint. Because the climbing roots can penetrate the surface, allowing moisture to enter, you will probalby see them unless you paint. This also re-seals the surface.

You didn't say what kind of wood composite you have (cement?), but the cleaning and painting will not harm it. Don't let it grow back up the wall.