Kansai wide area pass

Can you travel around the city of Osaka on a Kansai wide area pass or do you specifically need an icoca card to do this?

Can you travel around the city of Osaka on a Kansai wide area pass

You can use only JR trains, which are somewhat useful, but won't provide you access to all the city's sights. See a map of JR lines in Osaka here:
https://www.japan-guide.com/e/e4006.html#section_orientation

or do you specifically need an icoca card to do this?

No, it is not needed. But an IC card would be useful to ride on the transport not covered by the pass, such as the subway, non-JR trains and buses, as well for making purchases. But instead of an IC card you could also purchase paper tickets or consider day passes or other types of passes.