Tokyo to Shibukawa

Planning a day trip to Shibukawa/Gunma region and I have a question on the train tickets to get there.

I will have an ICOCA card with me (as I'm flying in from Osaka and leaving later from Tokyo). Google suggests that I take the route
1. Shinjuku Station to Omiya Station (Saiko Line) ->
2. Omiya Station to Takasaki Station (Hokuriku Shinkansen) ->
3. Takasaki Station to Shibukawa Staion (Agatsuma Line)

Question is, if I have my ICOCA card with me, can I use it for 1 and 3 and would I only need to purchase the Shinkansen ticket from Omiya to Takasaki? Or would I need to purchase a ticket all the way through?

I won't have a JR pass and it's working out cheaper for only a few days' rail travel.

Just buy tickets for the complete journey.

Yes. You can pay by ICOCA for 1 and 3.

Using the Shinkansen is about 50 min faster but from Shinjuku you can actually also go tge entire way by local trains. And need just one change.