Bullet train reservation during golden week

oh dear.. i just realised my holiday plan is right in the period of japan golden week (1 May to 9 May).

my plan is this:
1) landed in narita airport about 2pm on 1 May 2020 [collect JR pass & do all bullet train booking for reserved seats]
2) travel up to kyoto on 5 May 2020
3) Travel back to narita airport for 6pm flight from Kyoto on 9 May 2020

My qns is:
1) upon my arrival at narita airport, once i collected my JR pass. will i be facing a sold out situation of the reserved seating for bullet train ride from shinagawa to kyoto on 5 May 2020?
2) Similarly, will i have trouble to reserved a seat for my return from kyoto to narita on 9 May. i believe i will need to leave kyoto quite early, in order to reach narita by 4pm?

i will be traveling with a group of 4 adult and 2 toddler.

appreciate any kind local or experience japan travel for advice please.

To reach Narita Airport by 4 p.m. you need to leave Kyoto by noon - not particularly early.
For travel on those days, be flexible with timing - you might not be able to get your first choice train, or travel in the non-reserved cars.
With a rail pass, you will save a small amount of money on the round trip using a 7-day pass, but not lots. Consider not getting a pass and having more train options (Nozomi) between Tokyo and Kyoto and being able to book on-line in advance.

upon my arrival at narita airport, once i collected my JR pass. will i be facing a sold out situation of the reserved seating for bullet train ride from shinagawa to kyoto on 5 May 2020?

You are traveling in the opposite direction of the worst traffic. I think it is unlikely that all trains will be booked out on May 5, especially the ones in the early morning. Later in the day, it could be difficult to find vacancies.

Similarly, will i have trouble to reserved a seat for my return from kyoto to narita on 9 May. i believe i will need to leave kyoto quite early, in order to reach narita by 4pm?

May 9 is after the end of Golden Week and pretty much just a regular weekend. You will have no problems to make seat reservations if you do so on May 1.

More about Golden Week:
https://www.japan-guide.com/e/e2282.html