JR rail pass

Hello,

I will be traveling in Japan from April 19-May 4. I have been trying to figure out if a JR rail pass is worth it, based on my itinerary. I have been having trouble calculating the fares using the Hyperdia search, as I think (please correct me if I'm wrong) some of the trains listed in the schedules are not JR lines. How can I tell if the train listed would be valid with a JR Rail pass?

Here is my itinerary:

April 19 arrive at Narita spend 1 night in Tokyo
April 20 train to Nara, where I will be staying for 6 days
April 22 train to Osaka for a day trip
April 23 & 24 train to Kyoto for a day trip
April 25 open (maybe a day trip to Himeji or Wakayama?)
April 26 train to Magome
April 27 train from Nagiso Station to Nozawa Onsen
April 28 train from Nozawa Onsen to Yudanaka
April 29 train to Tokyo, where I will stay until May 4
May 4 depart from Narita

Thanks for any advice you might have!

Because your trip stretches over a long time period, I think that regular tickets will come considerably cheaper than a rail pass. Some comments:

April 26 train to Magome

Magome is not served by trains. The closest station is Nakatsugawa, from where ypu have to take the bus for 30 minutes. If you enter "Magome" into Hyperdia, it will refer you to a identically named subway station in Tokyo.

April 27 train from Nagiso Station to Nozawa Onsen

Nozawa Onsen is not accessible by train. The easiest way to access Nozawa is by bus from Nagano Station or "Togari Nozawa Onsen" Station.