Confused with the grammar in this sentence

仲間外れにされ、わすれられていく

What I can understand is: "I am being abandoned, left out, and forgotten."

I'm quite confused at the にされ part. され means to be distance and go away. Does the particle に mean "because" in this sentence? To me, the sentence can be fine without the にされ part. What purpose does it serve here? Or is the guy saying "Because I am being abandoned, so I will go away, and be forgotten"? He does walk away on his own after saying this...

Thanks in advance!

仲間外れ is a noun meaning someone out-of the group.
仲間外れにする　= to leave someone out, to make someone out-of-the group
仲間外れにされる　= to be made an outside, to get left out (by someone) (the passive)

So the whole sentence has just two verb phrases, saying,
I have been left out, and will be forgotten.

Not “outside” but I meant to write “outsider”

What might be making in difficult for you to understand is that more commonly
仲間外れにされ、わすれられていく
would be phrased as
仲間外れにされて、わすれられていく, using the -te form of 仲間外れにされる. So with that, it's just a basic [-te form verb] [dictionary form verb] sentence, like レストランに行ってラーメンを食べる (I go to a restaurant and eat ramen).

Using 仲間外れにされ instead of 仲間外れにされて gives the sentence a little more of a literary/dramatic feel. It isn't a super-rare way of speaking, but it's probably not the most common way to speak in a normal, casual conversation.

「仲間外れにされ、わすれられていく
What I can understand is: "I am being abandoned, left out, and forgotten"」

Not to split hairs, but technically that Japanese sentence doesn't include the idea of "abandoned." Granted, you could argue that being left out is always a form of abandonment, but the Japanese text only mentions two concepts: being left out, and being forgotten.