I’ve done all these in my life. I have family with a house in the area of Livingston Manor-Parksville.
Railroad museum in Roscoe (across from diner, off Rt. 17 and down the block from the IGA)
Delaware-Ulster Railroad (goes from Arkville to Roxbury). This is a really cool old fashioned railroad ride that goes round-trip.
Stone and Thistle Farm in East Meredith (they have grass-fed eggs and milk; if you keep chalav yisrael, they’ll let you milk the cows yourself)
Sunday the 28th, there’s a Civil War reenactment in Liberty.
There are tons of darling, charming little artisans throughout Sullivan and Delaware counties. The SCVA website has a map and guide to all the artists and their galleries.
There are lots of really nice farmer markets and flea markets to check out, as well. I recommend Silver Heights Farm in Cochecton Center.
I love anything involving nature, arts and crafts, music, antiques, animals, trains, etc. They also have lots of zip lines and historical sites and darling little historical reenactments and things like that.
You must have a car, though. I’d recommend coming on a Sunday and staying until Thursday. When I’d go up to the mountains, that’s what I’d do, so I could either be in the city for Shabbos, or I’d spend Shabbos with families in the bungalow colonies.