If you’re speaking to the rabbi, you refer to him as Rabbi Ginsburg even addressing him.
If you’re speaking about him to someone else you can call him Rabbi Ginsburg, or if there is more than one rabbi with that name you can refer to him as Rabbi Eliyahu Ginsburg.