No need to remove anything. You have to do something to get internet access, even if you have a modem/wi-fi card installed. You have to connect to the internet, or you have no internet access.
You can still run programs, but for example, you won’t be able send documents unless you print them off. You can still find some software which can be loaded from CD or DVD, assuming you have one installed in the computer. YOu probably can get by fine with an older, used, computer and buying very inexpensive second hand software.
And if you aren’t connected to the internet, you won’t have to worry about computer security.