How to change column display

This is probably a combination of many things happening. First off when you create the relationship that field will be created automatically as well as an "Add Questions" field for adding a related question.

If you go to the relationship you have created the fields are listed and you are able to delete them if you wish. This will automatically remove them from any reports as well. If you just want them removed from that report then you need to edit that report and remove the column up top. If it is set to the default columns and that is part of them you need to edit that at the default report settings. (Go to your table settings > Reports & Charts > Default Report Settings in top right dropdown.)

What I assume is happening here if it was automatically added to your report is you have the table set to add new fields to the default report and this report is using the default columns. You can adjust that setting in the same place I described above.