The strength of the structure needs to be investigated if you intend to go offroad. The rigours of constant corrugations is well known to break bike carriers
that are not designed specifically for offroad application. In fact, for this reason, single pole carriers are not suitable to mount on the rear of a trailer, instead an A frame bike carrier is recommended to be the only option that provides sufficient strength.
The structure of the bike carrier is critical. When it is left cantilevered out on the tow bar it is subjected to far higher forces than anything else in the vehicle. Bikes, and the rack need to be well engineered and looked after for long trips.
Further, the weight on the rear is exacerbated by the distance from the wheels increasing your tendency to bottom out going into dips, for example.
The bike rack and bikes reduces your angle of departure considerably too in most designs of bike racks which can be an issue even on bitumen roads.