The problem you have is the 3-4 lockout switch. This switch is either faulty or mispositioned, or both. My suggestion is to purchase a new switch (it is the same as the reverse light switch), and fit it. But before bringing the gearbox back into place, test that switch 20 times! to ensure that it is working correctly.

That 3-4 lockout switch is located on the front left of the remote control extension on the gearbox. It is the least accessible electrical component in the MGB. To get to it you've got to remove the console and the elliptical plate around the gear lever. Then you must drop the tail end of the gearbox. Remove the driveshaft and loosen the exhaust at the center and rear of the car. Then remove the four bolts that hold the cross member to the frame. The gearbox will fall several inches (nearly 10cm) down, allowing you to reach in, loosen the old switch and fit the new switch. Use lots of grease when fitting the new switch.