A rudder issue is more likely to have cased directional instability, not a rapid decent. It could be an autopilot issue. The autopilot can be switched pretty much straight after take off. The autopilot is more than capable of flying the departure, including the ascent. If for instance one of the inputs to the autopilot was compromised, and the pilots are not fully paying attention, you have a recipe for disaster.
One of the main inputs to the autopilot is the pitotstatic system. The pitotstatic system is a series of probes and vents which measure the dynamic and static barometric pressure to determine altitude and speed. If for instance one of these probes was damaged (even bent by a couple of degrees or so), or blocked by a cover (which a ground maintainer forgot to remove) or insect, then the input to the autopilot give incorrect readings and will likely cause the aircraft to behave erratically.
Pitot and Static vent checks are part of the AF/BF checks, and also part of the flight crew walk round checks. In this case, it could possibly a static vent issue, as the speed indications allowed for a successful take off. I could also be water in the tubes, which would essentially give the same problems, or one of the system water drain traps not being sealed properly, and the required sense and leak checks not being carried out. An alert flight crew would spot this kind of fault immediately, and revert to manual control.