Connecting VDSL modem with Cat6 cable

if it's a Zyxel VDSL P-870M, then don't do it

Instead use the supplied ethernet cable (Cat5 UTP)
We did install such a modem and connected it to the firewall system (HP rack server) and did install the PPPoE stuff.
The PPP authentication did work, but then the modem stopped responding on the ethernet port.
We did even switch network port on the server, changed the cable against another cat6 cable, connected a serial terminal to the management port etc.

In a last try we did then plug in the supplied yellow cable.... and it worked.

The supplied yellow cable is a cat5 unshielded cable, with only 4 wires connected.
So probably the modem and server did agree to a gigabit connection when all 8 wires where connected, but then the modem miserably failed as soon as data did flow at gigabit speed.
On a 4-wire cable connection, the server and modem did agree to a 100MBit connection and that's probably the speed the modem is able to handle.